yifeikong/curl-impersonate

An active fork of curl-impersonate with more versions and build targets.

curl-impersonate

Note

This is a (slightly) more active fork of curl-impersonate. Differences include:

1. Encrypted Client Hello(ECH) support introduced in Chrome 119.
2. ZSTD compression support introduced in Chrome 123.
3. X25519Kyber768 curve introduced in Chrome 124.
4. More options for impersonation Akamai http/2 fingerprints, especially for Safari.
5. Upgrade to more recent version of curl, 8.7.1 as of April, 2024.
6. Ability to change extension orders and enable/disable TLS grease.
7. (In progress) Single binary to support both Webkit-based and Gecko-based browsers, i.e. Chrome and Firefox.

A special build of curl that can impersonate the four major browsers: Chrome, Edge, Safari and Firefox(In progress). curl-impersonate is able to perform TLS and HTTP handshakes that are identical to that of a real browser.

curl-impersonate can be used either as a command line tool, similar to the regular curl, or as a library that can be integrated instead of the regular libcurl. See Usage below.

Why?

When you use an HTTP client with a TLS website, it first performs a TLS handshake. The first message of that handshake is called Client Hello. The Client Hello message that most HTTP clients and libraries produce differs drastically from that of a real browser.

If the server uses HTTP/2, then in addition to the TLS handshake there is also an HTTP/2 handshake where various settings are exchanged. The settings that most HTTP clients and libraries use differ as well from those of any real browsers.

For these reasons, some web services use the TLS and HTTP handshakes to fingerprint which client is accessing them, and then present different content for different clients. These methods are known as TLS fingerprinting and HTTP/2 fingerprinting respectively. Their widespread use has led to the web becoming less open, less private and much more restrictive towards specific web clients

With the modified curl in this repository, the TLS and HTTP handshakes look exactly like those of a real browser.

How?

To make this work, curl was patched significantly to resemble a browser. Specifically, The modifications that were needed to make this work:

• Compiling with BoringSSL, Google's TLS library, which is used by Chrome and Safari.
• Modifying the way curl configures various TLS extensions and SSL options.
• Adding support for new TLS extensions.
• Changing the settings that curl uses for its HTTP/2 connections.
• Running curl with some non-default flags, for example --ciphers, --curves and some -H headers.

The resulting curl looks, from a network perspective, identical to a real browser.

Read the full technical description in the blog posts: part a, part b.

Supported browsers

The following browsers can be impersonated.

Browser	Version	Build	OS	Target name	Wrapper script
	99	99.0.4844.51	Windows 10	chrome99	curl_chrome99
	100	100.0.4896.75	Windows 10	chrome100	curl_chrome100
	101	101.0.4951.67	Windows 10	chrome101	curl_chrome101
	104	104.0.5112.81	Windows 10	chrome104	curl_chrome104
	107	107.0.5304.107	Windows 10	chrome107	curl_chrome107
	110	110.0.5481.177	Windows 10	chrome110	curl_chrome110
	116	116.0.5845.180	Windows 10	chrome116	curl_chrome116
	119	119.0.6045.199	macOS Sonoma	chrome119	curl_chrome119
	120	120.0.6099.109	macOS Sonoma	chrome120	curl_chrome120
	123	123.0.6312.124	macOS Sonoma	chrome123	curl_chrome123
	124	124.0.6367.60	macOS Sonoma	chrome124	curl_chrome124
	99	99.0.4844.73	Android 12	chrome99_android	curl_chrome99_android
	99	99.0.1150.30	Windows 10	edge99	curl_edge99
	101	101.0.1210.47	Windows 10	edge101	curl_edge101
	15.3	16612.4.9.1.8	MacOS Big Sur	safari15_3	curl_safari15_3
	15.5	17613.2.7.1.8	MacOS Monterey	safari15_5	curl_safari15_5
	17.0	unclear	MacOS Sonoma	safari17_0	curl_safari17_0
	17.2	unclear	iOS 17.2	safari17_2_ios	curl_safari17_2_ios

Notes:

1. Chromium-based browsers all share the same fingerprints, except for the User-Agent header and sec-ch-ua-platform header. They will not be updated unless this assumption changed. Use your own header if you need to impersonate Edge, Chrome Android etc.
2. The original Safari fingerprints in the upstream fork are not correct.

~~This list is also available in the browsers.json file.()~~Needs to be updated.

Basic usage

For each supported browser there is a wrapper script that launches curl-impersonate with all the needed headers and flags. For example:

curl_chrome123 https://www.wikipedia.org

You can add command line flags and they will be passed on to curl. However, some flags change curl's TLS signature which may cause it to be detected.

Please note that the wrapper scripts use a default set of HTTP headers. If you want to change these headers, you may want to modify the wrapper scripts to fit your own purpose.

See Advanced usage for more options, including using libcurl-impersonate as a library.

Documentation

More documentation is available in the docs/ directory.

Installation

There are two versions of curl-impersonate for technical reasons. The chrome version is used to impersonate Chrome, Edge and Safari.

Pre-compiled binaries

Pre-compiled binaries for Windows, Linux and macOS are available at the GitHub releases page. Before you use them you may need to install zstd and CA certificates:

• Ubuntu - sudo apt install ca-certificates zstd libzstd-dev
• Red Hat/Fedora/CentOS - yum install ca-certificates zstd libzstd-devel
• Archlinux - pacman -S ca-certificates zstd
• macOS - brew install ca-certificates zstd

The pre-compiled binaries contain libcurl-impersonate and a statically compiled curl-impersonate for ease of use.

The pre-compiled Linux binaries are built for Ubuntu systems. On other distributions if you have errors with certificate verification you may have to tell curl where to find the CA certificates. For example:

curl_chrome123 https://www.wikipedia.org --cacert /etc/ssl/certs/ca-bundle.crt

Also make sure to read Notes on Dependencies.

Building from source

See INSTALL.md.

Docker images

Warning

New docker images added in this fork are work in progress.

Docker images based on Alpine Linux and Debian with curl-impersonate compiled and ready to use are available on Docker Hub. The images contain the binary and all the wrapper scripts. Use like the following:

# Chrome version, Alpine Linux
docker pull lwthiker/curl-impersonate:0.5-chrome
docker run --rm lwthiker/curl-impersonate:0.5-chrome curl_chrome110 https://www.wikipedia.org

Distro packages

Warning

This is for the upstream project

AUR packages are available to Archlinux users:

• Pre-compiled package: curl-impersonate-bin, libcurl-impersonate-bin.
• Build from source code: curl-impersonate-chrome, curl-impersonate-firefox.

Advanced usage

libcurl-impersonate

libcurl-impersonate.so is libcurl compiled with the same changes as the command line curl-impersonate.

It has an additional API function:

CURLcode curl_easy_impersonate(struct Curl_easy *data, const char *target,
                               int default_headers);

You can call it with the target names, e.g. chrome123, and it will internally set all the options and headers that are otherwise set by the wrapper scripts. If default_headers is set to 0, the built-in list of HTTP headers will not be set, and the user is expected to provide them instead using the regular CURLOPT_HTTPHEADER libcurl option.

Calling the above function sets the following libcurl options:

• CURLOPT_HTTP_VERSION
• CURLOPT_SSLVERSION,
• CURLOPT_SSL_CIPHER_LIST,
• CURLOPT_SSL_EC_CURVES,
• CURLOPT_SSL_ENABLE_NPN,
• CURLOPT_SSL_ENABLE_ALPN
• CURLOPT_HTTPBASEHEADER, if default_headers is non-zero (this is a non-standard HTTP option created for this project).
• CURLOPT_HTTP2_PSEUDO_HEADERS_ORDER, sets http2 pseudo header order, for exmaple: masp (non-standard HTTP/2 options created for this project).
• CURLOPT_HTTP2_SETTINGS sets the settings frame values, for example 1:65536;3:1000;4:6291456;6:262144 (non-standard HTTP/2 options created for this project).
• CURLOPT_HTTP2_WINDOW_UPDATE sets intial window update value for http2, for example 15663105 (non-standard HTTP/2 options created for this project).
• CURLOPT_SSL_ENABLE_ALPS, CURLOPT_SSL_SIG_HASH_ALGS, CURLOPT_SSL_CERT_COMPRESSION, CURLOPT_SSL_ENABLE_TICKET (non-standard TLS options created for this project).
• CURLOPT_SSL_PERMUTE_EXTENSIONS, whether to permute extensions like Chrome 110+. (non-standard TLS options created for this project).
• CURLOPT_TLS_GREASE, whether to enable the grease behavior. (non-standard TLS options created for this project).
• CURLOPT_TLS_EXTENSION_ORDER, explicit order or TLS extensions, in the format of 0-5-10. (non-standard TLS options created for this project).

Note that if you call curl_easy_setopt() later with one of the above it will override the options set by curl_easy_impersonate().

Using CURL_IMPERSONATE env var

If your application uses libcurl already, you can replace the existing library at runtime with LD_PRELOAD (Linux only). You can then set the CURL_IMPERSONATE env var. For example:

LD_PRELOAD=/path/to/libcurl-impersonate.so CURL_IMPERSONATE=chrome116 my_app

The CURL_IMPERSONATE env var has two effects:

• curl_easy_impersonate() is called automatically for any new curl handle created by curl_easy_init().
• curl_easy_impersonate() is called automatically after any curl_easy_reset() call.

This means that all the options needed for impersonation will be automatically set for any curl handle.

If you need precise control over the HTTP headers, set CURL_IMPERSONATE_HEADERS=no to disable the built-in list of HTTP headers, then set them yourself with curl_easy_setopt(). For example:

LD_PRELOAD=/path/to/libcurl-impersonate.so CURL_IMPERSONATE=chrome116 CURL_IMPERSONATE_HEADERS=no my_app

Note that the LD_PRELOAD method will NOT WORK for curl itself because the curl tool overrides the TLS settings. Use the wrapper scripts instead.

Notes on dependencies

If you intend to copy the self-compiled artifacts to another system, or use the Pre-compiled binaries provided by the project, make sure that all the additional dependencies are met on the target system as well. In particular, see the note about the Firefox version.

Contents

This repository contains these folders:

• chrome - Scripts and patches for building the Chrome version of curl-impersonate.
  • curl_chrome110, curl_chrome124 - Wrapper scripts that launch curl-impersonate with the correct flags.
  • curl-impersonate.patch - The main patch that makes curl use the same TLS extensions as Firefox. Also makes curl compile statically with libnghttp2.
  • boringssl.patch - The boringssl patch that tweaks boringssl behaviors.
• win - Scripts for building the Windows version of curl-impersonate, which is quite different from *nix.
• zigshim - We use the awesome zig toolchain to bring curl-impersonate to more archs on Linux. Special thanks to @bjia56 for making it possible.
• docker - Debian and alpine dockerfiles for this project.

Other files of interest:

• tests/signatures - YAML database of known browser signatures that can be impersonated.

Contributing

If you'd like to help, please check out the open issues in the origional repo and open issues here. You can open a pull request with your changes. Note that some of the upstream issues have been fixed.

This repository contains the build process for curl-impersonate. The actual patches to curl are maintained in a separate repository forked from lwthiker's fork of the upstream curl. The changes are maintained in the impersonate-firefox and impersonate-chrome branches.

You may also need the forked and patched BoringSSL.

Sponsors

Original sponsor info:

Sponsors help keep this project open and maintained. If you wish to become a sponsor, please contact me directly at: lwt at lwthiker dot com.

No one has sponsored this fork.

curl-impersonate's Issues

Enable/disable tls extensions by given option string

Upgrade to curl 8.7.1

Add fake delegated credentials

Release files for arm/aarch are not generated correctly

Some bug must be introduced in v0.6.0b7...v0.6.0b8

Add support for Safari fingerprints in macOS 15 and iOS 18

The http2 fingerprints have changed.

Impersonate Tor Browser

(This result is by someone sent me in PM)

TOR browser

Cipher Suites (11 suites)
    Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301)
    Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303)
    Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
    Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
    Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
    Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)

Second legacy firefox 115esr tor based on it

Cipher Suites (17 suites)
    Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301)
    Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303)
    Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
    Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
    Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
    Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
    Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
    Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)

I noticed that ja3 on the Tor browser on PC and on the Tor browser on Android is the same

Add support for mobile safari/chrome

libcurl.dll x86_64 exception occasionally

Has anyone encountered occasional exceptions with the libcurl.dll library in the Windows x86_64 version? Through debugging and analysis, it was found that in the function void Curl_resolver_cancel(struct Curl_easy *data) { destroy_async_data(&data->conn->resolve_async); }, the data->conn is already null. It seems that the following function is called twice:`/*

• Curl_detach_connection() removes the given transfer from the connection.
• This is the only function that should clear data->conn. This will
• occasionally be called with the data->conn pointer already cleared.
  */
  void Curl_detach_connection(struct Curl_easy *data);`
  The comment within this function also indicates that there may be occasional multiple clearings of data->conn. In actual use, this occasionally triggers exceptions, and there is no 100% reproducible method found. However, after switching to using the c-ares library, the problem has not been observed so far.

Pass curl tests with our patches

As our patches keep growing, it's becoming more and more likely that we will introduce extra bugs. It would make us much more confident if we could run the curl tests after the patches have been applied.

Set MACOSX_DEPLOYMENT_TARGET to be the earliest supported version

The MACOSX_DEPLOYMENT_TARGET env var controls which minimum MacOS version the compilation will target. For arm64, the minimum possible version is always MacOS 11.0 (even if the env var is set to be lower), but for x86_64 the version could be even lower (e.g. 10.15).

Without setting this env var, libcurl may not work on users running an older MacOS than the gh actions worker - e.g. in this issue: jeffreydwalter/arlo#205

Native Node.js addon

I was wondering if it was possible for you to integrate libcurl-impersonate into a native Node.js addon so it works in JS out-of-the-box, without the long and tedious process of building node-libcurl with the impersonate patch.
What I'm suggesting is a fork of node-libcurl because I feel like that would take a lot less work than creating a new thing from scratch. While I do understand there are a couple of forks which do that, they are all unmaintained and largely out of date. This repository seems somewhat alive so I hope it won't be an issue keeping it alive. (An occasional pull from upstream should be all of the maintenance it needs.)
I'm not very familiar with the development ecosystem of c++ and such so I'm afraid I won't be of much help, but logically it shouldn't be that big of a deal, right? Surely we can just replace which curl it uses in it's deps. (I understand it may not be this simple, but I really need a reliable impersonate library in Node. Thanks in advance!)

Build for armv7(?) Android Termux

Tried adding this to build-and-test-make.yml but it errors out when building zlib

          - arch: arm
            target: arm-linux-gnueabihf.2.17
            host: arm-linux-gnueabihf
            capture_interface: eth0
            cflags: "-mfpu=vfp -mfloat-abi=soft -mcpu=cortex-a53"

Add riscv64 target

The goal is to run curl-impersonate binaries and curl_cffi wheels on general purpose RISC-V SBCs.

crypto_test_data.cc build fail

I get the following errors when compiling on Debian 11.
How can I fix it ?
Thanks

# Fix the directory structure so that curl can compile against it.
# See https://everything.curl.dev/source/build/tls/boringssl
mkdir -p lib
ln -sf ../crypto/libcrypto.a lib/libcrypto.a
ln -sf ../ssl/libssl.a lib/libssl.a
cp -Rf ../include .
-- Configuring done
-- Generating done
-- Build files have been written to: /temp/curl-impersonate/build/boringssl/build
[5/747] Generating crypto_test_data.cc
FAILED: crypto_test_data.cc
cd /temp/curl-impersonate/build/boringssl && /usr/bin/go run util/embed_test_data.go -file-list /temp/curl-impersonate/build/boringssl/build/embed_test_data_args.txt > /temp/curl-impersonate/build/boringssl/build/crypto_test_data.cc
# command-line-arguments
util/embed_test_data.go:81:16: undefined: os.ReadFile
util/embed_test_data.go:131:16: undefined: os.ReadFile
note: module requires Go 1.19
[8/747] Generating aesv8-gcm-armv8-apple.S
ninja: build stopped: subcommand failed.
make: *** [Makefile:269: /temp/curl-impersonate/build/boringssl/build/lib/libssl.a] Error 1

Export windows building target

Maybe we should build natively with VC, not mingw?

Compile Linux releases with `zig cc`

The Zig compiler supports building C and C++ code with zig cc and zig c++, since it uses clang internally. With clang, [cross] compiling to x86_64, aarch64, and arm32 is possible with the same toolchain, and even cross compiling to Windows/MacOS if desired. Additionally, Zig allows targeting a specific glibc release, which would make it easier to pin to glibc 2.17 (for manylinux2014 compliance) without using old distro containers.

I've had success in my Portable Python project using Zig to compile all of CPython and its dependencies against glibc 2.17, so I think it should be possible to do the same here.

Chrome 'priority' header

Hello,

I've noticed that the priority header gets sent, however comparing it to browser requests (using webhook.site for example to get the request headers) shows that they do not in fact send it.
According to this article the header itself doesn't get sent.

No clue if that would break stuff in the impersonation level, however I've just bumped into issue with one site with the request headers being too big (http 431) and this might be a reason for such issues with servers that support up to http11 and not http2 and 3 which use the priority header.

Setup TLS details according to `user-agent`

https://github.com/Kikobeats/https-tls?tab=readme-ov-file

Setup TLS details according to user-agent

Could we use that project above to do things more dynamically?

Add support for other commonly used fingerprints like okhttp and winssl

Fix windows build on 8.5.0

It seems that our patch does not work on curl 8.5.0. curl is constantly changing how it compiles on Windows :(

v0.7.0 crashes on Windows 7 SP1

curl-impersonate v0.7.0 crashes when making a request (e.g. if the executable is given URL input) on Windows 7 SP1.

It abruptly aborts with an error message of "This application has requested the Runtime to terminate it in an unusual way."

If the executable is run with the --help flag it does not error.

Here is a side-by-side comparison of upstream curl v8.8.0 vs curl-impersonate v0.7.0:

I can reproduce the issue with both the curl-impersonate 0.7.0 windows .exe and libcurl-impersonate 0.7.0/curl_cffi 0.7.0. See the linked issue below for an example of the crash with curl_cffi:

• yt-dlp/yt-dlp#10426 (comment)

I cannot reproduce this issue with curl_cffi 0.5.10 or the upstream libcurl-impersonate. Both work fine with Windows 7.

---

• System info: fully patched Windows 7 Professional Service Pack 1 Build 7601

• Note: v0.7.0 works fine if VxKex is activated.

Unknown CMake command `openssl_check_symbol_exists` for `USE_ECH` option

I tried build with CMake but got error (Unknown CMake command "openssl_check_symbol_exists".) because of USE_ECH option. If I move ECH option snippet to below openssl_check_symbol_exists macro, it fixes.

Here it is new patch about ECH option snippet to fix the problem.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index a54c2ff..9b23a59 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -671,6 +671,29 @@ if(USE_OPENSSL OR USE_WOLFSSL)
   endif()
 endif()
 
+option(USE_ECH "Enable ECH support" OFF)
+if(USE_ECH)
+  if(USE_OPENSSL OR USE_WOLFSSL)
+    # Be sure that the OpenSSL/wolfSSL library actually supports ECH.
+    if(NOT DEFINED HAVE_ECH)
+      if(USE_OPENSSL AND HAVE_BORINGSSL)
+        openssl_check_symbol_exists(SSL_set1_ech_config_list "openssl/ssl.h" HAVE_ECH)
+      elseif(USE_OPENSSL)
+        openssl_check_symbol_exists(SSL_ech_set1_echconfig "openssl/ech.h" HAVE_ECH)
+      elseif(USE_WOLFSSL)
+        openssl_check_symbol_exists(wolfSSL_CTX_GenerateEchConfig "wolfssl/options.h;wolfssl/ssl.h" HAVE_ECH)
+      endif()
+    endif()
+    if(NOT HAVE_ECH)
+      message(FATAL_ERROR "ECH support missing in OpenSSL/BoringSSL/wolfSSL")
+    else()
+      message("ECH enabled.")
+    endif()
+  else()
+    message(FATAL_ERROR "ECH requires ECH-enablded OpenSSL, BoringSSL or wolfSSL")
+  endif()
+endif()
+
 option(USE_NGHTTP2 "Use nghttp2 library" OFF)
 if(USE_NGHTTP2)
   find_package(NGHTTP2 REQUIRED)

CMake configure command

cmake -GNinja -S curl -B build/curl -DCMAKE_BUILD_TYPE=Debug -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON -DBUILD_STATIC_CURL=ON -DCURL_USE_OPENSSL=ON -DCURL_BROTLI=ON -DCURL_ZSTD=ON -DUSE_ZLIB=ON -DUSE_WIN32_IDN=ON -DUSE_NGHTTP2=ON -DUSE_ECH=ON -DENABLE_WEBSOCKETS=ON -DCMAKE_PREFIX_PATH=packages -DCMAKE_INSTALL_PREFIX=packages

CMake version: cmake version 3.28.0-msvc1
Platform: Windows 10 22H2 - 19045.3693

Recheck boringssl's `OPENSSL_NO_ASM` for Windows build

BoringSSL won't link without this option on Windows, due to missing symbols of the fiat lib.

This option literally means no assembly code at all, but the code says otherwise, it is not clear whether it turns off assembly completely or not. If so, performance can be hurt, if not, maybe we can leave it as is.

Useful links:

1. grpc/grpc#9440
2. https://github.com/mit-plv/fiat-crypto
3. https://boringssl.googlesource.com/boringssl/+/HEAD/BUILDING.md
4. https://stackoverflow.com/questions/26963740/linking-in-assembly-files-with-mingw
5. https://github.com/google/boringssl/blob/master/util/generate_build_files.py
6. mxschmitt/action-tmate#86

Is the x86_64-win32 libcurl release compiled with ca-cert-bundle?

Hi Folks,

Want to start by saying love the project (only just discovered but very cool how you've opened up the curl impersonate even further to make on the fly customization even easier).

I'm having issues with certificate errors.

Is this because the files in the releases section were built without cert bundle or is it likely a miss-configuration on my end.
I wasn't sure if i maybe needed to add "-lboringssl" to my linker settings on compilation (Testing using windows mingw64 with vs code)

Error: 0x23e6c979530SSL peer certificate or SSH remote key was not OK

(This solves it but feels like it defeats the purpose: curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L); )

Edit: the below is a the rough outline of what I'm using in my program, I have other code around it but I believe this is the only curl impersonate relevant parts

void chrome::init()
{
    if (ready == false)
    {
#ifdef _WIN32
        HMODULE curlLib = LoadLibraryW(L"libcurl.dll");
#else
        void *curlLib = dlopen("chrome/libcurl.so", RTLD_NOW | RTLD_GLOBAL);
#endif
        if (curlLib)
        {
            using CurlEasyInit = CURL *(*)();
#ifdef _WIN32
            CurlEasyInit curlEasyInit = (CurlEasyInit)GetProcAddress(curlLib, "curl_easy_init");
#else
            CurlEasyInit curlEasyInit = (CurlEasyInit)dlsym(curlLib, "curl_easy_init");
#endif
            if (curlEasyInit)
            {
                curl = curlEasyInit();
                ready = (curl != nullptr);
            }
        }
    }
}

std::pair<std::string, std::string> chrome::runCurl(const std::string& url, const std::string& method, const std::string& requestData, bool firstrun, long timeout)
{
    std::string responseData;
    std::string error;

    // Initialize libcurl if not already initialized
    if (!curl)
    {
        curl = curl_easy_init();
    }

    if (curl)
    {
        // Set URL
        std::cout << url << std::endl;
        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());

        // Set method (GET or POST)
        if (method == "POST")
        {
            curl_easy_setopt(curl, CURLOPT_POST, 1L);

            if (!requestData.empty())
            {
                curl_easy_setopt(curl, CURLOPT_POSTFIELDS, requestData.c_str());
            }
        }
        else if (method != "GET")
        {
            error = "Invalid method. Only GET and POST supported.";
            return std::make_pair(responseData, error);
        }

        // Set data callback function
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, &responseData);

        curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout);

        // Perform the request
        CURLcode res = curl_easy_perform(curl);

        if (res != CURLE_OK)
        {
            if (res == CURLE_OPERATION_TIMEDOUT)
            {
                error = "Timeout error";
            }
            else
            {
                error = curl_easy_strerror(res);
            }
        }
        else
        {
            error = "Ok";
        }
    }
    else
    {
        error = "Failed to initialize libcurl.";
    }

    std::cout << error << std::endl;
    std::cout << responseData << std::endl;

    return std::make_pair(error, responseData);
}

size_t chrome::WriteCallback(void *contents, size_t size, size_t nmemb, std::string *data)
{
    size_t totalSize = size * nmemb;
    data->append(static_cast<char *>(contents), totalSize);
    return totalSize;
}

Add support for http3

• upgrading to curl 8.5.0
• build with ngtcp2 and nghttp3

Export Alpine building target

[BUG] IPv6 connectivity is broken with v0.6.0

Describe the bug
It is not possible to bind to an IPv6 address while creating
requests.Session(impersonate="chrome99", interface="ipv6_here")
Binding to ipv4 works.
The feature works until v0.6.0, it is broken for both 0.6.0 and 0.6.1
It works on 0.6.0b9

To Reproduce

session = requests.Session(impersonate="chrome99", interface="ipv6_here")`
ip = session.get('https://ifconfig.me/ip').text

Error Output

File "/home/user/Workspace/project/network/NetworkController.py", line 49, in init
self.ip = self.session.get('https://ifconfig.me/ip').text
, File "/home/user/.local/lib/python3.10/site-packages/curl_cffi/requests/session.py", line 834, in request
raise RequestsError(str(e), e.code, rsp) from e
, curl_cffi.requests.errors.RequestsError: Failed to perform, ErrCode: 45, Reason: 'Couldn't bind to 'IP HERE'. This may be a libcurl error, See https://curl.se/libcurl/c/libcurl-errors.html first for more details.

Expected behavior
Requests use the specified ip address for HTTP calls.

Versions

• OS: Ubuntu 22.04, x86_64
• curl_cffi version 0.6.0

Additional context

• I have only tried using Session.

Impersonate firefox using boringssl

Static link

Hi, i builded Dockerfile from /chrome
docker build .

and checked /build/out/curl-impersonate and this file contain two shared libraries libz, and libc.
i added --with-zlib flag in Dockerfile, with downloding and static linked zlib

But how link libc

Add support for zstd, which is available since Chrome 123

https://chromestatus.com/feature/6186023867908096

Reorder the list of impersonate targets as it continues to grow

By selecting a impersonate target, we have to iterate over the list. it makes more sense to put the recent ones up top. Or even better, use a dict-like sturcture.

Support Chrome 120

[BUG] `CIPHER_LIST` on Windows is wrong since in v0.6.x

While using impersonate="safari17_2_ios", I get different JA3 digest results between my Windows and macOS machines (8be0b641abb257fae7b13bcfd2657032 on Mac and a76d766e1e01aa4cfaee1331b1bada3b on Windows).
Unfortunately this issue is triggering cloudflare on the Windows machine while on Mac it works just fine every time.
Just for testing I've tried using different python versions with an older openssl version but that is not the problem as the digest doesn't change.

请求升级curl到8.9.1

我在调用libcurl的curl_multi_socket_action时出现NULL错误。我看到有Issue提到是libcurl自身的BUG，升级到8.9.1版本可能解决这个问题。我已尝试fork仓库自行处理，通过简单的替换libcurl版本号，不能够平滑升级，请帮助我升级libcurl到8.9.1版本。

参考Issues:

• socket_function callbacks for unrelated sockets that are never removed #14280
• 异常持续高 CPU 占用 #1100

Add options to change h2 priority weight and exclusive value

As discussed: #46 (comment)

Add an option to disable BoringSSL's GREASE behavior

Add support for Chrome 124 with X25519Kyber768 (25497)

Chrome 124 was released recently with a new quantum-resistent algorithm called X25519Kyber768.

See also:

• https://chromestatus.com/feature/5257822742249472
• https://tls.peet.ws/api/all

It seems to be in A/B testing as of Feb 2024, and shows up on my Chrome on macOS with M1.

[Question] Does chrome 119/120/123/124 only support macos but not windows?

I would like to know if there is any impact on using sec-ch-platform for windows in these versions.
What are the differences between different platforms?

Fix all the compile warnings

Contribution, new fingerprints, OS, browsers, versions

Hi,

README mentions custom header for Edge or Android. Wrapper scripts metion Wireshark.

Wanted to get your 2 cents on how to contribute to the project, adding more natively supported browsers. Such as chrome120 on Windows, Linux, Android, and different MacOS versions.

I am using libcurl ffi bindings via env vars, so I don't believe creating new wrapper scripts will be usable.

replace legacy firefox support with boringssl emulation

Capture chrome/firefox/edge fingerprints automatically

Using selenium docker images, we should be able to capture chrome's fingerprints automatically, and generate the testing targets for this repository.

Add fingerprints for Edge 120

Hello,

Would it be possible for you to add support for Edge 120 as a supported browser emulation?

Thank you much in advance for your consideration.

Export i686 arch for linux and windows

• #18
• #40

Cross compile MacOS arm64 binaries on GH actions

This should make building new curl-impersonate versions fully self-contained within the repo's CI.

Stretch goal (optional) could be to make universal2 binaries from x86_64 + arm64 binaries.

mem leak

Hello,

possible you will add this fix to your fork
lwthiker#196

Add build for Android NDK

Related:

lexiforest/curl_cffi#248
lexiforest/curl_cffi#74

Add an option to customized extension order

We need to add a BoringSSL function like

SSL_CTX_set_extension_order(SSL_CTX ctx, const char* order);

similar to what SSL_CTX_set_permute_extensions does.

Automatically capture Safari fingerprints

Add fake record_size_limit option