bogdanfinn/tls-client

net/http.Client like HTTP Client with options to select specific client TLS Fingerprints to use for requests.

Hi there 👋

tls-client's Issues

remote error: tls: unexpected message

It seems to work at pretty much every site, but when you make a simple GET request to google.com, it receives the error remote error: tls: unexpected message

It goes off in the utls package at line 698 [link to code]

Using the normal Chrome 105 client, Header and Headerorder perfectly set.

Thanks for helping

[Bug]: Need to follow Cookies' domain rules

TLS client version

v1.6.0

System information

Windows 10,
Linux Ubuntu

Issue description

Version: 1.6.0

I am using Python_TLS_Client and found this bug.

Python codes that used this project for requests:
`session = tls_client.Session(
client_identifier="chrome107",
random_tls_extension_order=True
)

session.get('https://httpbin.org/cookies/set/testcookie/12345')
session.get('https://httpbin.org/cookies/set/abc/67890')
print(session.cookies.get_dict())
session.cookies.set("test123", "test", domain="example.org")
res = session.get("https://httpbin.org/headers", proxy="http://localhost:8888")
print(res.text)`

Results from Fiddler inspection:
`Request sent 41 bytes of Cookie data:

testcookie=12345
abc=67890
test123=test

`

The correct result should be:

`Request sent X bytes of Cookie data:

testcookie=12345
abc=67890

`

The 'test123' cookie shouldn't be included in the cookies as the domain was different.

Steps to reproduce / Code Sample

1. Set a cookie without domain
2. Send a request and verify the cookie was in the request
3. Set a cookie with a domain that IS different than the url
4. Send a request and verify the first cookie was in the request, but not the second one as domain was different.

Callback?

I am trying to use the client in nodejs and with the examples you set in cffi_dist/example_node/index_memory.js i am using exactly like this however instead of await i want to callback more functions rather then await which is causing bottleneck in the application. Is there a support for callback?

get response error with long url(redirect)

Hello,
I tried to get response by using tls-client.
Request url is "https://www.mistore.jp/shopping/login".
when access this url, request is redirected.
redirect url is long like this.

https://login.api.mistore.jp/auth/realms/BB/protocol/openid-connect/auth?client_id=VS-API&redirect_uri=https://www.mistore.jp/on/demandware.store/Sites-seamless-Site/ja_JP/Login-OAuthReentryRX&response_type=code&state=1529753980&scope=openid&registration_uri=https%3A%2F%2Flogin.api.mistore.jp%2Fauth%2Frealms%2FBB%2Fprotocol%2Fopenid-connect%2Fauth%3Fclient_id%3DVS-API%26kc_action%3DEXT_ENTRY_CODE%26admsCd%3D2%26response_type%3Dcode%26scope%3Dopenid%26redirect_uri%3Dhttps%253A%252F%252Fwww.mipoint.jp%252Fmypage%252Fim%252Finfo%252Frp_member%252Fbpfregistpurchase%253FreturnUrl%253D%2526nextUrl%253Dhttps%253A%252F%252Fwww.mistore.jp%252Fshopping%252Fhome%253Frid%253D2714ec78a36f4acaba808373a39d822d%2526newMemberClass%253DWEB_PURCHASE_MEMBER

when redirect with this url, it seems to get wrong response.
please let me know what is a problem.
if try with standard http client, there is no problem.

[Bug]: undefined: tls.UtlsExtendedMasterSecretExtension

TLS client version

1.6.1

Issue description

Steps to reproduce / Code Sample

// your example code
package main

import (
	"fmt"
	"io"
	"log"

	http "github.com/bogdanfinn/fhttp"
	tls_client "github.com/bogdanfinn/tls-client"
	"github.com/bogdanfinn/tls-client/profiles"
)

func main() {
	jar := tls_client.NewCookieJar()
	options := []tls_client.HttpClientOption{
		tls_client.WithTimeoutSeconds(30),
		tls_client.WithClientProfile(profiles.Chrome_105),
		tls_client.WithNotFollowRedirects(),
		tls_client.WithCookieJar(jar), // create cookieJar instance and pass it as argument
	}

	client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), options...)
	if err != nil {
		log.Println(err)
		return
	}

	req, err := http.NewRequest(http.MethodGet, "https://tls.peet.ws/api/all", nil)
	if err != nil {
		log.Println(err)
		return
	}

	req.Header = http.Header{
		"accept":          {"*/*"},
		"accept-language": {"de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"},
		"user-agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36"},
		http.HeaderOrderKey: {
			"accept",
			"accept-language",
			"user-agent",
		},
	}

	resp, err := client.Do(req)
	if err != nil {
		log.Println(err)
		return
	}

	defer resp.Body.Close()

	log.Println(fmt.Sprintf("status code: %d", resp.StatusCode))

	readBytes, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Println(err)
		return
	}

	log.Println(string(readBytes))
}

[Bug]: Timeout Broken / Stream does not end when using StreamOutputPath option

TLS client version

v1.4.0

System information

tls-client-windows-64.dll

Issue description

When set StreamOutputPath Option timeoutSeconds or WithTimeoutMilliseconds do not work correctly

the stream connection does not end after the set timeout and causing a lock

without StreamOutputPath i am able to stream and timeout is working fine/closing connection at timeout

Steps to reproduce / Code Sample

set StreamOutputPath

Segmentation fault on

In Python 3.10.8 on Mac (Catalina 10.15.7) I obtain following error:

print("Loading Library...")
library = ctypes.cdll.LoadLibrary(f'{root_dir}/dependencies/tls-client-darwin-amd64-1.0.0.dylib')
print("...loaded!")

causes an error:
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

Seems to be an issue with Calatina, as others report no problems under BigSur.

Concurrent operation error

Running requests under 100 concurrent conditions often results in errors。How to solve it

runtime error: invalid memory address or nil pointer dereference

tsl_client transport not has timeout

mabe hangup to

#	0xc17228	internal/poll.runtime_pollWait+0x88					C:/Users/mpc/sdk/go1.20.1/src/runtime/netpoll.go:306
#	0xcad1b1	internal/poll.(*pollDesc).wait+0x31					C:/Users/mpc/sdk/go1.20.1/src/internal/poll/fd_poll_runtime.go:84
#	0xcae836	internal/poll.execIO+0xf6						C:/Users/mpc/sdk/go1.20.1/src/internal/poll/fd_windows.go:175
#	0xcaf537	internal/poll.(*FD).Read+0x2b7						C:/Users/mpc/sdk/go1.20.1/src/internal/poll/fd_windows.go:436
#	0xe464c8	net.(*netFD).Read+0x28							C:/Users/mpc/sdk/go1.20.1/src/net/fd_posix.go:55
#	0xe5d024	net.(*conn).Read+0x44							C:/Users/mpc/sdk/go1.20.1/src/net/net.go:183
#	0x164b2bc	github.com/bogdanfinn/utls.(*atLeastReader).Read+0x3c			C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:791
#	0xc6b217	bytes.(*Buffer).ReadFrom+0x97						C:/Users/mpc/sdk/go1.20.1/src/bytes/buffer.go:202
#	0x164b4a4	github.com/bogdanfinn/utls.(*Conn).readFromUntil+0xe4			C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:813
#	0x1648995	github.com/bogdanfinn/utls.(*Conn).readRecordOrCCS+0x115		C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:620
#	0x164c70c	github.com/bogdanfinn/utls.(*Conn).readRecord+0x6c			C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:586
#	0x164c70d	github.com/bogdanfinn/utls.(*Conn).readHandshake+0x6d			C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:1023
#	0x1676cd8	github.com/bogdanfinn/utls.(*UConn).clientHandshake+0x478		C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/u_conn.go:430
#	0x1675af1	github.com/bogdanfinn/utls.(*UConn).handshakeContext+0x371		C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/u_conn.go:284
#	0x17244a4	github.com/bogdanfinn/utls.(*UConn).HandshakeContext+0x5c4		C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/u_conn.go:219
#	0x1724488	github.com/bogdanfinn/utls.(*UConn).Handshake+0x5a8			C:/Users/mpc/go/pkg/mod/github.com/bogdanfinn/[email protected]/u_conn.go:206
#	0x17244a5	github.com/bogdanfinn/tls-client.(*roundTripper).dialTLS+0x5c5		

DisableCompression is ignored in http2 requests

When retrieving a resource via http2, fhttp library always decompresses the body, even if DisableCompression is set to true. According to the option description this should not happen:

	// DisableCompression, if true, prevents the Transport from
	// requesting compression with an "Accept-Encoding: gzip"
	// request header when the Request contains no existing
	// Accept-Encoding value. If the Transport requests gzip on
	// its own and gets a gzipped response, it's transparently
	// decoded in the Response.Body. However, if the user
	// explicitly requested gzip it is not automatically
	// uncompressed.

I looked into parts which read body in http1 and http2 and compared them. http1 has addedGzip flag which is checked before calling DecompressBody function (https://github.com/bogdanfinn/fhttp/blob/959a7f72d1ca6d6319e1ed82a26eea431658d3b3/transport.go#L2190). In http2 we have requestedGzip flag, which acts in similar way. However, requestedGzip is not checked (https://github.com/bogdanfinn/fhttp/blob/master/http2/transport.go#L2259).

The following change seem to solve the issue bogdanfinn/fhttp@8237b3b

CookieJar Option

Excuse me, there is no cookiejar option for tls-client?

Rubbish body if the server has returned data encoded using gzip

In some cases (when the "Accept-Encoding" header is used) rubbish is returned in the response body instead of readable data.

Here is a Python example:

import ctypes
import json
import os.path
from io import BytesIO

SCRIPT_FOLDER = os.path.dirname(__file__)

# load the tls-client shared package for your OS you are currently running your python script (i'm running on mac)
library = ctypes.cdll.LoadLibrary(
    os.path.join(SCRIPT_FOLDER, 'tls-client-windows-64-1.3.8.dll')
)

# extract the exposed request function from the shared package
request = library.request
request.argtypes = [ctypes.c_char_p]
request.restype = ctypes.c_char_p

def main():
    requestPayload = {
        "tlsClientIdentifier": "chrome_107",
        "followRedirects": True,
        "insecureSkipVerify": False,
        "withoutCookieJar": False,
        "withDefaultCookieJar": False,
        "isByteRequest": False,
        "additionalDecode": "",
        "forceHttp1": False,
        "withDebug": False,
        "catchPanics": False,
        "withRandomTLSExtensionOrder": False,
        "session": 0,
        "timeoutSeconds": 30,
        "timeoutMilliseconds": 0,
        "certificatePinningHosts": {},
        "proxyUrl": "",
        "isRotatingProxy": False,
        "headers": {
            "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
            "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36",
            "accept-encoding": "gzip, deflate, br",
            "accept-language": "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
        },
        "headerOrder": [
            "user-agent",
            "accept",
            "accept-encoding",
            "accept-language"
        ],
        "requestUrl": "https://outlook.live.com/owa/?nlp=1",
        "requestMethod": "GET",
        "requestBody": "",
        "requestCookies": []
    }
    response = request(json.dumps(requestPayload).encode('utf-8'))
    response_object = json.load(BytesIO(ctypes.string_at(response)))

    print('Content-Encoding: ', response_object['headers'].get('Content-Encoding'))
    print('Body: ', response_object['body'][:40], '...')


if __name__ ==  '__main__':
    main()

The script above returns the following:

Content-Encoding:  ['gzip']
Body:  ���\000\000\000\000\000�\000��[o�J�'�����v�-�)Y�_�0�n��K�" ...

And everything would work fine if you removed "accept-encoding" from the headers:

Content-Encoding:  None
Body:  <!-- Copyright (C) Microsoft Corporation ...

browsers doesnt reproduce updated tls specs

Issue:

When using the chrome browser and request tls.peet.ws/api/all, it shows me this ja3 string

(771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-10-11-13-16-23-43-51-65281-45-21,29-23-24,0)

and in the extensions, this is included:

{
        "name": "padding (21)",
        "padding_data_length": 514
      }

now when using that same ja3 and building a custom Client with it, the same site shows me everything perfectly fine, except it doesnt show me the padding Extension, its just gone in the extensions list and the shown ja3 string

(771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-10-11-13-16-23-43-51-65281-45,29-23-24,0)

I noticed that when I add other extensions like i.e. 17513 or 27, it then also shows the padding extension again. But then again, its not the same as my actual browser :/

[Feature Request]: Replace the utls library `refraction-networking/utls`

Describe the feature / enhancement and how it would improve things

Hi,
I can see the reasoning why you maintain your own utls library but as time changes since you started this excellent project refraction-networking/utls project is also grown and become a very solid library, it is actively maintained and used by many other great projects. Wouldn't it be better to use this library instead of carrying the burden of yours? So you can have more time to focus the client part.

Describe how your proposal will work, with code and/or pseudo-code

Since they are both derived from tls libary it can be used as a drop-in replacement.

Redirect not working correctly.

First of all:
"tls_client.WithNotFollowRedirects()," - somehow has the effect, that it follows redirects.

Furthermore when I'm trying to get "https://httpbin.org/cookies/set/name/value" it's stuck in an infinitive redirect loop.
This is because its not getting the new URL from the "Location"-header but instead just the old URL.

What is the license of this software?

What is the license of this software?

I could not find a LICENSE, COPYING or similar file.

I want to write an Arch linux package for this, and so I need to know the licensing terms.

Regards!

How can I add/change/delete session's headers

As I wrote in the title, I wanna change the headers after defining session. Is this possible? This is my code:

package main

import (
	"fmt"
	"io"
	"log"
	"net/url"

	http "github.com/bogdanfinn/fhttp"
	tls_client "github.com/bogdanfinn/tls-client"
)

func CreateSession() tls_client.HttpClient {
	baseHeader := http.Header{
		"authority":          {"hcaptcha.com"},
		"accept":             {"application/json"},
		"accept-language":    {"en-US,en;q=0.9"},
		"content-length":     {"0"},
		"content-type":       {"text/plain"},
		"origin":             {"https://newassets.hcaptcha.com"},
		"referer":            {"https://newassets.hcaptcha.com/"},
		"sec-ch-ua":          {`"Chromium";v="112", "Not)A;Brand";v="24", "Google Chrome";v="112"`},
		"sec-ch-ua-mobile":   {"?0"},
		"sec-ch-ua-platform": {`"Windows"`},
		"sec-fetch-dest":     {"empty"},
		"sec-fetch-mode":     {"cors"},
		"sec-fetch-site":     {"same-site"},
		"user-agent":         {"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"},
		http.HeaderOrderKey: {
			"authority",
			"accept",
			"accept-language",
			"content-length",
			"content-type",
			"origin",
			"referer",
			"sec-ch-ua",
			"sec-ch-ua-mobile",
			"sec-ch-ua-platform",
			"content-length",
			"sec-fetch-dest",
			"sec-fetch-mode",
			"sec-fetch-site",
			"user-agent",
		},
	}

	options := []tls_client.HttpClientOption{
		tls_client.WithTimeoutSeconds(30),
		tls_client.WithClientProfile(tls_client.Chrome_112),
		tls_client.WithNotFollowRedirects(),
		tls_client.WithDefaultHeaders(baseHeader),
	}

	client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), options...)
	if err != nil {
		log.Println(err)
	}

	return client

}
func main() {
	session := CreateSession()

	params := url.Values{}

	params.Add("v", "19148ad")
	params.Add("host", "accounts.hcaptcha.com")
	params.Add("sitekey", "a5f74b19-9e45-40e0-b45d-47ff91b7a6c2")
	params.Add("sc", "1")
	params.Add("swa", "1")
	params.Add("spst", "1")

	query := params.Encode()

	req, err := http.NewRequest(http.MethodPost, "https://httpbin.org/post?"+query, nil)

	if err != nil {
		log.Fatal(err)
		return
	}

	resp, err := session.Do(req)

	if err != nil {
		log.Fatal(err)
		return
	}

	defer resp.Body.Close()

	bodyText, err := io.ReadAll(resp.Body)

	if err != nil {
		log.Fatal(err)
		return
	}

	fmt.Printf("%s\n", bodyText)

}

For example, I wanna change only the content-type value, how can I do that?

The repository size is too large

You should consider reducing the repository size by some way, this isn't normal

Priority frames problem

Theres two things, first of all

{
        "frame_type": "WINDOW_UPDATE",
        "length": 4,
        "increment": 12517377
      }

this right here for firefox, on the normal browser its the second sent frame
when using the firefox client and doing the same thing, then its the 8th frame sent

the second thing is, in the HEADERS frame sent, the priority on browser is this:

"priority": {
        "weigth": 42,
        "depends_on": 13,
        "exclusive": 0
      }

on the client it is this:

"priority": {
          "weight": 256,
          "depends_on": 0,
          "exclusive": 1
        }

and idk where I could change that

[Bug]: Memory issues with nodejs ffi-napi

TLS client version

v.1.3.2

System information

node:current-bullseye image
vCPUs based on AMD

Issue description

I run a node js application in a docker environment using current:bullseye image. The application creates multiple forks of itself. Every fork has one TLSClient (TLS client based on the typescript client) active and has some proxy rotation as I read in the comments every proxy change causes a new TLSClient to be created so I make sure to call destorySessionasync and destroyAllAsync before moving to a new proxy URL but the memory consumption always increases until we get out of memory.

Steps to reproduce / Code Sample

This is part of my proxy rotation and session destruction code :

        await this.proxyManager.initProxies(this.proxyPools);
        this.options = this.proxyManager.randomizeOptions(); // this generates the new sessionId
        this.options.tlsPayload = this.proxyManager.getTlsClientPayload(); // this will assign the new sessionId
        if (this.currentSession) { // as the proxy URL change destroy the previous session
            await this.tlsClient.destroySession({sessionId: this.currentSession});
        }
        await this.tlsClient.destroyAllAsync(); // destroy everyting
        
        this.currentSession = this.options.sessionId; // keep the sessionId for the next destruction

carrying forward task jar handling from fhttp

there seems to be no way to call httpclient.CookieJar object in this client despite the older fhttp code allowed for this. can you possibly reintroduce it or add a way to share cookie jars between clients?

[Feature request] setCiphers()

So some websites use custom ciphers to prevent botting and it would be nice to easily set them with a function so please add this!

Please support aarch64/arm64 🙏

title

[Bug]: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.

proxy ip

Hello, the proxy is set when forwarding. Why can't I get the proxy's IP

Feature Request - add/export addCookieToSession method

Hi!

I'd love to see functionality whereby I can simply call a method to add cookies to the internal cookie jar. Yes I know I can send additional cookies in the requestCookies array, but better functionality (IMO) would be to have a function that I can call to add the additional cookies, without having to send them as part of the request.

[Feature Request]: How to support streaming response?

Describe the feature / enhancement and how it would improve things

How to support streaming response?

Describe how your proposal will work, with code and/or pseudo-code

How to support streaming response?

python async

how to access python async code???

Cookies Automatically Setting

When I'm making multiple request with the client, the cookies are automatically setting, and I'm not sure how to delete these auto set cookies. Also how to maintain cookie order if this is happening?

net/http: invalid byte '"' in Cookie.Value; dropping invalid bytes

Error: net/http: invalid byte '"' in Cookie.Value; dropping invalid bytes
Cookie in server response: set-cookie: gur="UDJ\05457038934595\054170621805"; Domain=.......

[Feature Request]: socks5h supports.

Describe the feature / enhancement and how it would improve things

werner@X10DAi:~/Public/repo/github.com/acheong08/ChatGPT-to-API.git$ http_proxy=socks5h://127.0.0.1:18890 SERVER_PORT=18080 ./freechatgpt
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /ping                     --> main.main.func1 (4 handlers)
[GIN-debug] PATCH  /admin/password           --> main.passwordHandler (5 handlers)
[GIN-debug] PATCH  /admin/tokens             --> main.tokensHandler (5 handlers)
[GIN-debug] PATCH  /admin/puid               --> main.puidHandler (5 handlers)
[GIN-debug] PATCH  /admin/openai             --> main.openaiHandler (5 handlers)
[GIN-debug] OPTIONS /v1/chat/completions      --> main.optionsHandler (4 handlers)
[GIN-debug] POST   /v1/chat/completions      --> main.nightmare (5 handlers)
[GIN-debug] GET    /v1/engines               --> main.engines_handler (5 handlers)
[GIN-debug] GET    /v1/models                --> main.engines_handler (5 handlers)
2023/08/02 14:26:28 17121 127.0.0.1:18080
[GIN] 2023/08/02 - 14:26:41 | 500 |  791.445879ms |       127.0.0.1 | POST     "/v1/chat/completions"

werner@X10DAi:~/Public/repo/github.com/acheong08/ChatGPT-to-API.git$ curl -v http://127.0.0.1:18080/v1/chat/completions -d '{"messages": [{"role": "user", "content": "Say this is a test!"}],"stream": true}'
*   Trying 127.0.0.1:18080...
* Connected to 127.0.0.1 (127.0.0.1) port 18080 (#0)
> POST /v1/chat/completions HTTP/1.1
> Host: 127.0.0.1:18080
> User-Agent: curl/7.81.0
> Accept: */*
> Content-Length: 81
> Content-Type: application/x-www-form-urlencoded
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Access-Control-Allow-Headers: *
< Access-Control-Allow-Methods: *
< Access-Control-Allow-Origin: *
< Content-Type: application/json; charset=utf-8
< Date: Wed, 02 Aug 2023 06:26:41 GMT
< Content-Length: 33
< 
* Connection #0 to host 127.0.0.1 left intact
{"error":"error sending request"}

Describe how your proposal will work, with code and/or pseudo-code

This is because tls-client does not support socks5h currently.

See acheong08/ChatGPT-to-API#104 (comment) for the related comment.

cannot load on linux

i'm trying to use the pre-built files from dist on linux.
const ffi = require('ffi-napi');
const tlsClientLibrary = ffi.Library(goPath, { 'request': ['string', ['string']] });

when i try to load the lib i get this error:

"Dynamic Symbol Retrieval Error: /usr/lib/libc.so.6: undefined symbol: request"

I'm on manjaro arch linux

[Bug]: failed to do request

TLS client version

v1.3.11

System information

Windows Server 2019 64 bit
Intel(R) Xenon(R) E-2136 CPU @ 3.30GHz
Nodejs v16.17

failed to do request: Post "https://xxxxxxxx.xxx/": unexpected EOF

I get the following error very often when using proxy. this issue sometimes causes the code to freeze without error.

After doing some research, I came across the following topic. is there a parameter that i can apply for tls client? And does it solve the problem?

https://stackoverflow.com/a/19006050

Issue description

I get this error very often when using proxy.

failed to do request: Get "https://www.example.com/": EOF

Steps to reproduce / Code Sample

let tlsClientLibrary = ffi.Library('./tls-client-windows-64-1.3.11.dll', {
    'request': ['string', ['string']],
    'getCookiesFromSession': ['string', ['string']],
    'addCookiesToSession': ['string', ['string']],
    'freeMemory': ["void", ['string']],
    'destroyAll': ['string', []],
    'destroySession': ['string', ['string']]
});

const requestPayloadDefault = {
    "tlsClientIdentifier": "chrome_105",
    "followRedirects": false,
    "insecureSkipVerify": false,
    "withoutCookieJar": false,
    "withDefaultCookieJar": false,
    "isByteRequest": false,
    "catchPanics": false,
    "additionalDecode": null,
    "withDebug": false,
    "forceHttp1": false,
    "withRandomTLSExtensionOrder": true,
    "timeoutSeconds": 30,
    "timeoutMilliseconds": 0,
    "sessionId": "my-session-id",
    "proxyUrl": "",
    "isRotatingProxy": true,
    "certificatePinningHosts": {},
    "headers": {},
    "headerOrder": [
        "accept",
        "user-agent",
        "accept-encoding",
        "accept-language"
    ],
    "requestUrl": "",
    "requestMethod": "GET",
    "requestBody": "",
    "requestCookies": []
}

let requestPayload = JSON.parse(JSON.stringify(requestPayloadDefault));
    requestPayload.requestUrl = `https://www.xxx.com/`;
    requestPayload.requestMethod = "GET";
    requestPayload.proxyUrl = proxy;
    requestPayload.sessionId = makeid(32);
    requestPayload.tlsClientIdentifier = options.tlsClientIdentifier;
    requestPayload.headers = {
        'User-Agent': headers['user-agent'],
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
        'Accept-Language': headers['accept-language'],
        'Accept-Encoding': 'gzip, deflate, br',
        'Cookie': headers['cookie'],
        'Upgrade-Insecure-Requests': '1',
        'Sec-Fetch-Dest': 'document',
        'Sec-Fetch-Mode': 'navigate',
        'Sec-Fetch-Site': 'cross-site',
        'Sec-Fetch-User': '?1',
        'Te': 'trailers',
        'Pragma': 'no-cache',
        'Cache-Control': 'no-cache',
        'Connection': 'close',
    }

    return await new Promise((resolve, reject) => {
        tlsClientLibrary.request.async(JSON.stringify(requestPayload), (err, response) => {
            resolve(JSON.parse(response))
        });
    }).then((response) => {
        return response;
    });

[Feature request] split out prebuilt binaries from this repository to some external "blob storage".

Suggestion:

Split source code repository and distribution of prebuilt library binaries.

So that a repository clone is smaller, and users are not confused whether prebuilt binaries are always up to date with souce code.

Having large binaries in version control also is not a good idea.

And prebuilt binaries are available via separate download URL.

Regards!

Cookie Error

When running latest Version for node.js,

When I define "requestCookies" in the Payload like this:

"requestCookies": ["locale=en-ES"]

results in this error:
{"id":"bc043707-4f84-45de-802c-386f5a0c3c08","body":"json: cannot unmarshal string into Go struct field RequestInput.requestCookies of type tls_client_cffi_src.CookieInput","cookies":null,"headers":null,"status":0,"target":"","usedProtocol":""}

Without any defined cookies, it works fine.

invalid response cookies

Any chance of adding a flag or something to disable the messages? they are redundant anyway because, apart from the message, the value is immediately sanitized and passed on. this happens on google and youtube - related to #23

[Bug]: not possible to set transportOptions when using tls-client-windows-64.dll

TLS client version

v1.4.0

System information

Windows 10
Python 3.9

Issue description

Hi

is not possible to set transportOptions when using tls-client-windows-64.dll with python?

in particular im trying to define the IdleConnectionTimeout but nothing i have tried seems to be working.

sometimes i need to pause for a few min between request but need the connection kept alive for the next request. but atm it doesnt seem possible

Steps to reproduce / Code Sample

TransportOptions = {
    "DisableKeepAlives": False,
    "IdleConnTimeout": 0
}

requestPayload = {
    "transportOptions": TransportOptions,
    "tlsClientIdentifier": "chrome_112",
    "followRedirects": False,
    "withoutCookieJar": True,
    "withDebug": False,
    "catchPanics": True,
    "withRandomTLSExtensionOrder": True,
    "timeoutSeconds": 30,
    "sessionId": "my-session-id",
    "proxyUrl": "",
    "isRotatingProxy": False,
    "headers": {
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
    },
    "headerOrder": [
        "accept",
        "user-agent",
        "accept-encoding",
        "accept-language"
    ],
    "requestUrl": "https://microsoft.com",
    "requestMethod": "GET",
    "requestBody": "",
    "requestCookies": []
}

while True:
    response = request(json.dumps(requestPayload).encode('utf-8'))
    response_bytes = ctypes.string_at(response)
    response_string = response_bytes.decode('utf-8')
    response_object = json.loads(response_string)
	sleep(120)

[FEATURE REQUEST] Support arm32

.

[Bug]: Running a python script that imports tls-client causes error

TLS client version

v0.2.1

System information

Raspbian (Debian GNU/Linux)
BCM2835 (4) @ 1.800GHz

Issue description

Each time I attempt to run a python script that imports tls-client, I get this error:

OSError: /home/matthew/.local/lib/python3.9/site-packages/tls_client/dependencies/tls-client-arm64.so: cannot open shared object file: No such file or directory

Steps to reproduce / Code Sample

Install tls-client package from pip3
Make a new python script in a Raspbian environment
Import tls-client
Run the python script

[Bug]: This may be a bug related to memory leaks

TLS client version

v1.4

System information

Window 11

Issue description

Hi, brother

When I use multi coroutine testing, the memory keeps increasing! This may be a bug related to memory leaks

Here is my test code:

Steps to reproduce / Code Sample

package main

import (
	"fmt"
	http "github.com/bogdanfinn/fhttp"
	tls_client "github.com/bogdanfinn/tls-client"
	"io"
	"log"
	"time"
)

func main() {
	for i := 0; i < 500; i++ {
		go request()
	}
	time.Sleep(time.Hour)
}
func request() {
	for {
		test()
		time.Sleep(time.Second * 5)
	}

}
func test() {
	jar := tls_client.NewCookieJar()
	options := []tls_client.HttpClientOption{
		tls_client.WithTimeoutSeconds(30),
		tls_client.WithClientProfile(tls_client.Chrome_105),
		tls_client.WithNotFollowRedirects(),
		tls_client.WithCookieJar(jar), // create cookieJar instance and pass it as argument
	}
	client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), options...)
	if err != nil {
		log.Println(err)
		return
	}
	//client.SetProxy("http://127.0.0.1:8888")
	req, err := http.NewRequest(http.MethodGet, "https://www.baidu.com", nil)
	if err != nil {
		log.Println(err)
		return
	}
	req.Header = http.Header{
		"accept":          {"*/*"},
		"accept-language": {"de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"},
		"user-agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36"},
		http.HeaderOrderKey: {
			"accept",
			"accept-language",
			"user-agent",
		},
	}

	resp, err := client.Do(req)
	if err != nil {
		log.Println(err)
		return
	}
	defer resp.Body.Close()
	log.Println(fmt.Sprintf("status code: %d", resp.StatusCode))
	io.ReadAll(resp.Body)

}

[Feature Request]: `certificatePinningHosts` wildcard support

Describe the feature / enhancement and how it would improve things

I was wondering if it's possible to add wildcard support (like OkHttp) to the certificatePinningHosts option.

Describe how your proposal will work, with code and/or pseudo-code

For example:

certificatePinningHosts: {
    "*.example.com": [
        "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=",
    ],
},

This would match all subdomains including: test.example.com, dev.example.com, etc.

[Feature Request]: Provide an option to set default headers (and header orders)

Describe the feature / enhancement and how it would improve things

I didn't find the proper settings to set the headers in a client-wise fashion. (Like in C# HttpClients SetDefaultRequestHeaders) It allows you to set headers for the client's lifetime. Setting a header in a request level will override the default headers. If it is absent it would be nice to have such a feature.

Describe how your proposal will work, with code and/or pseudo-code

options := []tlsClient.HttpClientOption{
		tlsClient.WithTimeoutSeconds(30),
		tlsClient.WithClientProfile(tlsClient.MappedTLSClients[tlsIdentifier]),
		//tlsClient.WithDebug(),
		tlsClient.WithRandomTLSExtensionOrder(),
		tlsClient.WithProxyUrl(proxy),
		tlsClient.WithNotFollowRedirects(),
		tlsClient.WithInsecureSkipVerify(),
		tlsClient.WithCookieJar(jar),
		tlsClient.WithCatchPanics(),
                tlsClient.WithDefaultHeaders(someHeadersHere), // <--
	}

HTTP/2 keep alive timeout

The client closes the TCP connection after 30 seconds (even if I send a request every second). Can this be fixed?
Opening a new TCP connection every time is an extra resource consumption =)

[Bug]: panic: runtime error: invalid memory address or nil pointer dereference

TLS client version

v1.4.0

System information

windiows 10

Issue description

when i use streamOutputPath options

Steps to reproduce / Code Sample

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff936f0fcb4]

goroutine 17 [running, locked to thread]:
github.com/bogdanfinn/tls-client/cffi_src.(*TLSClientError).Error(0x1148c31d745a6199?)
/Users/bogdan/Development/Solem8s/tls-client/cffi_src/types.go:20 +0x14
github.com/bogdanfinn/tls-client/cffi_src.(*TLSClientError).Error(...)
/Users/bogdan/Development/Solem8s/tls-client/cffi_src/types.go:20
main.handleErrorResponse({0xc00001c5d0, 0x24}, 0x1, 0xc00005d270)
/Users/bogdan/Development/Solem8s/tls-client/cffi_dist/main.go:283 +0xcf
main.request(0x0?)
/Users/bogdan/Development/Solem8s/tls-client/cffi_dist/main.go:259 +0x795

raspberry pi support?

need to run some project on raspberry pi with tls and other libraries does not support it. so matbe there's a way to somehow run it? its armv6l and i would really apreciate it if someone helped with it

thx

Is it possible to get RAW HTTP response instead of JSON response

Hi, first I want to say thanks for this awesome project! This project literally open up a door for my Python application to allow it to bypass many TLS Fingerprint websites.

From this Issue: FlorianREGAZ/Python-Tls-Client#32

Currently tls-client sends back JSON data:
{....,"headers":{"Access-Control-Allow-Credentials":["true"],"Access-Control-Allow-Origin":["https://rr6-sn-8qj-i5o6k.gooqlevideo.com"],"Age":["72715"],"Alt-Svc":["h3=\":443\"; ma=86400, h3-29=\":443\"; ma=86400"],"Cache-Control":["max-age=31536000"],"Cf-Cache-Status":["HIT"],"Cf-Ray":["79f028e9cf569e28-SIN"],"Content-Encoding":["br"],"Content-Security-Policy":["default-src 'self';base-uri 'self';block-all-mixed-content;font-src 'self' https: data:;form-action 'self';frame-ancestors 'self';img-src 'self' data:;object-src 'none';script-src 'self';script-src-attr 'none';style-src 'self' https: 'unsafe-inline';upgrade-insecure-requests"],"Content-Type":["text/html; charset=utf-8"],"Cross-Origin-Embedder-Policy":["require-corp"],"Cross-Origin-Opener-Policy":["same-origin"],"Date":["Sat, 25 Feb 2023 11:39:19 GMT"],"Expect-Ct":["max-age=0"],"Origin-Agent-Cluster":["?1"],"Referrer-Policy":["no-referrer"],"Server":["cloudflare"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"Vary":["Accept-Encoding","Origin"],"X-Content-Type-Options":["nosniff"],"X-Dns-Prefetch-Control":["off"],"X-Download-Options":["noopen"],"X-Frame-Options":["SAMEORIGIN"],"X-Permitted-Cross-Domain-Policies":["none"],"X-Powered-By":["centminmod"],"X-Xss-Protection":["0","1; mode=block"]},"cookies":{"__cf_bm":"yxyOVhUi38n1UfQujprACZl5yxXyhYZMYl6boMzKKh0-1677324536-0-AdwLPV4avEeaFZNNLGA7VKzwHM3C2rbm2jL/SPwv1WNoRDaahAKij0hWwlApbK52ErrqT3pGG+pCYWIZlOzkM9o="}}

instead of RAW HTTP data like this:

HTTP/1.1 200 OK
Cache-Control: no-store
Pragma: no-cache
Date: Fri, 24 Feb 2023 18:51:16 GMT
Content-Type: text/html; charset=utf-8
Vary: Accept-Encoding
Vary: Origin
Content-Security-Policy: default-src * 'unsafe-inline' 'unsafe-eval' blob: data:;base-uri 'self';block-all-mixed-content;font-src 'self' https: data:;form-action 'self';frame-ancestors 'self';img-src * 'self' blob: data: 'unsafe-inline' 'unsafe-eval';object-src 'none';script-src * 'unsafe-inline' 'unsafe-eval' blob: data:;script-src-attr 'none';style-src * 'unsafe-inline' 'unsafe-eval' blob: data:;upgrade-insecure-requests;prefetch-src 'none';
Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Opener-Policy: same-origin
X-DNS-Prefetch-Control: off
Expect-CT: max-age=0
X-Frame-Options: SAMEORIGIN
X-Download-Options: noopen
X-Content-Type-Options: nosniff
Origin-Agent-Cluster: ?1
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: no-referrer
X-XSS-Protection: 0
X-XSS-Protection: 1; mode=block
Access-Control-Allow-Origin: https://rr6-sn-8qj-i5o6k.gooqlevideo.com
Access-Control-Allow-Credentials: true
X-Powered-By: centminmod
Old-Cache-Control: max-age=31536000
CF-Cache-Status: HIT
Age: 363739
Server: cloudflare
CF-RAY: 79ea644469696bc7-SIN
Old-Content-Encoding: br
Old-Transfer-Encoding: chunked
Content-Length: 778320

<long.................body data>

Which is not ideal for streaming data back to another client, and streaming data is very usueful in cases:

• The data is too big for client memory to handle (download 8GB, even multiple 8GB files at the same time), which causes OOM (out-of-memory) error
• Sometimes client want to abort request as soon as possible, so downloading the whole response body is kinda waste, especially if response body is huge (100MB+)

[Bug]: Crash after servals hour on concurrency

TLS client version

v1.3.12

System information

windows 10
amd ryzen 7 3700x

Issue description

github.com/bogdanfinn/fhttp.setRequestCancel.func4()
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/fhttp/client.go:398 +0x8b
created by github.com/bogdanfinn/fhttp.setRequestCancel
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/fhttp/client.go:397 +0x44a
goroutine 439378 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x181b7c2e2e8, 0x72)
        C:/Program Files/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc3afeae5b260f336?, 0xc005b4b6c0?, 0x0)
        C:/Program Files/Go/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.execIO(0xc004a09418, 0xe04580)
        C:/Program Files/Go/src/internal/poll/fd_windows.go:175 +0xf7
internal/poll.(*FD).Read(0xc004a09400, {0xc001098000, 0x1500, 0x1500})
        C:/Program Files/Go/src/internal/poll/fd_windows.go:436 +0x2b8
net.(*netFD).Read(0xc004a09400, {0xc001098000?, 0xc001098005?, 0x36?})
        C:/Program Files/Go/src/net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc00022f2d8, {0xc001098000?, 0x0?, 0x0?})
        C:/Program Files/Go/src/net/net.go:183 +0x45
github.com/bogdanfinn/utls.(*atLeastReader).Read(0xc0044d6678, {0xc001098000?, 0xc0044d6678?, 0x0?})
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/utls/conn.go:791 +0x3d
bytes.(*Buffer).ReadFrom(0xc001127430, {0xe74660, 0xc0044d6678})
        C:/Program Files/Go/src/bytes/buffer.go:202 +0x98
github.com/bogdanfinn/utls.(*Conn).readFromUntil(0xc001127180, {0x181b72aa0f8?, 0xc00022f2d8}, 0xc000bf4000?)
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/utls/conn.go:813 +0xe5
github.com/bogdanfinn/utls.(*Conn).readRecordOrCCS(0xc001127180, 0x0)
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/utls/conn.go:620 +0x116
github.com/bogdanfinn/utls.(*Conn).readRecord(...)
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/utls/conn.go:586
github.com/bogdanfinn/utls.(*Conn).Read(0xc001127180, {0xc004104000, 0x1000, 0xc003ee5e00?})
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/utls/conn.go:1300 +0x16f
bufio.(*Reader).Read(0xc002ec1b60, {0xc0036b6658, 0x9, 0x30?})
        C:/Program Files/Go/src/bufio/bufio.go:237 +0x1bb
io.ReadAtLeast({0xe740e0, 0xc002ec1b60}, {0xc0036b6658, 0x9, 0x9}, 0x9)
        C:/Program Files/Go/src/io/io.go:332 +0x9a
io.ReadFull(...)
        C:/Program Files/Go/src/io/io.go:351
github.com/bogdanfinn/fhttp/http2.readFrameHeader({0xc0036b6658?, 0x9?, 0xcd1ca2?}, {0xe740e0?, 0xc002ec1b60?})
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/fhttp/http2/frame.go:238 +0x6e
github.com/bogdanfinn/fhttp/http2.(*Framer).ReadFrame(0xc0036b6620)
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/fhttp/http2/frame.go:494 +0x95
github.com/bogdanfinn/fhttp/http2.(*clientConnReadLoop).run(0xc005b4bfa0)
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/fhttp/http2/transport.go:2100 +0x170
github.com/bogdanfinn/fhttp/http2.(*ClientConn).readLoop(0xc002c6c1a0)
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/fhttp/http2/transport.go:2021 +0x79
created by github.com/bogdanfinn/fhttp/http2.(*Transport).newClientConn
        D:/Newfolder/PROJECT/GolandProjects/Apitest/vendor/github.com/bogdanfinn/fhttp/http2/transport.go:860 +0xc92

Steps to reproduce / Code Sample

type request struct {
	client       tls_client.HttpClient
	Thread       int
	userdata     string
        data string
}

func init_(thread int) request {
	tl, _ := tls_client.NewHttpClient(nil, option()...)
	return request{
		client:       tl,
		Thread:       thread,
		userdata:     userdataitter(),
	}
}

func (ms *request) Today() bool {
	url := "https://api.mydomain.com/data"
	payload := fmt.Sprintf("data=today", ms.userdata)
	req, err := http.NewRequest(http.MethodPost, url, strings.NewReader(payload))
	if err != nil {
		PrintLogF(err.Error(), ms.Thread)
		return false
	}

	req.Header = bheader
	resp, err := ms.client.Do(req)
	if err != nil {
		ms.clientChanger()
		PrintLogE(err.Error(), ms.Thread)
		return false
	}
        defer resp.Body.Close()

	readBytes, err := io.ReadAll(resp.Body)
	if err != nil {
		PrintLogF(err.Error(), ms.Thread)
		return false
	}

	var data map[string]interface{}
	err = json.Unmarshal(readBytes, &data)
	if err != nil {
		PrintLogF("Error unmarshalling JSON:"+err.Error(), ms.Thread)
		return false, false
	}
        ms.data = data

	return true

}

func (ms *request) postdata() bool {

	url := "https://api.seconddomain.com/postdata"

	req, err := http.NewRequest(http.MethodPost, url, bytes.NewBuffer(ms.data))
	if err != nil {
		PrintLogF(err.Error(), ms.Thread)
		return false
	}

	req.Header = ms.headerstream
	resp, err := ms.client.Do(req)
	if err != nil {
		PrintLogE(err.Error(), ms.Thread)
		time.Sleep(5 * time.Second)
		return false
	}
        defer resp.Body.Close()
	if resp.StatusCode == 201 {
		return true
	} else {
		readBytes, err := io.ReadAll(resp.Body)
		if err != nil {
			PrintLogF(err.Error(), ms.Thread)
			return false
		}
		PrintLogM(fmt.Sprintf("status : %d dat : %s", resp.StatusCode, readBytes), ms.Thread)
	}
	return false
}

func run(Threads int) bool {
	self := init_(Threads)
	for i := 0; i < 3; i++{
		anid := self.Today()
		if !anid{
			if i == 2{
				PrintLogF(fmt.Sprintf("FAIL %s", self.userdata), self.Thread)
				self.client.CloseIdleConnections()
				return false
			}
			continue
		}
		break
	}
	for i := 0; i < 3; i++{
		anid := self.postdata()
		if !anid{
			if i == 2{
				PrintLogF(fmt.Sprintf("FAIL POST %s", self.userdata), self.Thread)
				self.client.CloseIdleConnections()
				return false
			}
			continue
		}
		break
	}
	self.client.CloseIdleConnections()

	return true
}

func main() {
	
	threads := CurrentConfig.Threads // Define the number of threads
	//threads = 1

	var wg sync.WaitGroup
	wg.Add(threads)

	for i := 1; i <= threads; i++ {
		go func(threadNum int) {
			defer wg.Done()
			run(threadNum)
		}(i)
		time.Sleep(500 * time.Millisecond)
	}
	wg.Wait()
	PrintLog("DONE", 0)
}

Using JA3 to build Spec results in errors

When using already preset ClientSpecs (like HelloChrome_105), it works fine as intended.
Then getting the ja3 String (771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0 the one I get for Chrome105) and trying to make a CustomClient with that doesnt work.

Getting the error local error: tls: bad record MAC, and on further investigation it happens here at conn.go. The "real" error is this: cipher: message authentication failed

What I did see is that the JA3 string, when both (Chrome105 and CustomClient from JA3 of Chrome105) parsed at https://tls.peet.ws/api/tls both have the same ja3 and ja3 hash, the only difference is one works for sites and one doesnt and I dont know why (without or with grease, both dont work in the ja3 customclient).

crashes when too many requests

I get a crash error when I make too many synchronous requests with nodejs. When I send requests at 100ms intervals, it gives an error after a short while.

#
# Fatal error in , line 0
# Check failed: result.second.
#
#
#
#FailureMessage Object: 0000005DD19FE0D0
 1: 00007FF7FAECB1EF v8::internal::CodeObjectRegistry::~CodeObjectRegistry+123599
 2: 00007FF7FADE7E7F std::basic_ostream<char,std::char_traits<char> >::operator<<+65407
 3: 00007FF7FBAC4482 V8_Fatal+162
 4: 00007FF7FB52EC6D v8::internal::BackingStore::Reallocate+637
 5: 00007FF7FB7781D9 v8::ArrayBuffer::GetBackingStore+137
 6: 00007FF7FAE9AD29 napi_get_typedarray_info+393
 7: 00007FFE71348828
 8: 00007FFE71347C19
 9: 00007FFE7134D003
10: 00007FFE7134ED79
11: 00007FFE7134DD09
12: 00007FFE7134949E
13: 00007FFE713480B7
14: 00007FFE7134EF63
15: 00007FF7FAE954EB node::Stop+32747
16: 00007FF7FB746BE6 v8::internal::Builtins::code_handle+172806
17: 00007FF7FB7467D9 v8::internal::Builtins::code_handle+171769
18: 00007FF7FB746A9C v8::internal::Builtins::code_handle+172476
19: 00007FF7FB746900 v8::internal::Builtins::code_handle+172064
20: 00007FF7FB819FE1 v8::internal::SetupIsolateDelegate::SetupHeap+494641
21: 000001D623F3D2BB

[Bug]: zoominfo网站请求不通

TLS client version

v1.3.12

System information

url = 'https://www.zoominfo.com/c/apple-inc/2441797'
code = 403

Issue description

url = 'https://www.zoominfo.com/c/apple-inc/2441797'
code = 403

Steps to reproduce / Code Sample

url = 'https://www.zoominfo.com/c/apple-inc/2441797'
code = 403