ultrafunkamsterdam/undetected-chromedriver

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

undetected_chromedriver

https://github.com/ultrafunkamsterdam/undetected-chromedriver

Optimized Selenium Chromedriver patch which does not trigger anti-bot services like Distill Network / Imperva / DataDome / Botprotect.io Automatically downloads the driver binary and patches it.

• Tested until current chrome beta versions
• Works also on Brave Browser and many other Chromium based browsers, but you need to know what you're doing and needs some tweaking.
• Python 3.6++**

Installation

pip install undetected-chromedriver

or , if you're feeling adventurous, install directly via github

pip install git+https://www.github.com/ultrafunkamsterdam/undetected-chromedriver@master     # replace @master with @branchname for other branches

---

Message for all

I will be putting limits on the issue tracker. It has beeen abused too long.
any good news?
Yes, i've opened Undetected-Discussions which i think will help us better in the long run.

---

What this is not

THIS PACKAGE DOES NOT, and i repeat DOES NOT hide your IP address, so when running from a datacenter (even smaller ones), chances are large you will not pass! Also, if your ip reputation at home is low, you won't pass!

Running following code from home , and from a datacenter.

import undetected_chromedriver as uc
driver = uc.Chrome(headless=True,use_subprocess=False)
driver.get('https://nowsecure.nl')
driver.save_screenshot('nowsecure.png')

3.5.0

• selenium 4.10 caused some issues. 3.5.0 is compatible and has selenium 4.9 or above pinned. I can't support <4.9 any longer.
• Removed some kwargs from constructor: service_args, service_creationflags, service_log_path.
• added find_elements_recursive generator function. which is more of a convenience funtion as lots of websites seem to serve different content from different frames, making it hard to use find_elements

3.4.5

• What a week. Had the recent advancedments in Automation-Detection algorithms pwned (so i thought) with 3.4.0, but apparently, for some OS-es this caused an error when interacting with elements. Had to revert back using a different method, fix bugs, and now eventually was still able to stick to the initial idea (+ fixing bugs)
• Update to chrome 110 caused another surprise, this time for HEADLESS users.
• although headless is unsupported officially, i did patch it!
• happy to announce IT IS NOW UNDETECTED AS WELL (but still unsupported ;))
• special thanks here to @mdmintz and @abdulzain6
• also special thanks to @sebdelsol for his help troughout the issues section completely voluntarily, you must be crazy :)

3.4.0

Big update! be careful as it -potentially- could break your code.

• rewritten the anti-detection mechanism instead of removing and renaming variables, we just keep them, but prevent them from being injected in the first place. This will keep us safe from detection at least for the near future.

• rewritten the file naming, to prevent ending up with 1000 of {randomstring}_chromedriver.exe 's instead it is just called undetected_chromedriver.exe

• cleanup removed compat,v2 files and tests folder

3.2.0

• added an example containing some typical webdriver code, answers to commonly asked questions, pitfalls + showcasing some tricks to ditch the need for multithreading.

>>>> example code here <<<<

• added WebElement.click_safe() method, which you can try in case you get detected after clicking a link. This is not guaranteed t o work.

• added WebElement.children(self, tag=None, recursive=False) to easily get/find child nodes. example:

  body = driver.find_element('tag name', 'body')
  
  # get the 6th child (any tag) of body, and grab all img's within (recursive). 
  images = body.children()[6].children('img', True)
  srcs = list(map(lambda _:_.attrs.get('src'), images))
  

• added example.py where i can point people at when asking silly questions (no, its actually quite cool, everyone should see it)

• added support for lambda platform

• added support for x86_32

• added support for systems reporting as linux2

• some refactoring

3.1.6

still passing strong

• use_subprocess now defaults to True. too many people don't understand multiprocessing and name == 'main, and after testing, it seems not to make a difference anymore in chrome 104+

• added no_sandbox, which defaults to True, and this without the annoying "you are using unsecure command line ..." bar.

• update Docker image. you can now vnc or rdp into your container to see the actual browser window

• of course, "regular" mode works as well

3.1.0

this version might break your code, test before update!

• added new anti-detection logic!

• v2 has become the main module, so no need for references to v2 anymore. this mean you can now simply use:

  import undetected_chromedriver as uc
  driver = uc.Chrome()
  driver.get('https://nowsecure.nl')

  for backwards compatibility, v2 is not removed, but aliassed to the main module.

• Fixed "welcome screen" nagging on non-windows OS-es. For those nagfetishists who ❤ welcome screens and feeding google with even more data, use Chrome(suppress_welcome=False).

• replaced executable_path in constructor in favor of browser_executable_path which should not be used unless you are the edge case (yep, you are) who can't add your custom chrome installation folder to your PATH environment variable, or have an army of different browsers/versions and automatic lookup returns the wrong browser

• "v1" (?) moved to _compat for now.

• fixed dependency versions

• ChromeOptions custom handling removed, so it is compatible with webdriver.chromium.options.ChromiumOptions.

• removed Chrome.get() fu and restored back to "almost" original:

  • no with statements needed anymore, although it will still work for the sake of backward-compatibility.
  • no sleeps, stop-start-sessions, delays, or async cdp black magic!
  • this will solve a lot of other "issues" as well.

• test success to date: 100%

• just to mention it another time, since some people have hard time reading: headless is still WIP. Raising issues is needless

3.0.4 changes

• change process creation behavior to be fully detached

• changed .get(url) method to always use the contextmanager

• changed .get(url) method to use cdp under the hood.

  ... the with statement is not necessary anymore ..

• todo: work towards asyncification and selenium 4

words of wisdom:

Whenever you encounter the daunted

from session not created: This version of ChromeDriver only supports Chrome version 96 # or what ever version

the solution is simple:

   import undetected_chromedriver as uc
   driver = uc.Chrome( version_main = 95 )

July 2021: Currently busy implementing selenium 4 for undetected-chromedriver

newsflash: #255

Usage

To prevent unnecessary hair-pulling and issue-raising, please mind the important note at the end of this document .

easy

Literally, this is all you have to do. Settings are included and your browser executable is found automagically. This is also the snippet i recommend using in case you experience an issue.

import undetected_chromedriver as uc


driver = uc.Chrome()
driver.get( 'https://nowsecure.nl' )  # my own test test site with max anti-bot protection

more advanced way, including setting profie folder

Literally, this is all you have to do. If a specified folder does not exist, a NEW profile is created. Data dirs which are specified like this will not be autoremoved on exit.

import undetected_chromedriver as uc


options = uc.ChromeOptions()

# setting profile
options.user_data_dir = "c:\\temp\\profile"

# use specific (older) version
driver = uc.Chrome(
    options = options , version_main = 94
    )  # version_main allows to specify your chrome version instead of following chrome global version

driver.get( 'https://nowsecure.nl' )  # my own test test site with max anti-bot protection

expert mode, including Devtool/Wire events

Literally, this is all you have to do. You can now listen and subscribe to the low level devtools-protocol. I just recently found out that is also on planning for future release of the official chromedriver. However i implemented my own for now. Since i needed it myself for investigation.

import undetected_chromedriver as uc
from pprint import pformat

driver = uc.Chrome(enable_cdp_events=True)

def mylousyprintfunction(eventdata):
    print(pformat(eventdata))
    
# set the callback to Network.dataReceived to print (yeah not much original)
driver.add_cdp_listener("Network.dataReceived", mylousyprintfunction)
driver.get('https://nowsecure.nl')  # known url using cloudflare's "under attack mode"


def mylousyprintfunction(message):
    print(pformat(message))


# for more inspiration checkout the link below
# https://chromedevtools.github.io/devtools-protocol/1-3/Network/

# and of couse 2 lousy examples
driver.add_cdp_listener('Network.requestWillBeSent', mylousyprintfunction)
driver.add_cdp_listener('Network.dataReceived', mylousyprintfunction)

# hint: a wildcard captures all events!
# driver.add_cdp_listener('*', mylousyprintfunction)

# now all these events will be printed in my console

driver.get('https://nowsecure.nl')


{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'type': 'other'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Upgrade-Insecure-Requests': '1',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'VeryHigh',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/'},
            'requestId': '449906A5C736D819123288133F2797E6',
            'timestamp': 190010.996717,
            'type': 'Document',
            'wallTime': 1621835932.112026}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/',
                        ':scheme': 'https',
                        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'document',
                        'sec-fetch-mode': 'navigate',
                        'sec-fetch-site': 'none',
                        'sec-fetch-user': '?1',
                        'upgrade-insecure-requests': '1',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '449906A5C736D819123288133F2797E6'}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'alt-svc': 'h3-27=":443"; ma=86400, h3-28=":443"; '
                                   'ma=86400, h3-29=":443"; ma=86400',
                        'cache-control': 'private, max-age=0, no-store, '
                                         'no-cache, must-revalidate, '
                                         'post-check=0, pre-check=0',
                        'cf-ray': '65444b779ae6546f-LHR',
                        'cf-request-id': '0a3e8d7eba0000546ffd3fa000000001',
                        'content-type': 'text/html; charset=UTF-8',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'expect-ct': 'max-age=604800, '
                                     'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                        'expires': 'Thu, 01 Jan 1970 00:00:01 GMT',
                        'nel': '{"report_to":"cf-nel","max_age":604800}',
                        'permissions-policy': 'accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()',
                        'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=CAfobYlmWImQ90e%2B4BFBhpPYL%2FyGyBvkcWAj%2B%2FVOLoEq0NVrD5jU9m5pi%2BKI%2BOAnINLPXOCoX2psLphA5Z38aZzWNr3eW%2BDTIK%2FQidc%3D"}],"group":"cf-nel","max_age":604800}',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding',
                        'x-frame-options': 'SAMEORIGIN'},
            'requestId': '449906A5C736D819123288133F2797E6',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '449906A5C736D819123288133F2797E6',
            'response': {'connectionId': 158,
                         'connectionReused': False,
                         'encodedDataLength': 851,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'alt-svc': 'h3-27=":443"; ma=86400, '
                                                'h3-28=":443"; ma=86400, '
                                                'h3-29=":443"; ma=86400',
                                     'cache-control': 'private, max-age=0, '
                                                      'no-store, no-cache, '
                                                      'must-revalidate, '
                                                      'post-check=0, '
                                                      'pre-check=0',
                                     'cf-ray': '65444b779ae6546f-LHR',
                                     'cf-request-id': '0a3e8d7eba0000546ffd3fa000000001',
                                     'content-type': 'text/html; charset=UTF-8',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'expect-ct': 'max-age=604800, '
                                                  'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                                     'expires': 'Thu, 01 Jan 1970 00:00:01 GMT',
                                     'nel': '{"report_to":"cf-nel","max_age":604800}',
                                     'permissions-policy': 'accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()',
                                     'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=CAfobYlmWImQ90e%2B4BFBhpPYL%2FyGyBvkcWAj%2B%2FVOLoEq0NVrD5jU9m5pi%2BKI%2BOAnINLPXOCoX2psLphA5Z38aZzWNr3eW%2BDTIK%2FQidc%3D"}],"group":"cf-nel","max_age":604800}',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding',
                                     'x-frame-options': 'SAMEORIGIN'},
                         'mimeType': 'text/html',
                         'protocol': 'h2',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/',
                                            ':scheme': 'https',
                                            'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'document',
                                            'sec-fetch-mode': 'navigate',
                                            'sec-fetch-site': 'none',
                                            'sec-fetch-user': '?1',
                                            'upgrade-insecure-requests': '1',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932177.923,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'TLS 1.3',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 503,
                         'statusText': '',
                         'timing': {'connectEnd': 40.414,
                                    'connectStart': 0,
                                    'dnsEnd': 0,
                                    'dnsStart': 0,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 60.361,
                                    'requestTime': 190011.002239,
                                    'sendEnd': 41.348,
                                    'sendStart': 41.19,
                                    'sslEnd': 40.405,
                                    'sslStart': 10.853,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/'},
            'timestamp': 190011.06449,
            'type': 'Document'}}
{'method': 'Page.frameStartedLoading',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700'}}
{'method': 'Page.frameNavigated',
 'params': {'frame': {'adFrameType': 'none',
                      'crossOriginIsolatedContextType': 'NotIsolated',
                      'domainAndRegistry': 'nowsecure.nl',
                      'gatedAPIFeatures': ['SharedArrayBuffers',
                                           'SharedArrayBuffersTransferAllowed'],
                      'id': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
                      'loaderId': '449906A5C736D819123288133F2797E6',
                      'mimeType': 'text/html',
                      'secureContextType': 'Secure',
                      'securityOrigin': 'https://nowsecure.nl',
                      'url': 'https://nowsecure.nl/'}}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 9835,
            'encodedDataLength': 0,
            'requestId': '449906A5C736D819123288133F2797E6',
            'timestamp': 190011.093343}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 10713,
            'requestId': '449906A5C736D819123288133F2797E6',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.064011}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'stack': {'callFrames': [{'columnNumber': 51,
                                                    'functionName': '',
                                                    'lineNumber': 114,
                                                    'scriptId': '8',
                                                    'url': 'https://nowsecure.nl/'},
                                                   {'columnNumber': 9,
                                                    'functionName': '',
                                                    'lineNumber': 115,
                                                    'scriptId': '8',
                                                    'url': 'https://nowsecure.nl/'}]},
                          'type': 'script'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'Low',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f'},
            'requestId': '17180.2',
            'timestamp': 190011.106133,
            'type': 'Script',
            'wallTime': 1621835932.221325}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'columnNumber': 13,
                          'lineNumber': 117,
                          'type': 'parser',
                          'url': 'https://nowsecure.nl/'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'Low',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f'},
            'requestId': '17180.3',
            'timestamp': 190011.106911,
            'type': 'Image',
            'wallTime': 1621835932.222102}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'type': 'parser', 'url': 'https://nowsecure.nl/'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'Low',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f'},
            'requestId': '17180.4',
            'timestamp': 190011.109527,
            'type': 'Image',
            'wallTime': 1621835932.224719}}
{'method': 'Page.domContentEventFired', 'params': {'timestamp': 190011.110345}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f',
                        ':scheme': 'https',
                        'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'image',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '17180.3'}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f',
                        ':scheme': 'https',
                        'accept': '*/*',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'script',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '17180.2'}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f',
                        ':scheme': 'https',
                        'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'image',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '17180.4'}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'accept-ranges': 'bytes',
                        'cache-control': 'max-age=7200\npublic',
                        'cf-ray': '65444b781d1de604-LHR',
                        'content-length': '42',
                        'content-type': 'image/gif',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'etag': '"60a4d856-2a"',
                        'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                        'last-modified': 'Wed, 19 May 2021 09:20:22 GMT',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding',
                        'x-content-type-options': 'nosniff',
                        'x-frame-options': 'DENY'},
            'requestId': '17180.3',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'accept-ranges': 'bytes',
                        'cache-control': 'max-age=7200\npublic',
                        'cf-ray': '65444b781d1fe604-LHR',
                        'content-length': '42',
                        'content-type': 'image/gif',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'etag': '"60a4d856-2a"',
                        'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                        'last-modified': 'Wed, 19 May 2021 09:20:22 GMT',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding',
                        'x-content-type-options': 'nosniff',
                        'x-frame-options': 'DENY'},
            'requestId': '17180.4',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.resourceChangedPriority',
 'params': {'newPriority': 'High',
            'requestId': '17180.4',
            'timestamp': 190011.171057}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '17180.3',
            'response': {'connectionId': 0,
                         'connectionReused': False,
                         'encodedDataLength': 214,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'accept-ranges': 'bytes',
                                     'cache-control': 'max-age=7200\npublic',
                                     'cf-ray': '65444b781d1de604-LHR',
                                     'content-length': '42',
                                     'content-type': 'image/gif',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'etag': '"60a4d856-2a"',
                                     'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                                     'last-modified': 'Wed, 19 May 2021 '
                                                      '09:20:22 GMT',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding',
                                     'x-content-type-options': 'nosniff',
                                     'x-frame-options': 'DENY'},
                         'mimeType': 'image/gif',
                         'protocol': 'h3-29',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f',
                                            ':scheme': 'https',
                                            'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'referer': 'https://nowsecure.nl/',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'image',
                                            'sec-fetch-mode': 'no-cors',
                                            'sec-fetch-site': 'same-origin',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932265.169,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'QUIC',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 200,
                         'statusText': '',
                         'timing': {'connectEnd': 26.087,
                                    'connectStart': 0,
                                    'dnsEnd': 0,
                                    'dnsStart': 0,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 40.709,
                                    'requestTime': 190011.109386,
                                    'sendEnd': 26.346,
                                    'sendStart': 26.182,
                                    'sslEnd': 26.087,
                                    'sslStart': 0,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f'},
            'timestamp': 190011.174536,
            'type': 'Image'}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 42,
            'encodedDataLength': 0,
            'requestId': '17180.3',
            'timestamp': 190011.174737}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 0,
            'encodedDataLength': 44,
            'requestId': '17180.3',
            'timestamp': 190011.17524}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 258,
            'requestId': '17180.3',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.152073}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '17180.4',
            'response': {'connectionId': 0,
                         'connectionReused': True,
                         'encodedDataLength': 178,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'accept-ranges': 'bytes',
                                     'cache-control': 'max-age=7200\npublic',
                                     'cf-ray': '65444b781d1fe604-LHR',
                                     'content-length': '42',
                                     'content-type': 'image/gif',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'etag': '"60a4d856-2a"',
                                     'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                                     'last-modified': 'Wed, 19 May 2021 '
                                                      '09:20:22 GMT',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding',
                                     'x-content-type-options': 'nosniff',
                                     'x-frame-options': 'DENY'},
                         'mimeType': 'image/gif',
                         'protocol': 'h3-29',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f',
                                            ':scheme': 'https',
                                            'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'referer': 'https://nowsecure.nl/',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'image',
                                            'sec-fetch-mode': 'no-cors',
                                            'sec-fetch-site': 'same-origin',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932268.067,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'QUIC',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 200,
                         'statusText': '',
                         'timing': {'connectEnd': -1,
                                    'connectStart': -1,
                                    'dnsEnd': -1,
                                    'dnsStart': -1,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 42.415,
                                    'requestTime': 190011.110341,
                                    'sendEnd': 25.713,
                                    'sendStart': 25.609,
                                    'sslEnd': -1,
                                    'sslStart': -1,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f'},
            'timestamp': 190011.175727,
            'type': 'Image'}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 42,
            'encodedDataLength': 0,
            'requestId': '17180.4',
            'timestamp': 190011.175856}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 0,
            'encodedDataLength': 44,
            'requestId': '17180.4',
            'timestamp': 190011.176133}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 222,
            'requestId': '17180.4',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.153335}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'alt-svc': 'h3-27=":443"; ma=86400, h3-28=":443"; '
                                   'ma=86400, h3-29=":443"; ma=86400',
                        'cache-control': 'max-age=0, must-revalidate',
                        'cf-ray': '65444b781d1ee604-LHR',
                        'cf-request-id': '0a3e8d7f140000e60496387000000001',
                        'content-encoding': 'br',
                        'content-type': 'text/javascript',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'expect-ct': 'max-age=604800, '
                                     'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                        'nel': '{"report_to":"cf-nel","max_age":604800}',
                        'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=ZtI%2Bx8B7DpI8%2FsDA72maecFVCPvIsfBOyJjT8weyiqfmrHrmcBYpRhc%2FI%2F6JmIlnxW%2F%2BBohxLi1F8mpjAUabJ0kXLYnmjGKp2Ndio9M%3D"}],"group":"cf-nel","max_age":604800}',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding'},
            'requestId': '17180.2',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '17180.2',
            'response': {'connectionId': 0,
                         'connectionReused': True,
                         'encodedDataLength': 510,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'alt-svc': 'h3-27=":443"; ma=86400, '
                                                'h3-28=":443"; ma=86400, '
                                                'h3-29=":443"; ma=86400',
                                     'cache-control': 'max-age=0, '
                                                      'must-revalidate',
                                     'cf-ray': '65444b781d1ee604-LHR',
                                     'cf-request-id': '0a3e8d7f140000e60496387000000001',
                                     'content-encoding': 'br',
                                     'content-type': 'text/javascript',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'expect-ct': 'max-age=604800, '
                                                  'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                                     'nel': '{"report_to":"cf-nel","max_age":604800}',
                                     'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=ZtI%2Bx8B7DpI8%2FsDA72maecFVCPvIsfBOyJjT8weyiqfmrHrmcBYpRhc%2FI%2F6JmIlnxW%2F%2BBohxLi1F8mpjAUabJ0kXLYnmjGKp2Ndio9M%3D"}],"group":"cf-nel","max_age":604800}',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding'},
                         'mimeType': 'text/javascript',
                         'protocol': 'h3-29',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f',
                                            ':scheme': 'https',
                                            'accept': '*/*',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'referer': 'https://nowsecure.nl/',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'script',
                                            'sec-fetch-mode': 'no-cors',
                                            'sec-fetch-site': 'same-origin',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932301.817,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'QUIC',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 200,
                         'statusText': '',
                         'timing': {'connectEnd': -1,
                                    'connectStart': -1,
                                    'dnsEnd': -1,
                                    'dnsStart': -1,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 78.885,
                                    'requestTime': 190011.107975,
                                    'sendEnd': 27.934,
                                    'sendStart': 27.809,
                                    'sslEnd': -1,
                                    'sslStart': -1,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f'},
            'timestamp': 190011.188468,
            'type': 'Script'}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 31556,
            'encodedDataLength': 0,
            'requestId': '17180.2',
            'timestamp': 190011.188663}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 6737,
            'encodedDataLength': 11251,
            'requestId': '17180.2',
            'timestamp': 190011.198249}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 0,
            'encodedDataLength': 2049,
            'requestId': '17180.2',
            'timestamp': 190011.200943}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 13810,
            'requestId': '17180.2',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.198142}}
{'method': 'Page.loadEventFired', 'params': {'timestamp': 190011.204711}}
{'method': 'Page.frameScheduledNavigation',
 'params': {'delay': 12,
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'reason': 'metaTagRefresh',
            'url': 'https://nowsecure.nl/'}}
{'method': 'Page.frameStoppedLoading',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700'}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'type': 'other'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'High',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/favicon.ico'},
            'requestId': '17180.5',
            'timestamp': 190011.210491,
            'type': 'Other',
            'wallTime': 1621835932.325683}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [{'blockedReasons': [],
                                   'cookie': {'domain': 'nowsecure.nl',
                                              'expires': 1621839532,
                                              'httpOnly': False,
                                              'name': 'cf_chl_prog',
                                              'path': '/',
                                              'priority': 'Medium',
                                              'sameParty': False,
                                              'secure': False,
                                              'session': False,
                                              'size': 12,
                                              'sourcePort': 443,
                                              'sourceScheme': 'Secure',
                                              'value': 'e'}}],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/favicon.ico',
                        ':scheme': 'https',
                        'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'cookie': 'cf_chl_prog=e',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'image',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},

# hopefullly you get the idea.

the easy way (v1 old stuff)

import undetected_chromedriver as uc


driver = uc.Chrome()
driver.get( 'https://distilnetworks.com' )

target specific chrome version (v1 old stuff)

import undetected_chromedriver as uc


uc.TARGET_VERSION = 85
driver = uc.Chrome()

monkeypatch mode (v1 old stuff)

Needs to be done before importing from selenium package

import undetected_chromedriver as uc


uc.install()

from selenium.webdriver import Chrome


driver = Chrome()
driver.get( 'https://distilnetworks.com' )

the customized way (v1 old stuff)

import undetected_chromedriver as uc


# specify chromedriver version to download and patch
uc.TARGET_VERSION = 78

# or specify your own chromedriver binary (why you would need this, i don't know)

uc.install(
    executable_path = 'c:/users/user1/chromedriver.exe' ,
    )

opts = uc.ChromeOptions()
opts.add_argument( f'--proxy-server=socks5://127.0.0.1:9050' )
driver = uc.Chrome( options = opts )
driver.get( 'https://distilnetworks.com' )

datadome.co example (v1 old stuff)

These guys have actually a powerful product, and a link to this repo, which makes me wanna test their product. Make sure you use a "clean"ip for this one.

#
# STANDARD selenium Chromedriver
#
from selenium import webdriver


chrome = webdriver.Chrome()
chrome.get( 'https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/' )
chrome.save_screenshot( 'datadome_regular_webdriver.png' )
True  # it caused my ip to be flagged, unfortunately

#
# UNDETECTED chromedriver (headless,even)
#
import undetected_chromedriver as uc


options = uc.ChromeOptions()
options.headless = True
options.add_argument( '--headless' )
chrome = uc.Chrome( options = options )
chrome.get( 'https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/' )
chrome.save_screenshot( 'datadome_undetected_webddriver.png' )

Check both saved screenhots here

undetected-chromedriver's Issues

Install more

Hello!

Have a possibility to install and execute this "undetected-chromedriver" like a "original Chromedriver" ?? Install and execute without any command line "pip"?

Only download the archive and then execute via python + selenium point via executable_path ?
It would be possible?

Many thanks!

403 Error: Nike

hello I am currently getting detected using the undetected_chromedriver, I have been using it for a couple weeks now... it ran flawlessly a day or two ago though when tested today on the same instance (website/drop style) I got detected this time. Would appreciate the help when you get the chance, I ran the navigator command to see if the flag was true, it returns undetected despite the error log.

NOTE: I did not update to the new version when first receiving the error, I was using 1.3.7 though updating to the latest version did not change anything

www.nike.com-1601653606474.log

Update: I believe I have figured out the same-cookie issue I took a screen shot of above, I do not think it relates to the "bigger picture" (why I am getting detected on nike) I am receiving error similar to the log above the file above is with the cookie error
fullLOG.log
this is the full log without receiving the same-site cookie error

Still gets detected with Recaptcha v3

Any website that uses recaptcha v3 and which I try to automate with this "undetected" version of the chromedriver just makes recaptcha v3 respond with "There was an error trying to send your message. Please try again later."

So I'm sure it's detected :)

Getting detected if any linkedin url called

Hi,

I am able to load up undetected-chromedriver and it shows undetected initially.
When I call any linkedin person profile. Eg. https://www.linkedin.com/in/arnavsaxena/
Immediately after calling it, the webdriver gets detected and it fails.
OS : MacOS Sierra 10.12.6
Chrome Version : 83.0
Could you please help explain this?

import undetected_chromedriver as uc
chromedriver_path = '/Users/arnavsaxena/Downloads/chromedriver'
uc.install(executable_path=chromedriver_path)
from selenium import webdriver

options = webdriver.ChromeOptions()
# opts.add_argument(f'--proxy-server=socks5://127.0.0.1:9050')

options.add_argument("user-data-dir=/Users/arnavsaxena/Library/Application Support/Google/Chrome/Profile 1/Default") #Path to your chrome profile
driver = webdriver.Chrome(chrome_options=options)
test_site = 'https://www.linkedin.com/in/arnavsaxena/'

driver.get('https://bot.sannysoft.com')
# driver.get(test_site)

Contact

Hey, am looking to hire the creator of this, any contact available?

Can't load extensions in headless mode

Extension I was trying to add https://chrome.google.com/webstore/detail/domain-blocker/ggdcjplapccgoinblmidpkoocfafajfa which I used https://crx-downloader.com/ to download it.

My code works perfectly fine in headlessmode off, but when I turn it on I can't add / use an extension at all. Noticed this with lots of other extensions too, and I'm wondering why?

Code:

def create_session(headless):
	import undetected_chromedriver as uc
	options = uc.ChromeOptions()
	options.headless = headless
	options.add_extension(r"C:\Users\..\Desktop\domain_blocker.crx")
	global driver
	driver = uc.Chrome(options=options)
create_session(True)

Errors in:

Traceback (most recent call last):
  File "C:\Users\..\Desktop\project.py", line 158, in <module>
    create_session(True)
  File "C:\Users\..\Desktop\project.py", line 45, in create_session
    driver = uc.Chrome(options=options)
  File "C:\Program Files\Python38\lib\site-packages\undetected_chromedriver\__init__.py", line 51, in __new__
    instance.__init__(*args, **kwargs)
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 76, in __init__
    RemoteWebDriver.__init__(
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: failed to wait for extension background page to load: chrome-extension://ggdcjplapccgoinblmidpkoocfafajfa/_generated_background_page.html
from unknown error: page could not be found: chrome-extension://ggdcjplapccgoinblmidpkoocfafajfa/_generated_background_page.html```

Message: session not created: This version of ChromeDriver only supports Chrome version 87

Hello. I have seen issue #8 but proposed solution doesn't sole my issue.

I'm running a debian with ChromeDriver 83.0.4103.39 and google-chrome 83 and undetected-chromedriver 1.5.1. Had no issue running this configuration with selenium. Recently, I need to bypass CloudFare 5 sec redirection page.

I have tried the 3 methods in the readme of undetected-chromedriver. Got same error each time.

Nov 30 18:21:54 carnac python[855]: selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 87
Nov 30 18:21:54 carnac python[855]: Current browser version is 83.0.4103.106 with binary path /usr/bin/google-chrome

I have tried:

import undetected_chromedriver as uc
uc.TARGET_VERSION = 85
opts = uc.ChromeOptions()
opts.add_argument("--headless")
driver = uc.Chrome(options=opts, enable_console_log=True)
driver.get(my_target_url)

and

import undetected_chromedriver as uc
uc.TARGET_VERSION = 85
uc.install()

from selenium.webdriver import Chrome
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(options=chrome_options) 
driver.get(my_target_url)

Any idea ?

Release patched Drivers

Hi,

could you upload already patched drivers for all common browsers?
So it could be used in any language! Currently searching such a solution for NodeJS.

Regards

p.s.:
sorry for issuing, but did not find any contacts from you :(

Message: session not created: This version of ChromeDriver only supports Chrome version 81

when I run this code :

from undetected_chromedriver import Chrome, ChromeOptions 
browser = Chrome()
browser.get('https://www.quora.com/')

I am getting this error :
raise exception_class(message, screen, stacktrace) selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 81

Note : I have Chrome version 83

ModuleNotFoundError

Hello,

I am amateur working with python3 + Selenium in Pycharm
I have installed and working with selenium and python3 via: pip3

In your readme I view that you said install via "pip install undetected-chromedriver"
but i think that in my case I need to install via "pip3 install undetected-chromedriver"
It would be correct?

I have tried installing using "pip3" And it seems that it has been installed successfully but when go with the script have this error:
ModuleNotFoundError: No module named 'undetected_chromedriver'

This is part of my code:

import undetected_chromedriver as uc
opts = uc.ChromeOptions()
opts.add_argument("--incognito")
opts.add_argument("--disable-blink-features")  
opts.add_argument("--disable-blink-features=AutomationControlled") 
browser = uc.Chrome(options=opts)

Could you tell me that I'm doing wrong?
thanks!

Headless mode

hallo and goedenavond,

thanks for the repo, it's great. I got a question that is not really an issue but maybe you can help me. I'm trying to scrape a few websites that use distil network protection using your chromedriver and rotating proxies and I'm observing the following strange behavior: If I run the exact same code in headless mode I'm getting detected after a few webpages but if I run without headless on a regular display I don't get detected. Detection happens if I use a virtual display as well.

Is there any test that can detect the presence of an actual display that's not patched yet? Or anything else you might think that could cause this?

Thanks

Getting Detected on Remote Linux(Ubuntu) vs Undetected On Mac

I am working on a white hat side project, my intention is to scrape behind a login screen of my own data so that I can plot it :)

I am able to run the code on local env (mac os/ details below) it logins in and able to go to the desired behind login page.

However when promoted to remote linux server (ubuntu/ details below) it fails to login and is rerouted back to login page.

At first I thought it was ip/ dns registering as blacklisted but then I ran both behind a nordvpn (server: us5793) and was still getting the same result: (Works on local not on Remote)

This is the same result for local env and remote env

IP Location	Chicago, Illinois (US)
NordVPN
64.44.80.68, 198.143.57.3
Mac OS X
Chrome 83.0.4103.97
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36
1920px X 1080px
Enabled
Enabled

The expected result is that the function below returns success in finding the "mytrips" text within the html. This indicates the login was a success.

My Speculation Is one of two things,
1 the chromedriver binary responds differently to the cdc replacement you do in your code acts differently in my linux env
2 the way javascript is reinjected back into the code isn't correctly working in linux.

Other found resources:
How to inject JS and beat detection
Many Tests for bot indication
general chrome headless My code passes this for both environments

I'm going to continue hacking away at this thing and would love to help develop a solution for this and other things moving forward :) , Ideally would love to have the equivalent of the networking tab in inspect to debug these things.

Local MacOS (success) -- Login Success
sys.platform: darwin
sysname: Darwin
version: Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64
release: 19.3.0
machine: x86_64
selenium  : 3.141.0

Tried this in python3.6 & 3.8. No luck on either.

Remote Linux(fail) -- Login Fail -- Shouldn't matter with vpn, but this lives in AWS Ec2
sys.platform: linux
sysname: Linux
version: #21~18.04.1-Ubuntu SMP Mon May 11 12:33:03 UTC 2020
release: 5.3.0-1019-aws
machine: x86_64
selenium  : 3.141.0

achieved running behind nordvpn with a shell script

#!/bin/bash

echo "Executing Nord VPN"
nordvpn connect us5793


echo "Executing Python"
python3.8 /home/ubuntu/test.py

echo "Disconnecting VPN"
nordvpn disconnect

**Created a fake account for you to test on as well **

import os
import sys


print(f""" \n
sys.platform: {sys.platform}
sysname: {os.uname().sysname}
version: {os.uname().version}
release: {os.uname().release}
machine: {os.uname().machine}
\n
""")

import undetected_chromedriver as uc
uc.install() #important this is first
from selenium.webdriver import Chrome, ChromeOptions
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup

from time import sleep



class BotDriver:
    def __init__(self,username, pw, start_url, url_behind_login, headless_input = True):
        self.username = username
        self.pw = pw
        chrome_options = ChromeOptions()
        chrome_options.headless = headless_input
        chrome_options.add_argument("--incognito")
        chrome_options.add_argument('--disable-extensions')
        chrome_options.add_argument("--start-maximized")

        self.driver = Chrome(chrome_options=chrome_options)
        
        self.start_url = start_url
        self.url_behind_login = url_behind_login
        self.driver.get('https://www.iplocation.net/')
        self.driver.get_screenshot_as_file(f"iplocation.png")
        self.driver.get(start_url)
        self.waitdriver =  WebDriverWait( self.driver, 10)

    def get_element(self,findby,argument_to_click):
        element = self.waitdriver.until(EC.element_to_be_clickable((findby, argument_to_click))) 

        return element
    def slow_keys(self,input_keys,element,speed=.2):
        for character in input_keys:
            sleep(speed)
            element.send_keys(character)
        sleep(1)
    def main(self):
        element0 = self.get_element( By.LINK_TEXT,"Sign In or Join" )
        element0.click()
        element1 = self.get_element( By.XPATH,'//*[@id="user-id"]' )
        element1.click()
        self.slow_keys(self.username,element1)
        element2 = self.get_element( By.XPATH,'//*[@id="password"]' )
        element2.click()
        self.slow_keys(self.pw,element2)
        self.driver.get_screenshot_as_file(f"before_submit.png")
        element3 = self.get_element( By.XPATH,"//button[@name='submitButton']" )
        element3.click()
        self.driver.get_screenshot_as_file(f"after_submit.png")
        sleep(3)
        #test string to find
        soup = BeautifulSoup(self.driver.page_source, 'lxml')
        test = soup.body.findAll(text='My Trips')
        if len(test) > 1:
            print(f'\n\n\n Login Success ({test} len {len(test)})\n\n\n')
        else:
            print(f'\n\n\n Login failed ({test} len {len(test)})\n\n\n')
        self.driver.get(self.url_behind_login)
        self.driver.get_screenshot_as_file(f"last.png")

if __name__ == "__main__":
    username = input('Enter your login email: ')
    pw = input('Enter PW: ')
    start_url = 'https://www.marriott.com/default.mi'
    url_behind_login = 'https://www.marriott.com/loyalty/findReservationList.mi'
    pbd = BotDriver(username, pw, start_url, url_behind_login, headless_input = True)
    pbd.main()

Webdriver property

The webdriver test at https://bot.sannysoft.com/ fails. It works by checking if the navigator object has the property webdriver (navigator.webdriver || _.has(navigator, "webdriver") returns true)

Edit: managed to resolve

Its getting Detected By PerimeterX

Im try to scrape chegg.com and it has been secured by PerimeterX, Is there any fix to bypass perimeterX?
@ultrafunkamsterdam

Auth Proxies

There's any possibility to run the browser with auth Proxies? (the ones with this format ip:port:user:pass)

C#?

I know this is not an issue, but I did not find any way to contact you.

First of all, I love this repo! But my question is if I patch it before using it in python, would I be able to use that patched chromedriver.exe within a C# project?

Thanks!

Cloudflare DDOS protection

Hi,

working with the latest version (1.5) , with ChromeDriver 85.0.4183.87,
I'm getting blocked by sites with cloudflare DDOS protection (Under Attack Mode).
Example of such site - http://void.to

driver gets detected and gets stuck in the "Checking your browser" page

bet365.com

Do you know why bet365.com won't load when using normal selenium or this undetected chromedriver?

Undetected chrome driver detected by a site

I'm not sure what it does to detect it. But it seems to happen with the web driver only. Loading in normal chrome works perfectly fine.

The link can be found here.
https://secure.runescape.com/m=weblogin/loginform?theme=runescape&mod=www&ssl=1&dest=community

It was recently updated yesterday to reflect this change.

Maybe you might know the problem

File "/usr/local/lib/python2.7/dist-packages/undetected_chromedriver/__init__.py", line 77 logger.info(f"starting undetected_chromedriver.Chrome({args}, {kwargs})") ^ SyntaxError: invalid syntax

how can i fix it?

Undetected Chromedriver gets detected by Cloudflare and Botfront

When trying to visit a Website protected by cloudflare such as https://beta.welovecloudflare.de it just reloads the 5sec UAM Site.
Doesn't happen with when using my real browser.
Another example would be https://lhcdn.cloud/.
These Sites are Dstats and are mostly there to see if you are able to bypass them.
The undetected Chrome driver isnt going past this.
Any Idea why this happens?

Anyways this a very cool project and works mostly!
edit: i am using the updated version btw, it should be headless as well muchos lovos <3

[AliExpress Bonus Buddies] Mobile Taps Being Detected!

If someone can help me out.. would mean alot!
I have spend all night trying to resolve.. so the problem is

I made a Automate Script for AliExpress Bonus Buddies, before the update it didn't required Touch! But now it require touch.. i used your chrome driver before and it couldn't be detected but now I am having issue

If Bot press tap it isn't accepted, If I press the tap it's accepted!!

Human Touch (Mobile View) - iPhone X:

Bot Touch (Mobile View) - iPhone X:

Though.. undected-chromedriver fixed this issue! before when it can be just get clicked with click() but now it doesn't work

To Use Touch I had to disable W3C! maybe that can be the cause, would love a solution for this problem!

Multi-Tab example

Multi-tab doesn´t work like explained in the important note section.

The code I´ve tried looks like this:

import undetected_chromedriver as uc
options = uc.ChromeOptions()
chrome = uc.Chrome(options=options)
chrome.get('https://example.com')
print('Main window detectable: ' + str(chrome.execute_script('return navigator.webdriver')))
chrome.execute_script('window.open();')
chrome.switch_to.window(chrome.window_handles[1])
chrome.get('data:,')
chrome.get('https://example.com')
print('New tab detectable: ' + str(chrome.execute_script('return navigator.webdriver')))

Is there some point I´m missing here?
Would it be possible to update the documentation with a working example?

By the way thanks for publishing this repo, it saves my day. :)

Patch remote selenium

Hello and thank you for the great project. Patching perfectly bypasses the detection in my use case. However I also use the Remote class to connect to the remote selenium standalone server. I assume something similar to what is done in the function patch_selenium_webdriver needs to be done explicitly in corresponding source code. Am I correct? Do you have any indication what needs to be modified?

Most common way to make this completely undetectable.

Hey there !

What are (except from Request Header, different UserAgents, using HQ Proxies (Residential,Mobile), Delay requests)

The most common ways for a crawler build with this module to make it as hard detectable as possible ?

We try to make it as sophisticated and hidden as possible.

What are your suggestions ?

Best regards

vs code IntelliSense no autocomplete

vs code IntelliSense no autocomplete

C#

Is there any way to use this with c#?

bet365

Up until a few days ago this worked on bet365 but now I am just getting a white browser window and a spinning icon showing that the page is not loading.

SyntaxError: Unexpected token '='

Just downloaded it but now when I run my program I keep getting this error selenium.common.exceptions.WebDriverException: Message: unknown error: Runtime.evaluate threw exception: SyntaxError: Unexpected token '='

I changed it back to normal selenium chromedriver and it worked fine so I am not sure what's going wrong as it does not seem to be my code

Here is the full error message
Selenium patched. Safe to import Chrome / ChromeOptions
Selenium patched. Safe to import Chrome / ChromeOptions
Traceback (most recent call last):
File "botc.pyx", line 89, in
order(keys)
File "botc.pyx", line 21, in wrapper
result = method(args, **kw)
File "botc.pyx", line 40, in order
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//[contains(text(), "Large")]'))).click()
File "/usr/local/lib/python3.8/dist-packages/selenium-3.141.0-py3.8.egg/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/usr/local/lib/python3.8/dist-packages/selenium-3.141.0-py3.8.egg/selenium/webdriver/remote/webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python3.8/dist-packages/selenium-3.141.0-py3.8.egg/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.8/dist-packages/selenium-3.141.0-py3.8.egg/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Runtime.evaluate threw exception: SyntaxError: Unexpected token '='
(Session info: chrome=81.0.4044.92)

Google login

Trying to login in Google Account,
Shows:

This browser or app may not be secure. Learn more
Try using a different browser. If you’re already using a supported browser, you can refresh your screen and try again to sign in.

typeError: __init__() got an unexpected keyword argument 'options'

The following is my environment and code
MacOS Mojave 10.14.4
chrome:84

import undetected_chromedriver as uc
chromedriver_path='./chromedriver'
uc.install(executable_path=chromedriver_path)
from selenium.webdriver import Chrome,ChromeOptions
option = ChromeOptions()
option.add_argument('--disable-popup-blocking')
driver = Chrome(chrome_options=option)
driver.get('https://distilnetworks.com')

The following is the error message

Selenium patched. Safe to import Chrome / ChromeOptions
Selenium patched. Safe to import Chrome / ChromeOptions
Selenium patched. Safe to import Chrome / ChromeOptions
Selenium patched. Safe to import Chrome / ChromeOptions
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1664, in
main()
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1068, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/Users/mac_lee/Documents/iCollections/work/python/workspace/bet365_spider/test0810.py", line 11, in
driver = Chrome(chrome_options=option)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/undetected_chromedriver/init.py", line 49, in new
instance.init(*args, **kwargs)
TypeError: init() got an unexpected keyword argument 'options'

Can you know why this is?

Cookie errror

evTools listening on ws://127.0.0.1:54000/devtools/browser/2af26cf7-59cd-49c3-9ae4-f5a8513fdf0c
[0620/205233.599:INFO:CONSOLE(0)] "Access to XMLHttpRequest at 'chrome-extension://jnhgnonknehpejjnehehllkliplmbmhn/images/icon16.png' from origin 'https://geo.captc
ha-delivery.com' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: http, data, chrome, https.", source: https://geo.cap
tcha-delivery.com/captcha/?initialCid=AHrlqAAAAAMAalO8hPIYkaIAWAUn3A%3D%3D&hash=14D062F60A4BDE8CE8647DFC720349&cid=avy1TMopuL3owhfevmncrn39VVTAAsugo_sL6JjQWBNr3~wcNd
FZiWXOwocFBPUgCX3o4tni1hzhVkSE7QXFR.G8SMu9jp1XjStiGWh2el&t=fe (0)
[0620/205233.830:INFO:CONSOLE(0)] "A cookie associated with a cross-site resource at http://captcha-delivery.com/ was set without the SameSite attribute. A future
release of Chrome will only deliver cookies with cross-site requests if they are set with SameSite=None and Secure. You can review cookies in developer tools und
er Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/563352162218803
2.", source: https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/ (0)
[0620/205233.920:INFO:CONSOLE(0)] "A cookie associated with a cross-site resource at http://api-na.geetest.com/ was set without the SameSite attribute. A future re
lease of Chrome will only deliver cookies with cross-site requests if they are set with SameSite=None and Secure. You can review cookies in developer tools under
Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
", source: https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/ (0)
[0620/205234.224:INFO:CONSOLE(1)] "The deviceorientation events are blocked by feature policy. See https://github.com/WICG/feature-policy/blob/master/features.md#sen
sor-features", source: https://static.geetest.com/static/js/fullpage.8.9.6.js (1)

Does it support remotedebugging mode?

I tried to attach driver to an existing chrome process, but failed.
codes :
options = uc.ChromeOptions()
options.add_experimental_option("debuggerAddress", "127.0.0.1:8999")
driver = uc.Chrome(options=options)

error message:
Message: invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: unrecognized chrome option: excludeSwitches

Getting Detected on Amazon Ec2

First off SUPER cool project. Works flawlessly local on mac.

Getting Detected on Amazon Ec2 Instance. My instinct is telling me I'm getting detected by some Amazon/ Ec2 Detection from the site.

Let me know what details I can provide and how I can contribute to the project! :) ps - I'll be uploading the repo I'm working on that uses yours some time today, just have to package.

Detected by Distill Network

I was trying to scrap g2.com. First time i went to the website it works fine, but the second time i return it detects suspicious activity. The message it pops is:

Pardon Our Interruption...
As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

You're a power user moving through this website with super-human speed.
You've disabled JavaScript and/or cookies in your web browser.
A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article.
After completing the CAPTCHA below, you will immediately regain access to the site again

Brief:
The fact is that i did not moved fast at all, just tried to acces a page. First time it worked, second time i got caught.

New protection, not bypassable?

Nice work on this one!

There are two websites: https://www.mc-market.org and https://v3rmillion.net, it doesnt seem to work on these ones, what I see in the network traffic is that there are some 503 errors and the page keeps reloading.

Seems you are an expert in this field, do you have any idea what this could be?

EDIT: Found another site with this protection, seems like its from cloudflare: https://www.sythe.org/login/login

Google seems to be able to detect WebDriver usage somehow

Hello!

This is a great project and seems to work great indeed for sites that use Distil etc, but, it seems that Google itself is able to detect it.

This can be observed by going to the Google login page at e.g. https://accounts.google.com/, entering a username, and clicking next. When the browser is being controlled by the Web Driver (even with the undetected-chromedriver) it will take you to a "This browser is not safe" page, and not allow you to login.

Note that they seem to be using some kind of heuristic system to decide whether or not to block the login attempt, so when trying to login to certain accounts, it will actually let you through (either with undetected-chromedriver or with the default one). In order to reproduce the issue it is thus best to pick an account at random (e.g., "randomaccount5", or any name that exists).

Do you know how they could be detecting it, and/or whether it would also be possible to implement countermeasures with this project?

I've tried to have a look at the Google login code without much luck (it's somewhat obfuscated and all), but you're definitely an expert at this.

Thanks & best regards!

Cloudfare protection

The driver is still detected by cloudfare protection, Platzi is a website im trying to web-scrap and i haven't been able to.

Recommendation on variable replacement

Just thinking out loud about the latest changes: I wouldn't use a hardcoded replacement like this, because it is also very easy to block, just like "cdc_adoQpo[...]" (if any anti-bot service follows this repo). Instead, I would generate a random string (equal in length to the original string) and use that, as a replacement.

However I do love the fact and the easter egg, that you used "undetectedchromeDRiver" as it is exactly, perfectly the same length as the original, made up variable.

[AliExpress Bonus Buddies] Selenium Getting Dectected

So, I did some search like you said, I use the vanilla way tested everything.. it's undetected on most of the bot protection site.
But as soon I use driver.get('https://sale.aliexpress.com/__mobile/daily-cash-out-gaming_m.htm?_addShare=no') or the ref link.
AliExpress blocks my account as it detect it, now after even if I change the ip and anything it will always give system error
So basically account become useless.

I get this system error, no use as it I even do a manually click after opening the ref link it will give this error, but If i paste the link manually it will work!
Link: Will only work in mobile view!

solved

Cant see individual requests?

Im trying to see each individual request while using this like in normal Selenium to sort by method, etc but it says
for request in driver.requests: AttributeError: 'WebDriver' object has no attribute 'requests'

Is there something else i should do to sort requests? And how do i print headers? Cookies works fine by doing .get_cookies() but headers doesnt work whichever way i try?

Using proxy server authentication

Is there any way we can include proxy auth from selenium wire and combine it with undetected chromedriver?

[question] Patching binary

Hello mate,

I would like to thank your amazing work, I do understand everything you do and its absolutely brilliant. I already investigated this a while ago but used a really worse solution than you that was using a MITM proxy and injecting javascript on each request to remove webdriver property etc.

I have one question though, why do you patch the binary and what do you exactly do with that? I see you modify the binary but don't understand how.

Thanks a lot,
Aleix

Got a question!

edit: kinda solved it myself, so it's whatever

Error : undetected_chromedriver/init.py", line 49, in new instance.init(*args, **kwargs) TypeError: init() got an unexpected keyword argument 'options'

Hi

My environement : windows server 2012, Chrome Last version, python 3.7 & python 3.8

My Code :
import undetected_chromedriver as uc
driver = uc.Chrome()
driver.get('https://distilnetworks.com')

The Error :
/python3.8/site-packages/undetected_chromedriver/init.py", line 49, in new instance.init(*args, **kwargs) TypeError: init() got an unexpected keyword argument 'options'

This is the screenshot :
https://i.ibb.co/PMPts9L/Error-bot.png

NOTE :
i have two windows server 2012 with the same configuration and same chrome & python & same code
one of them work well, and the other i have the error.

Thank you

Need ur discord

Hey @ultrafunkamsterdam can u give me ur discord
mine is TECHTANIC#8090

Detection.

Hello! Tested now your chromedriver and it's detected on nike.com

Thank you :)

Hello,

I just want to thank you for the amazing work you have done with this custom version of selenium. it is really awesome.

thanks again!

execute_script gets detected

On websites protected by Distill, like endclothing, they can detect if you execute javascript from the method "execute_script" instead of executing it directly on the console. Is it possible to avoid this detection? I see that selenium applies some scripts before executing mine.
Thanks.