seralexger/selenium-capturetraffic

Simple class to capture http and https traffic using Selenium and Browsermob proxy

Selenium CaptureTraffic

I have built a simple class in python to capture http and https traffic with Selenium and Browsermob proxy.

Getting Started

I develop in mac, if you develop in linux or windows you will have to replace the geckodriver with the one appropriate for your system. You can download from (https://github.com/mozilla/geckodriver/releases). You must have java(jdk) installed to be able to use it from the command line.

All the necesary libraries for run the project are in the requirements.txt.

Use of the class

There is a simple practical example in capture_example.py.

from capturetraffic import CaptureTraffic


sniffer = CaptureTraffic()
data = sniffer.capture_traffic(self, url, wait_time = 30, save = False, save_path = 'data/xhr_re.json')

Data scheme example

Inside entries there is an array with the requests and its responses.

{
    "log": {
        "version": "1.2",
        "creator": {
            "name": "BrowserMob Proxy",
            "version": "2.1.4",
            "comment": ""
        },
        "pages": [
            {
                "id": "Page 0",
                "startedDateTime": "2018-08-29T21:24:10.862+02:00",
                "title": "Page 0",
                "pageTimings": {
                    "comment": ""
                },
                "comment": ""
            }
        ],
        "entries": []
        
    }
}

selenium-capturetraffic's Issues

Unable to find a matching set of capabilities

Hi there,

I tried and the script failed.

[root@shell selenium-capturetraffic]# python3.6 capture_example.py
Traceback (most recent call last):
File "capture_example.py", line 15, in
sniffer = CaptureTraffic()
File "/root/xpto/selenium-capturetraffic/capturetraffic.py", line 38, in init
self.start_session()
File "/root/xpto/selenium-capturetraffic/capturetraffic.py", line 79, in start_session
self.start_driver()
File "/root/xpto/selenium-capturetraffic/capturetraffic.py", line 69, in start_driver
self.driver = webdriver.Firefox(options=options, firefox_profile=profile, executable_path='utils/geckodriver')
File "/usr/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 167, in init
keep_alive=True)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 156, in init
self.start_session(capabilities, browser_profile)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 251, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities

[root@shell selenium-capturetraffic]

Thanks