chrome-php/chrome

Instrument headless chrome/chromium instances from PHP

Chrome PHP

This library lets you start playing with chrome/chromium in headless mode from PHP.

Can be used synchronously and asynchronously!

Features

• Open Chrome or Chromium browser from php
• Create pages and navigate to pages
• Take screenshots
• Evaluate javascript on the page
• Make PDF
• Emulate mouse
• Emulate keyboard
• Always IDE friendly

Happy browsing!

Requirements

Requires PHP 7.4-8.3 and a Chrome/Chromium 65+ executable.

Note that the library is only tested on Linux but is compatible with macOS and Windows.

Installation

The library can be installed with Composer and is available on packagist under chrome-php/chrome:

$ composer require chrome-php/chrome

Usage

It uses a simple and understandable API to start Chrome, to open pages, take screenshots, crawl websites... and almost everything that you can do with Chrome as a human.

use HeadlessChromium\BrowserFactory;

$browserFactory = new BrowserFactory();

// starts headless Chrome
$browser = $browserFactory->createBrowser();

try {
    // creates a new page and navigate to an URL
    $page = $browser->createPage();
    $page->navigate('http://example.com')->waitForNavigation();

    // get page title
    $pageTitle = $page->evaluate('document.title')->getReturnValue();

    // screenshot - Say "Cheese"! 😄
    $page->screenshot()->saveToFile('/foo/bar.png');

    // pdf
    $page->pdf(['printBackground' => false])->saveToFile('/foo/bar.pdf');
} finally {
    // bye
    $browser->close();
}

Using different Chrome executable

When starting, the factory will look for the environment variable "CHROME_PATH" to use as the Chrome executable. If the variable is not found, it will try to guess the correct executable path according to your OS or use "chrome" as the default.

You are also able to explicitly set up any executable of your choice when creating a new object. For instance "chromium-browser":

use HeadlessChromium\BrowserFactory;

// replace default 'chrome' with 'chromium-browser'
$browserFactory = new BrowserFactory('chromium-browser');

Debugging

The following example disables headless mode to ease debugging

use HeadlessChromium\BrowserFactory;

$browserFactory = new BrowserFactory();

$browser = $browserFactory->createBrowser([
    'headless' => false, // disable headless mode
]);

Other debug options:

[
    'connectionDelay' => 0.8,            // add 0.8 second of delay between each instruction sent to Chrome,
    'debugLogger'     => 'php://stdout', // will enable verbose mode
]

About debugLogger: this can be any of a resource string, a resource, or an object implementing LoggerInterface from Psr\Log (such as monolog or apix/log).

API

Browser Factory

Options set directly in the createBrowser method will be used only for a single browser creation. The default options will be ignored.

use HeadlessChromium\BrowserFactory;

$browserFactory = new BrowserFactory();
$browser = $browserFactory->createBrowser([
    'windowSize'   => [1920, 1000],
    'enableImages' => false,
]);

// this browser will be created without any options
$browser2 = $browserFactory->createBrowser();

Options set using the setOptions and addOptions methods will persist.

$browserFactory->setOptions([
    'windowSize' => [1920, 1000],
]);

// both browser will have the same 'windowSize' option
$browser1 = $browserFactory->createBrowser();
$browser2 = $browserFactory->createBrowser();

$browserFactory->addOptions(['enableImages' => false]);

// this browser will have both the 'windowSize' and 'enableImages' options
$browser3 = $browserFactory->createBrowser();

$browserFactory->addOptions(['enableImages' => true]);

// this browser will have the previous 'windowSize', but 'enableImages' will be true
$browser4 = $browserFactory->createBrowser();

Available options

Here are the options available for the browser factory:

Option name	Default	Description
connectionDelay	0	Delay to apply between each operation for debugging purposes
customFlags	none	An array of flags to pass to the command line. Eg: ['--option1', '--option2=someValue']
debugLogger	null	A string (e.g "php://stdout"), or resource, or PSR-3 logger instance to print debug messages
disableNotifications	false	Disable browser notifications
enableImages	true	Toggles loading of images
envVariables	none	An array of environment variables to pass to the process (example DISPLAY variable)
headers	none	An array of custom HTTP headers
headless	true	Enable or disable headless mode
ignoreCertificateErrors	false	Set Chrome to ignore SSL errors
keepAlive	false	Set to true to keep alive the Chrome instance when the script terminates
noSandbox	false	Enable no sandbox mode, useful to run in a docker container
noProxyServer	false	Don't use a proxy server, always make direct connections. Overrides other proxy settings.
proxyBypassList	none	Specifies a list of hosts for whom we bypass proxy settings and use direct connections
proxyServer	none	Proxy server to use. usage: 127.0.0.1:8080 (authorisation with credentials does not work)
sendSyncDefaultTimeout	5000	Default timeout (ms) for sending sync messages
startupTimeout	30	Maximum time in seconds to wait for Chrome to start
userAgent	none	User agent to use for the whole browser (see page API for alternative)
userDataDir	none	Chrome user data dir (default: a new empty dir is generated temporarily)
userCrashDumpsDir	none	The directory crashpad should store dumps in (crash reporter will be enabled automatically)
windowSize	none	Size of the window. usage: $width, $height - see also Page::setViewport
excludedSwitches	none	An array of Chrome flags that should be removed from the default set (example --enable-automation)

Persistent Browser

This example shows how to share a single instance of Chrome for multiple scripts.

The first time the script is started we use the browser factory in order to start Chrome, afterwards we save the uri to connect to this browser in the file system.

The next calls to the script will read the uri from that file in order to connect to the Chrome instance instead of creating a new one. If Chrome was closed or crashed, a new instance is started again.

use \HeadlessChromium\BrowserFactory;
use \HeadlessChromium\Exception\BrowserConnectionFailed;

// path to the file to store websocket's uri
$socket = \file_get_contents('/tmp/chrome-php-demo-socket');

try {
    $browser = BrowserFactory::connectToBrowser($socket);
} catch (BrowserConnectionFailed $e) {
    // The browser was probably closed, start it again
    $factory = new BrowserFactory();
    $browser = $factory->createBrowser([
        'keepAlive' => true,
    ]);

    // save the uri to be able to connect again to browser
    \file_put_contents($socketFile, $browser->getSocketUri(), LOCK_EX);
}

Browser API

Create a new page (tab)

$page = $browser->createPage();

Get opened pages (tabs)

$pages = $browser->getPages();

Close the browser

$browser->close();

Set a script to evaluate before every page created by this browser will navigate

$browser->setPagePreScript('// Simulate navigator permissions;
const originalQuery = window.navigator.permissions.query;
window.navigator.permissions.query = (parameters) => (
    parameters.name === 'notifications' ?
        Promise.resolve({ state: Notification.permission }) :
        originalQuery(parameters)
);');

Page API

Navigate to an URL

// navigate
$navigation = $page->navigate('http://example.com');

// wait for the page to be loaded
$navigation->waitForNavigation();

When using $navigation->waitForNavigation() you will wait for 30sec until the page event "loaded" is triggered. You can change the timeout or the event to listen for:

use HeadlessChromium\Page;

// wait 10secs for the event "DOMContentLoaded" to be triggered
$navigation->waitForNavigation(Page::DOM_CONTENT_LOADED, 10000);

Available events (in the order they trigger):

• Page::DOM_CONTENT_LOADED: dom has completely loaded
• Page::FIRST_CONTENTFUL_PAINT: triggered when the first non-white content element is painted on the screen
• Page::FIRST_IMAGE_PAINT: triggered when the first image is painted on the screen
• Page::FIRST_MEANINGFUL_PAINT: triggered when the primary content of a page is visible to the user
• Page::FIRST_PAINT: triggered when any pixel on the screen is painted, including the browser's default background color
• Page::INIT: connection to DevTools protocol is initialized
• Page::INTERACTIVE_TIME: scripts have finished loading and the main thread is no longer blocked by rendering or other tasks
• Page::LOAD: (default) page and all resources are loaded
• Page::NETWORK_IDLE: page has loaded, and no network activity has occurred for at least 500ms

When you want to wait for the page to navigate 2 main issues may occur. First, the page is too long to load and second, the page you were waiting to be loaded has been replaced. The good news is that you can handle those issues using a good old try-catch:

use HeadlessChromium\Exception\OperationTimedOut;
use HeadlessChromium\Exception\NavigationExpired;

try {
    $navigation->waitForNavigation()
} catch (OperationTimedOut $e) {
    // too long to load
} catch (NavigationExpired $e) {
    // An other page was loaded
}

Evaluate script on the page

Once the page has completed the navigation you can evaluate arbitrary script on this page:

// navigate
$navigation = $page->navigate('http://example.com');

// wait for the page to be loaded
$navigation->waitForNavigation();

// evaluate script in the browser
$evaluation = $page->evaluate('document.documentElement.innerHTML');

// wait for the value to return and get it
$value = $evaluation->getReturnValue();

Sometimes the script you evaluate will click a link or submit a form, in this case, the page will reload and you will want to wait for the new page to reload.

You can achieve this by using $page->evaluate('some js that will reload the page')->waitForPageReload(). An example is available in form-submit.php

Call a function

This is an alternative to evaluate that allows calling a given function with the given arguments in the page context:

$evaluation = $page->callFunction(
    "function(a, b) {\n    window.foo = a + b;\n}",
    [1, 2]
);

$value = $evaluation->getReturnValue();

Add a script tag

That's useful if you want to add jQuery (or anything else) to the page:

$page->addScriptTag([
    'content' => file_get_contents('path/to/jquery.js')
])->waitForResponse();

$page->evaluate('$(".my.element").html()');

You can also use an URL to feed the src attribute:

$page->addScriptTag([
    'url' => 'https://code.jquery.com/jquery-3.3.1.min.js'
])->waitForResponse();

$page->evaluate('$(".my.element").html()');

Set the page HTML

You can manually inject html to a page using the setHtml method.

// Basic
$page->setHtml('<p>text</p>');

// Specific timeout & event
$page->setHtml('<p>text</p>', 10000, Page::NETWORK_IDLE);

When a page's HTML is updated, we'll wait for the page to unload. You can specify how long to wait and which event to wait for through two optional parameters. This defaults to 3000ms and the "load" event.

Note that this method will not append to the current page HTML, it will completely replace it.

Get the page HTML

You can get the page HTML as a string using the getHtml method.

$html = $page->getHtml();

Add a script to evaluate upon page navigation

$page->addPreScript('// Simulate navigator permissions;
const originalQuery = window.navigator.permissions.query;
window.navigator.permissions.query = (parameters) => (
    parameters.name === 'notifications' ?
        Promise.resolve({ state: Notification.permission }) :
        originalQuery(parameters)
);');

If your script needs the dom to be fully populated before it runs then you can use the option "onLoad":

$page->addPreScript($script, ['onLoad' => true]);

Set viewport size

This feature allows changing the size of the viewport (emulation) for the current page without affecting the size of all the browser's pages (see also option "windowSize" of BrowserFactory::createBrowser).

$width = 600;
$height = 300;
$page->setViewport($width, $height)
    ->await(); // wait for the operation to complete

Make a screenshot

// navigate
$navigation = $page->navigate('http://example.com');

// wait for the page to be loaded
$navigation->waitForNavigation();

// take a screenshot
$screenshot = $page->screenshot([
    'format'  => 'jpeg',  // default to 'png' - possible values: 'png', 'jpeg', 'webp'
    'quality' => 80,      // only when format is 'jpeg' or 'webp' - default 100
    'optimizeForSpeed' => true // default to 'false' - Optimize image encoding for speed, not for resulting size
]);

// save the screenshot
$screenshot->saveToFile('/some/place/file.jpg');

Screenshot an area on a page

You can use the option "clip" to choose an area on a page for the screenshot

use HeadlessChromium\Clip;

// navigate
$navigation = $page->navigate('http://example.com');

// wait for the page to be loaded
$navigation->waitForNavigation();

// create a rectangle by specifying to left corner coordinates + width and height
$x = 10;
$y = 10;
$width = 100;
$height = 100;
$clip = new Clip($x, $y, $width, $height);

// take the screenshot (in memory binaries)
$screenshot = $page->screenshot([
    'clip'  => $clip,
]);

// save the screenshot
$screenshot->saveToFile('/some/place/file.jpg');

Full-page screenshot

You can also take a screenshot for the full-page layout (not only the viewport) using $page->getFullPageClip with attribute captureBeyondViewport = true

// navigate
$navigation = $page->navigate('https://example.com');

// wait for the page to be loaded
$navigation->waitForNavigation();

$screenshot = $page->screenshot([
    'captureBeyondViewport' => true,
    'clip' => $page->getFullPageClip(),
    'format' => 'jpeg', // default to 'png' - possible values: 'png', 'jpeg', 'webp'
]);

// save the screenshot
$screenshot->saveToFile('/some/place/file.jpg');

Print as PDF

// navigate
$navigation = $page->navigate('http://example.com');

// wait for the page to be loaded
$navigation->waitForNavigation();

$options = [
    'landscape'           => true,             // default to false
    'printBackground'     => true,             // default to false
    'displayHeaderFooter' => true,             // default to false
    'preferCSSPageSize'   => true,             // default to false (reads parameters directly from @page)
    'marginTop'           => 0.0,              // defaults to ~0.4 (must be a float, value in inches)
    'marginBottom'        => 1.4,              // defaults to ~0.4 (must be a float, value in inches)
    'marginLeft'          => 5.0,              // defaults to ~0.4 (must be a float, value in inches)
    'marginRight'         => 1.0,              // defaults to ~0.4 (must be a float, value in inches)
    'paperWidth'          => 6.0,              // defaults to 8.5 (must be a float, value in inches)
    'paperHeight'         => 6.0,              // defaults to 11.0 (must be a float, value in inches)
    'headerTemplate'      => '<div>foo</div>', // see details above
    'footerTemplate'      => '<div>foo</div>', // see details above
    'scale'               => 1.2,              // defaults to 1.0 (must be a float)
];

// print as pdf (in memory binaries)
$pdf = $page->pdf($options);

// save the pdf
$pdf->saveToFile('/some/place/file.pdf');

// or directly output pdf without saving
header('Content-Description: File Transfer');
header('Content-Type: application/pdf');
header('Content-Disposition: inline; filename=filename.pdf');
header('Content-Transfer-Encoding: binary');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Pragma: public');

echo base64_decode($pdf->getBase64());

Options headerTemplate and footerTemplate:

Should be valid HTML markup with the following classes used to inject printing values into them:

• date: formatted print date
• title: document title
• url: document location
• pageNumber: current page number
• totalPages: total pages in the document

Save downloads

You can set the path to save downloaded files.

// After creating a page.
$page->setDownloadPath('/path/to/save/downloaded/files');

Mouse API

The mouse API is dependent on the page instance and allows you to control the mouse's moves, clicks and scroll.

$page->mouse()
    ->move(10, 20)                             // Moves mouse to position x=10; y=20
    ->click()                                  // left-click on position set above
    ->move(100, 200, ['steps' => 5])           // move mouse to x=100; y=200 in 5 equal steps
    ->click(['button' => Mouse::BUTTON_RIGHT]; // right-click on position set above

// given the last click was on a link, the next step will wait
// for the page to load after the link was clicked
$page->waitForReload();

You can emulate the mouse wheel to scroll up and down in a page, frame, or element.

$page->mouse()
    ->scrollDown(100) // scroll down 100px
    ->scrollUp(50);   // scroll up 50px

Finding elements

The find method will search for elements using querySelector and move the cursor to a random position over it.

try {
    $page->mouse()->find('#a')->click(); // find and click at an element with id "a"

    $page->mouse()->find('.a', 10); // find the 10th or last element with class "a"
} catch (ElementNotFoundException $exception) {
    // element not found
}

This method will attempt to scroll right and down to bring the element to the visible screen. If the element is inside an internal scrollable section, try moving the mouse to inside that section first.

Keyboard API

The keyboard API is dependent on the page instance and allows you to type like a real user.

$page->keyboard()
    ->typeRawKey('Tab') // type a raw key, such as Tab
    ->typeText('bar');  // type the text "bar"

To impersonate a real user you may want to add a delay between each keystroke using the setKeyInterval method:

$page->keyboard()->setKeyInterval(10); // sets a delay of 10 milliseconds between keystrokes

Key combinations

The methods press, type, and release can be used to send key combinations such as ctrl + v.

// ctrl + a to select all text
$page->keyboard()
    ->press('control') // key names are case insensitive and trimmed
        ->type('a')    // press and release
    ->release('Control');

// ctrl + c to copy and ctrl + v to paste it twice
$page->keyboard()
    ->press('Ctrl') // alias for Control
        ->type('c')
        ->type('V') // upper and lower cases should behave the same way
    ->release();    // release all

You can press the same key several times in sequence, this is the equivalent to a user pressing and holding the key. The release event, however, will be sent only once per key.

Key aliases

Key	Aliases
Control	Control, Ctrl, Ctr
Alt	Alt, AltGr, Alt Gr
Meta	Meta, Command, Cmd
Shift	Shift

Cookie API

You can set and get cookies for a page:

Set Cookie

use HeadlessChromium\Cookies\Cookie;

$page = $browser->createPage();

// example 1: set cookies for a given domain

$page->setCookies([
    Cookie::create('name', 'value', [
        'domain' => 'example.com',
        'expires' => time() + 3600 // expires in 1 hour
    ])
])->await();


// example 2: set cookies for the current page

$page->navigate('http://example.com')->waitForNavigation();

$page->setCookies([
    Cookie::create('name', 'value', ['expires'])
])->await();

Get Cookies

use HeadlessChromium\Cookies\Cookie;

$page = $browser->createPage();

// example 1: get all cookies for the browser

$cookies = $page->getAllCookies();

// example 2: get cookies for the current page

$page->navigate('http://example.com')->waitForNavigation();
$cookies = $page->getCookies();

// filter cookies with name == 'foo'
$cookiesFoo = $cookies->filterBy('name', 'foo');

// find first cookie with name == 'bar'
$cookieBar = $cookies->findOneBy('name', 'bar');
if ($cookieBar) {
    // do something
}

Set user agent

You can set up a user-agent per page:

$page->setUserAgent('my user-agent');

See also BrowserFactory option userAgent to set up it for the whole browser.

Advanced usage

The library ships with tools that hide all the communication logic but you can use the tools used internally to communicate directly with Chrome debug protocol.

Example:

use HeadlessChromium\Communication\Connection;
use HeadlessChromium\Communication\Message;

// Chrome devtools URI
$webSocketUri = 'ws://127.0.0.1:9222/devtools/browser/xxx';

// create a connection
$connection = new Connection($webSocketUri);
$connection->connect();

// send method "Target.activateTarget"
$responseReader = $connection->sendMessage(new Message('Target.activateTarget', ['targetId' => 'xxx']));

// wait up to 1000ms for a response
$response = $responseReader->waitForResponse(1000);

Create a session and send a message to the target

// given a target id
$targetId = 'yyy';

// create a session for this target (attachToTarget)
$session = $connection->createSession($targetId);

// send message to this target (Target.sendMessageToTarget)
$response = $session->sendMessageSync(new Message('Page.reload'));

Debugging

You can ease the debugging by setting a delay before each operation is made:

  $connection->setConnectionDelay(500); // wait for 500ms between each operation to ease debugging

Browser (standalone)

use HeadlessChromium\Communication\Connection;
use HeadlessChromium\Browser;

// Chrome devtools URI
$webSocketUri = 'ws://127.0.0.1:9222/devtools/browser/xxx';

// create connection given a WebSocket URI
$connection = new Connection($webSocketUri);
$connection->connect();

// create browser
$browser = new Browser($connection);

Interacting with DOM

Find one element on a page by CSS selector:

$page = $browser->createPage();
$page->navigate('http://example.com')->waitForNavigation();

$elem = $page->dom()->querySelector('#index_email');

Find all elements inside another element by CSS selector:

$elem = $page->dom()->querySelector('#index_email');
$elem->querySelectorAll('a.link');

Find all elements on a page by XPath selector:

$page = $browser->createPage();
$page->navigate('http://example.com')->waitForNavigation();

$elem = $page->dom()->search('//div/*/a');

Wait for an element by CSS selector:

$page = $browser->createPage();
$page->navigate('http://example.com')->waitForNavigation();

$page->waitUntilContainsElement('div[data-name=\"el\"]');

If a string is passed to Page::waitUntilContainsElement, an instance of CSSSelector is created for you by Page::waitForElement. To use other selectors, you can pass an instance of the required Selector.

Wait for element by XPath selector:

use HeadlessChromium\Dom\Selector\XPathSelector;

$page = $browser->createPage();
$page->navigate('http://example.com')->waitForNavigation();

$page->waitUntilContainsElement(new XPathSelector('//div[contains(text(), "content")]'));

You can send out a text to an element or click on it:

$elem->click();
$elem->sendKeys('Sample text');

You can upload file to file from the input:

$elem->sendFile('/path/to/file');

You can get element text or attribute:

$text = $elem->getText();
$attr = $elem->getAttribute('class');

Contributing

See CONTRIBUTING.md for contribution details.

License

This project is licensed under the The MIT License (MIT).

chrome's Issues

Keyboard support or event with "istrusted"

Hi
would it be possible to add support to keyboard events or the possibility to trigger events that are "trusted", as keyboards events 'or other kinds of events) triggered by javascript can be detected as "not trusted" and not accepted by the website

Add a form to a page and submit

$evaluation = $page->evaluate( '(() => { document.body.innerHTML += "<form id=\""dynForm\"" action=\""http://example.com/\"" method=\""post\""></form>"; document.querySelector("#dynForm").submit(); })()' );

is this the inccorect way to add it to the page?
document.body.innerHTML += "<

Using the chrome console it works.

The page has navigated to an other page and this navigation expired

Good day all,

The following was installed;

root@server003:/var/www/html_test# composer require chrome-php/chrome
sudo apt-get install chromium-browser

The following works perfectly:

echo '<pre>';
require_once('vendor/autoload.php');
use HeadlessChromium\BrowserFactory;
$factory = new BrowserFactory('chromium-browser --ignore-certificate-errors');
$browser = $factory->createBrowser([
      'debugLogger'     => 'php://stdout'
]);
$page = $browser->createPage();
$page->navigate('https://www.google.nl/')->waitForNavigation();
$value = $page->evaluate('document.querySelector("input[name=\"btnK\"]").value')->getReturnValue();
var_dump($value);

But then when I run the following:

echo '<pre>';
require_once('vendor/autoload.php');
use HeadlessChromium\BrowserFactory;
$factory = new BrowserFactory('chromium-browser --ignore-certificate-errors');
$browser = $factory->createBrowser([
		'debugLogger'     => 'php://stdout'
]);

$page = $browser->createPage();
$page->navigate('https://ip-address:8443/login')->waitForNavigation();
$evaluation = $page->evaluate(
'(() => {
        document.querySelector("#username").value = "***********";
        document.querySelector("#password").value = "***********";
        document.querySelector("form[name=\"login\"]").submit();
    })()'
);
$evaluation->waitForPageReload();
$value = $page->evaluate('document.querySelector(".duration .ng-binding").innerHTML')->getReturnValue();
var_dump($value);

The following error is generated:

Fatal error:  Uncaught HeadlessChromium\Exception\NavigationExpired: The page has navigated to an other page and this navigation expired in /var/www/html_test/vendor/chrome-php/chrome/src/PageUtils/PageNavigation.php:165
Stack trace:
#0 /var/www/html_test/vendor/chrome-php/chrome/src/Utils.php(51): HeadlessChromium\PageUtils\PageNavigation->navigationComplete('load')
#1 /var/www/html_test/vendor/chrome-php/chrome/src/PageUtils/PageNavigation.php(109): HeadlessChromium\Utils::tryWithTimeout(30000000, Object(Generator))
#2 /var/www/html_test/tst.php(41): HeadlessChromium\PageUtils\PageNavigation->waitForNavigation()
#3 {main}
  thrown in /var/www/html_test/vendor/chrome-php/chrome/src/PageUtils/PageNavigation.php on line 165

Looking through the closed issues, I modified the createBrowser() call like this:

$browser = $factory->createBrowser([
		'headless'        => false,         // disable headless mode
		'connectionDelay' => 0.8,           // add 0.8 second of delay between each instruction sent to chrome,
		'debugLogger'     => 'php://stdout'
]);

Now the following is shown:

Fatal error:  Uncaught HeadlessChromium\Exception\NavigationExpired: The page has navigated to an other page and this navigation expired in /var/www/html_test/vendor/chrome-php/chrome/src/PageUtils/PageNavigation.php:165
Stack trace:
#0 /var/www/html_test/vendor/chrome-php/chrome/src/Utils.php(51): HeadlessChromium\PageUtils\PageNavigation->navigationComplete('load')
#1 /var/www/html_test/vendor/chrome-php/chrome/src/PageUtils/PageNavigation.php(109): HeadlessChromium\Utils::tryWithTimeout(30000000, Object(Generator))
#2 /var/www/html_test/tst.php(41): HeadlessChromium\PageUtils\PageNavigation->waitForNavigation()
#3 {main}
  thrown in /var/www/html_test/vendor/chrome-php/chrome/src/PageUtils/PageNavigation.php on line 165

Does anyone have any ideas?

Thank you very much.

Waiting for javascript variable

Hey dude, really cool work!

Is it possible to wait for javascript evaluation (poll for a global variable) rather than navigation such as Page::NETWORK_IDLE?

I'm faced with an interesting issue where I pull a lot of data via ajax, and it takes more than 1000ms to populate the data in the dom (fancy charts etc) therefore Page::NETWORK_IDLE evacuates to true and the image/pdf is captured before the next ajax request fires for the final piece of content.

Thanks!

Page navigation timeout is not parametrized

Hello there 👋

thank you for this cool package in advance, i'm using it to access information from sites which do not offer APIs and it is a very helpful tool.

On of the things i require to be able is to set the timeout when calling \HeadlessChromium\Page::navigate.
Currently there is no parameter, therefor \HeadlessChromium\Communication\Connection::$sendSyncDefaultTimeout will be used which can't be set and is hardcoded to 3 seconds. When downloading files from some pages the download might take longer than 3 seconds (because the file is big/needs to be generated by the server first) and $page->navigate($downloadUrl); will throw an exception.

I will provide a non-breaking pull request shortly and i'd be glad if you could merge it into your awesome lib 🙂

auth

how to do basic auth with headless chrome?

screenshot DOM object

Is this library ready enough to screenshot DOM objects? I have a project that has a requirement to take full page screenshots as well as screenshots of specific DOM objects, I have been using PhantomJS to do this but its just to slow. if this is not supported yet is it possible to use things like page.evaluate with"getBoundingClientRect()" to return the object position then snip the screenshot?

chrome: command not found

Hi guys,

Got chrome installed on my local env which is OSX running php 7.2. I've got chrome installed with an aliase to the following

alias chrome="/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome"
chrome version is "Google Chrome 66.0.3359.117 "

but i Keep getting this stack trace

[2018-04-25 15:59:14] local.ERROR: Cannot get chrome version, make sure you provided the correct chrome binaries using (chrome). sh: chrome: command not found {"exception":"[object] (RuntimeException(code: 0): Cannot get chrome version, make sure you provided the correct chrome binaries using (chrome). sh: chrome: command not found at /Users/dansmacbook/projects/project-scrapper/vendor/chrome-php/chrome/src/BrowserFactory.php:78)

code execution is this

  $browser = $browserFactory->createBrowser([
                'headless'        => true,         
                'connectionDelay' => 0.8,           
                'debugLogger'     => 'php://stdout'
            ]);
        $page = $browser->createPage();
        $response = $page->navigate($url)->waitForNavigation();
        $browser->close();
        dd($response);

any thoughts on how to resolve this?

How to: set user agent

Hi there,

Would it be possible to let chromium act as different browser by setting the user agent?

Some questions about your project

1. can I set cookies for any domain? - if so how?
2. Install user agent? - if so how?
3. Does the current stable version support working via proxy? - if so how?
4. Getting cookies that have been set by the site? - if so how?

User Agent error

$browser = $browserFactory->createBrowser(['userAgent' => 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko']);

Wil return:


Fatal error: Uncaught RuntimeException: Chrome process stopped before startup completed. Additional info: sh: 1: Syntax error: "(" unexpected in /opt/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php:355
Stack trace:
#0 /opt/vendor/chrome-php/chrome/src/Utils.php(51): HeadlessChromium\Browser\BrowserProcess->HeadlessChromium\Browser{closure}(Object(Symfony\Component\Process\Process))
#1 /opt/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php(390): HeadlessChromium\Utils::tryWithTimeout(30000000, Object(Generator))
#2 /opt/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php(125): HeadlessChromium\Browser\BrowserProcess->waitForStartup(Object(Symfony\Component\Process\Process), 30000000)
#3 /opt/vendor/chrome-php/chrome/src/BrowserFactory.php(80): HeadlessChromium\Browser\BrowserProcess->start('chromium-browse...', Array)
#4 /var/www/html/domains/x/public_html/x/index.php(57): HeadlessChromium\BrowserFactory->createBrowser(Array)
#5 {main}
thrown in /opt/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php on line 355

evaluate quote problem

$evaluation = $page->evaluate(
'(() => {
element1.setAttribute("name", 'I need to have a quote " in here');

the ' quote is not working so I end up with:
element1.setAttribute("name", "I need to have a quote " in here");

But of course that doesn't work.
Any tips advice?

constantly getting a no target with given id found

This only happens when using the userData, I need to keep the environment instead of having an entire fresh instance created on every load.

[2018-09-29 00:03:49] DEBUG socket(1): ← receiving data:{"error":{"code":-32602,"message":"No target with given id found"},"id":22}

Which eventually breaks the Session constructor because the sessionId is not returned.

PHP Fatal error:  Uncaught TypeError: Argument 2 passed to HeadlessChromium\Communication\Session::__construct() must be of the type string, null given, called in .../vendor/chrome-php/chrome/src/Communication/Connection.php on line 240 and defined in .../vendor/chrome-php/chrome/src/Communication/Session.php:42

Current workaround:

src/Communication/Connection.php
commented out the return type of createSession so I can:

$sessionId = $response['result']['sessionId'] ?? false;
if(!$sessionId)
  return;

src/Browser.php
Not creating a target if no session is returned

if(!$session)
  return;

Not sure if it's the best approach as I just started to fiddle around with this library...
I will update once I track everything down.

Great work by the way!

DOM interaction

Does this library support DOM interaction? Ex. click events?

Noobie question how to install this?

Hi guys,

So I have a apache with php 7 installed on a Ubuntu VPS.
I installed the library to the html directory of apache and created a php page.
On the page I have: require 'vendor/autoload.php';
After that the example code of the installation page.
I get the error:

Fatal error: Uncaught RuntimeException: Chrome process stopped before startup completed in /var/www/html/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php:326 Stack trace: #0 /var/www/html/vendor/chrome-php/chrome/src/Utils.php(51): HeadlessChromium\Browser\BrowserProcess->HeadlessChromium\Browser{closure}(Object(Symfony\Component\Process\Process)) #1 /var/www/html/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php(361): HeadlessChromium\Utils::tryWithTimeout(30000000, Object(Generator)) #2 /var/www/html/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php(124): HeadlessChromium\Browser\BrowserProcess->waitForStartup(Object(Symfony\Component\Process\Process), 30000000) #3 /var/www/html/vendor/chrome-php/chrome/src/BrowserFactory.php(59): HeadlessChromium\Browser\BrowserProcess->start('chrome', Array) #4 /var/www/html/index.php(16): HeadlessChromium\BrowserFactory->createBrowser() #5 {main} thrown in /var/www/html/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php on line 326

Any idea what to do?

Set window size and viewport size

Hi,

would it be possible to set width (and height or height = fullpage, ...) für die screenshot() function?

best regards
stefan

inject jquery

hi,
is it possible to inject jquery to get DOM elements values more easy, ie to use jquery selectors?

smth like this
$page->addPreScript(file_get_contents ('jquery-latest.min.js'));
$page->navigate('https://www.somepage.com/feed.html')->waitForNavigation();
$value = $page->evaluate('jQuery("#pagelet_group_mall")')->getReturnValue();

Fatal error: Uncaught HeadlessChromium\Exception\JavascriptException: Error during javascript evaluation: ReferenceError: jQuery is not defined
at :1:1 in F:\web\chrome\headless-chromium-php\src\PageUtils\PageEvaluation.php:78

How to get secure cookies

I use:
$cookies = $page->evaluate ('document.cookie')->getReturnValue();

Only this does not return all the cookies, the secure cookies are not returned.
Is there a different way to get the cookies that Headless chrome receives?

While trying to run the first example I got the following error

[2018-12-26 22:32:53] DEBUG Factory: chrome version: Google Chrome 71.0.3578.98
[2018-12-26 22:32:53] DEBUG process: initializing
[2018-12-26 22:32:53] DEBUG process: using directory: /tmp/chromium-php-U0Asfm
[2018-12-26 22:32:53] DEBUG process: starting process: google-chrome --remote-debugging-port=0 --disable-background-networking --disable-background-timer-throttling --disable-client-side-phishing-detection --disable-default-apps --disable-extensions --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-translate --metrics-recording-only --no-first-run --safebrowsing-disable-auto-update --enable-automation --password-store=basic --use-mock-keychain --headless --disable-gpu --hide-scrollbars --mute-audio --user-data-dir=/tmp/chromium-php-U0Asfm
[2018-12-26 22:32:53] DEBUG process: waiting for 30 seconds for startup
[2018-12-26 22:32:53] DEBUG process: chrome output:mkdir: cannot create directory '/.local': Permission denied
touch: cannot touch '/.local/share/applications/mimeapps.list': No such file or directory
[2018-12-26 22:32:53] DEBUG process: ignoring output:mkdir: cannot create directory '/.local': Permission denied
[2018-12-26 22:32:53] DEBUG process: ignoring output:touch: cannot touch '/.local/share/applications/mimeapps.list': No such file or directory
[2018-12-26 22:32:53] DEBUG process: chrome output:Fontconfig warning: "/etc/fonts/fonts.conf", line 86: unknown element "blank"
[2018-12-26 22:32:53] DEBUG process: ignoring output:Fontconfig warning: "/etc/fonts/fonts.conf", line 86: unknown element "blank"
[2018-12-26 22:32:54] DEBUG process: chrome output:[1226/223254.000921:ERROR:gpu_process_transport_factory.cc(967)] Lost UI shared context.

DevTools listening on ws://127.0.0.1:38553/devtools/browser/0e5d4d2d-5d62-4666-bb55-69f729995a44
[2018-12-26 22:32:54] DEBUG process: ignoring output:[1226/223254.000921:ERROR:gpu_process_transport_factory.cc(967)] Lost UI shared context.
[2018-12-26 22:32:54] DEBUG process: ✓ accepted output
[2018-12-26 22:32:54] DEBUG process: connecting using ws://127.0.0.1:38553/devtools/browser/0e5d4d2d-5d62-4666-bb55-69f729995a44
[2018-12-26 22:32:54] DEBUG socket(1): connecting
[2018-12-26 22:32:54] DEBUG socket(1): ✓ connected
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":1,"method":"Target.setDiscoverTargets","params":{"discover":true}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"231c9529-1486-4254-be86-4d1ae0878540","type":"browser","title":"","url":"","attached":false}}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"DB2B5C3E6A760C4F705280AC10A61DBF","type":"page","title":"","url":"about:blank","attached":false,"browserContextId":"7B78E93131F1E4984B43E5677B645745"}}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"b57e2158-65b6-4c1d-acbf-bfddf97d034f","type":"browser","title":"","url":"","attached":true}}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":1,"result":{}}
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.targetCreated
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.targetCreated
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.targetCreated
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":2,"method":"Target.createTarget","params":{"url":"about:blank"}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","type":"page","title":"","url":"","attached":false,"browserContextId":"7B78E93131F1E4984B43E5677B645745"}}}
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.targetCreated
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":2,"result":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"DB2B5C3E6A760C4F705280AC10A61DBF","type":"page","title":"about:blank","url":"about:blank","attached":false,"browserContextId":"7B78E93131F1E4984B43E5677B645745"}}}
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":3,"method":"Target.attachToTarget","params":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.targetInfoChanged
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","type":"page","title":"","url":"about:blank","attached":true,"browserContextId":"7B78E93131F1E4984B43E5677B645745"}}}
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.targetInfoChanged
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.attachedToTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","targetInfo":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","type":"page","title":"","url":"about:blank","attached":true,"browserContextId":"7B78E93131F1E4984B43E5677B645745"},"waitingForDebugger":false}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":3,"result":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7"}}
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.attachedToTarget
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":5,"method":"Target.sendMessageToTarget","params":{"message":"{"id":4,"method":"Page.getFrameTree","params":[]}","sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7"}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":5,"result":{}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","type":"page","title":"about:blank","url":"about:blank","attached":true,"browserContextId":"7B78E93131F1E4984B43E5677B645745"}}}
[2018-12-26 22:32:54] DEBUG connection: ⇶ dispatching method:Target.targetInfoChanged
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"id":4,"result":{"frameTree":{"frame":{"id":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","url":"about:blank","securityOrigin":"://","mimeType":"text/html"}}}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":7,"method":"Target.sendMessageToTarget","params":{"message":"{"id":6,"method":"Page.enable","params":[]}","sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7"}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":7,"result":{}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"id":6,"result":{}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":9,"method":"Target.sendMessageToTarget","params":{"message":"{"id":8,"method":"Network.enable","params":[]}","sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7"}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":9,"result":{}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"id":8,"result":{}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":11,"method":"Target.sendMessageToTarget","params":{"message":"{"id":10,"method":"Runtime.enable","params":[]}","sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7"}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":11,"result":{}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","auxData":{"isDefault":true,"type":"default","frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:32:54] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Runtime.executionContextCreated
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"id":10,"result":{}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:32:54] DEBUG socket(1): → sending data:{"id":13,"method":"Target.sendMessageToTarget","params":{"message":"{"id":12,"method":"Page.setLifecycleEventsEnabled","params":{"enabled":true}}","sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7"}}
[2018-12-26 22:32:54] DEBUG socket(1): ← receiving data:{"id":13,"result":{}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Page.lifecycleEvent","params":{"frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","name":"commit","timestamp":7008.348604}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Page.lifecycleEvent","params":{"frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","name":"DOMContentLoaded","timestamp":7008.349236}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Page.lifecycleEvent","params":{"frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","name":"load","timestamp":7008.349361}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Page.lifecycleEvent","params":{"frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","name":"networkAlmostIdle","timestamp":7008.349759}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Page.lifecycleEvent","params":{"frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","name":"networkIdle","timestamp":7008.349759}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"id":12,"result":{}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Page.lifecycleEvent","params":{"frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","name":"networkAlmostIdle","timestamp":7008.349759}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","message":"{"method":"Page.lifecycleEvent","params":{"frameId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","loaderId":"C1B1AA7DADCC400D31F6A673C3F784E9","name":"networkIdle","timestamp":7008.349759}}","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}}
[2018-12-26 22:33:01] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Page.lifecycleEvent

( ! ) Fatal error: Uncaught HeadlessChromium\Exception\OperationTimedOut: Operation timed out (3sec) in /var/www/html/vendor/chrome-php/chrome/src/Utils.php on line 65
( ! ) HeadlessChromium\Exception\OperationTimedOut: Operation timed out (3sec) in /var/www/html/vendor/chrome-php/chrome/src/Utils.php on line 65
Call Stack
#	Time	Memory	Function	Location
1	0.0003	377168	{main}( )	.../index.php:0
2	0.1578	1724192	HeadlessChromium\Browser\ProcessAwareBrowser->createPage( )	.../index.php:278
3	0.1881	1879624	HeadlessChromium\Communication\Session->sendMessageSync( )	.../Browser.php:149
4	0.1883	1880560	HeadlessChromium\Communication\SessionResponseReader->waitForResponse( )	.../Session.php:79
5	0.1883	1881136	HeadlessChromium\Utils::tryWithTimeout( )	.../ResponseReader.php:103

[2018-12-26 22:33:01] DEBUG process: killing chrome [2018-12-26 22:33:01] DEBUG process: trying to close chrome gracefully [2018-12-26 22:33:01] DEBUG socket(1): → sending data:{"id":14,"method":"Browser.close","params":[]} [2018-12-26 22:33:01] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Page.lifecycleEvent [2018-12-26 22:33:01] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Page.lifecycleEvent [2018-12-26 22:33:01] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Page.lifecycleEvent [2018-12-26 22:33:01] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Page.lifecycleEvent [2018-12-26 22:33:01] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Page.lifecycleEvent [2018-12-26 22:33:01] DEBUG session(C989FE6F0EA1D398F72EA8F014FC73F7): ⇶ dispatching method:Page.lifecycleEvent [2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"id":14,"result":{}} [2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA","type":"page","title":"about:blank","url":"about:blank","attached":false,"browserContextId":"7B78E93131F1E4984B43E5677B645745"}}} [2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.detachedFromTarget","params":{"sessionId":"C989FE6F0EA1D398F72EA8F014FC73F7","targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}} [2018-12-26 22:33:01] DEBUG socket(1): ← receiving data:{"method":"Target.targetDestroyed","params":{"targetId":"CF5F317D588BA3DFE0EE1263FEC5DEAA"}} [2018-12-26 22:33:01] DEBUG socket(1): disconnecting [2018-12-26 22:33:01] DEBUG socket(1): ✗ could not disconnect [2018-12-26 22:33:01] DEBUG process: waiting for process to close [2018-12-26 22:33:01] DEBUG process: cleaning temporary resources:/tmp/chromium-php-U0Asfm

how to get the file that is downloaded when pressing a button

So the situation:
First I need to login onto a page
Then go to the download page
Fill in some information
Press a button

The button trigger's some jQuery on the page that does a post json to other page that downloads the file.

I manage to login go to the download page and press the button but any idea how to fetch the file of the jQuery json call when pressing the button?

HeadlessChromium saving first image as a new directory folder instead of an image..

The first time I run this via JavaScript POST, HeadlessChromium saves out a new directory folder instead of an image (with the image_name.jpg as the folder name). But when my JavaScript continues to call the script, it saves out an image on each consecutive call..

What would cause HeadlessChromium to save the first image as a directory, but all of the others as JPGs?

My Javascript seems fine.. it's sending the same type of data each time, the script doesn't change on each consecutive call, yet I keep getting a different result on the first call only..

Anyone know what's up here?

<?php
    require 'vendor/autoload.php';
    use HeadlessChromium\BrowserFactory;


    $delay = 15; // 15 seconds

    $bannerName = $_POST['bannerName'];
    $htmlFile = $_POST['htmlFile'];
    $sessionId = $_POST['sessionId'];



    // headless-chromium-php
    $browserFactory = new BrowserFactory('/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome');
    $browser = $browserFactory->createBrowser([
        'headless'        => true,         // disable headless mode
        'connectionDelay' => 0,           // add 0.8 second of delay between each instruction sent to chrome,
        'debugLogger'     => 'php://stdout',  // will enable verbose mode
        'windowSize' => [900, 650],
        'startupTimeout' => 50
    ]);
    $page = $browser->createPage();
    $page->navigate('http://localhost:8888/phpBackend/'.$htmlFile)->waitForNavigation('networkIdle', 10000);  
    $pageTitle = $bannerName; //$page->evaluate('document.title')->getReturnValue();
    sleep(15);
    $screenshot = $page->screenshot([
        'format'  => 'jpeg',  // default to 'png' - possible values: 'png', 'jpeg',
        'quality' => 80       // only if format is 'jpeg' - default 100 
    ])->saveToFile('./backups/'.$sessionId."/".$bannerName.".jpg");
    $browser->close();


    $responsed = array("bannerName" => $bannerName, "htmlFile" => $htmlFile);

    echo json_encode($responsed);

?>

Error logs:

[13-Aug-2018 23:24:09 Europe/Berlin] PHP Warning:  fopen(./backups/C30465E8-748F-70B0-B570-79D9E867FB3B/en_300x250_banner.jpg): failed to open stream: Is a directory in /Applications/MAMP/htdocs/phpBackend/vendor/chrome-php/chrome/src/PageUtils/PageScreenshot.php on line 96
[13-Aug-2018 23:24:09 Europe/Berlin] PHP Warning:  stream_filter_append() expects parameter 1 to be resource, boolean given in /Applications/MAMP/htdocs/phpBackend/vendor/chrome-php/chrome/src/PageUtils/PageScreenshot.php on line 97
[13-Aug-2018 23:24:09 Europe/Berlin] PHP Warning:  fwrite() expects parameter 1 to be resource, boolean given in /Applications/MAMP/htdocs/phpBackend/vendor/chrome-php/chrome/src/PageUtils/PageScreenshot.php on line 98
[13-Aug-2018 23:24:09 Europe/Berlin] PHP Warning:  fclose() expects parameter 1 to be resource, boolean given in /Applications/MAMP/htdocs/phpBackend/vendor/chrome-php/chrome/src/PageUtils/PageScreenshot.php on line 99

OperationTimedOut in waitForStartup

I've been trying to utilise this, I've defined "chromium-browser" as the headless browser in the factory, like so;

       $browserFactory = new BrowserFactory('chromium-browser');

        // starts headless chrome
        $browser = $browserFactory->createBrowser();

        try {
            // creates a new page and navigate to an url
            $page = $browser->createPage($payload->uri);
            $page->navigate($payload->uri);
        } catch(NoResponseAvailable $exception) {
            dd($exception);
        } 

        $browser->close();

However, I'm getting OperationTimedOut in waitForStartup, it seems like the regex does match and does get back a DevTools ws URL. A process does get started (and not removed interestingly) - I'm unsure how to debug this further. Any advice would be greatly appreciated.

Thanks!

Interact with Dev Tools?

Trying to access the websockets, using a page, so i've got the browser, created the page, and started setting up a websocket Uri, but every time i try and do any thing it just says the connection is closed, and I'm thinking its not connecting to the websocket at all..

$webSocketUri = 'ws://127.0.0.1:9222/devtools/browser';
           $connection = new Connection($webSocketUri);
           $connection->connect();
           Log::info(dd($connection));

HeadlessChromium\Communication\Connection {#645
  #strict: true
  #delay: null
  -lastMessageSentTime: null
  #wsClient: HeadlessChromium\Communication\Socket\Wrench {#700
    #client: Wrench\Client {#746
      #uri: "ws://127.0.0.1:9222/devtools/browser/"
      #origin: "http://127.0.0.1"
      #socket: Wrench\Socket\ClientSocket {#748
        #scheme: "tcp"
        #host: "127.0.0.1"
        #port: 9222
        #socket: false
        #context: null
        #connected: false
        #name: null
        #options: array:5 [
          "protocol" => Wrench\Protocol\Rfc6455Protocol {#749}
          "timeout_socket" => 5
          "timeout_connect" => 2
          "ssl_verify_peer" => false
          "ssl_allow_self_signed" => true
        ]
        #protocol: Wrench\Protocol\Rfc6455Protocol {#749}
      }
      #headers: []
      #connected: false
      #payloadHandler: Wrench\Payload\PayloadHandler {#750
        #callback: array:2 [
          0 => Wrench\Client {#746}
          1 => "onData"
        ]
        #payload: null
        #options: array:4 [
          "protocol" => Wrench\Protocol\Rfc6455Protocol {#747}
          "socket_class" => "Wrench\Socket\ClientSocket"
          "on_data_callback" => null
          "socket_options" => []
        ]
        #protocol: Wrench\Protocol\Rfc6455Protocol {#747}
      }
      #received: []
      #options: array:4 [
        "protocol" => Wrench\Protocol\Rfc6455Protocol {#747}
        "socket_class" => "Wrench\Socket\ClientSocket"
        "on_data_callback" => null
        "socket_options" => []
      ]
      #protocol: Wrench\Protocol\Rfc6455Protocol {#747}
    }
    #socketId: 2
    #logger: Psr\Log\NullLogger {#690}
  }
  #responseBuffer: []
  #sendSyncDefaultTimeout: 3000
  #sessions: []
  #listeners: []
  #onceListeners: []
  #logger: Psr\Log\NullLogger {#690}
}

installation issue

Hi
I am trying to install this library using composer, I already installed latest chrome, when I run this command composer require chrome-php/chrome
I get the following error
Your requirements could not be resolved to an installable set of packages.

Problem 1
- Can only install one of: evenement/evenement[v3.0.1, v2.0.0].
- Can only install one of: evenement/evenement[v3.0.1, v2.0.0].
- Can only install one of: evenement/evenement[v3.0.1, v2.0.0].
- chrome-php/chrome v0.6.0 requires evenement/evenement ^3.0.1 -> satisfiable by evenement/evenement[v3.0.1].
- Installation request for chrome-php/chrome ^0.6.0 -> satisfiable by chrome-php/chrome[v0.6.0].
- Installation request for evenement/evenement (locked at v2.0.0) -> satisfiable by evenement/evenement[v2.0.0].

Installation failed, reverting ./composer.json to its original content.

Set headers?

I don't understand how I set headers. For example Accept-Language etc..

Browser user agent no quotes

When setting the browser userAgent, the string is not quoted, thus spaces break the command.

In file src/Browser/BrowserProcess.php at line 322
$args[] = '--user-agent=' . $options['userAgent'];

Current workaround: using double quotes on the userAgent option or using page->setUserAgent

Wait until javascript is executed

$page->navigate('some - url')->waitForNavigation();
$content = $page->evaluate('document.documentElement.innerHTML')->getReturnValue();

Not all javascript is executed when doing the getReturnValue.
Is there a way to wait until it's done?

What are your plans?

When do you plan to add support for the following features?

1. Possibility to install cookies
2. The ability to read cookies
3. Install user agent
4. Working through a proxy server?

Save image

How to get image from site and save(not screenshot) via headless chromium?

waitForPageReload: Error Executing Test. The operation has timed out

Hi,

Since a few days I have a problem with a form that I submit. I now get the error:
Error Executing Test. The operation has timed out

Strange thing before it was working!

$evaluation = $page->evaluate(
'(() => {
document.querySelector("#username").value = "'.USERNAME.'";
document.querySelector("#password").value = "'.PASSWORD.'";
document.querySelector("#submitButton").click();
})()'
);

// wait for the page to be reloaded
$evaluation->waitForPageReload();

Any tips or idea's?

use chrome official settings instead of array of $browser_option

ex:

 $browserFactory->add_argument(['--disable-gpu','--no-sandbox','--blink-settings=imagesEnabled=false']);

It's more flexible than fixed settings.
Thank you.

Delay Before Screenshot

I need to create a delay before the screenshot is taken..

I want the page to load, and then have headless-chromium-php take a screenshot after 15 seconds (I have an animation running that I want to capture it's end frame).

How do I do this?

Close tab without closing browser

Is there a way to destroy a tab?

$page = $browser->createPage($uri) // I need to destroy this tab.

Is that possible?

Thanks 👍.

Package name

The package is published as chrome/chrome on Packagist. Please use a proper vendor name next time instead of the project name itself.

Print page and save to PDF

I read you have the issue in your TODO list

I am currently using the command line of chrome to generate PDFs but its very lacking in options compared to the puppeteer version. Since i dont have nodejs in my current env, i am stuck with it :(

My use case is, that i generate the HTML and feed it to chrome and grab the resulting file

Since this library supports sending direct commands to chrome,
has anyone successfully generated and saved PDFs?

Thank you for your time

Loading pages with network errors

I am getting the following exception when trying to load a page with network errors

<b>Fatal error</b>:  Uncaught TypeError: Return value of HeadlessChromium\Communication\ResponseReader::waitForResponse() must be an instance of HeadlessChromium\Communication\Response, none returned in /var/www/html/application/vendor/chrome-php/chrome/src/Communication/ResponseReader.php:105
Stack trace:
#0 /var/www/html/application/vendor/chrome-php/chrome/src/Communication/Connection.php(228): HeadlessChromium\Communication\ResponseReader-&gt;waitForResponse(3000)
#1 /var/www/html/application/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php(197): HeadlessChromium\Communication\Connection-&gt;sendMessageSync(Object(HeadlessChromium\Communication\Message))
#2 [internal function]: HeadlessChromium\Browser\BrowserProcess-&gt;kill()
#3 {main}
  thrown in <b>/var/www/html/application/vendor/chrome-php/chrome/src/Communication/ResponseReader.php</b> on line <b>105</b><br />

Real browser console ( errors are due to CORS policy )

Access to font at 'https://xxxx/assets/fonts/open-sans-v15-vietnamese_latin_greek_cyrillic-700.ttf' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Is there a way to ignore those errors?

readData takes request response

Not sure where the best place of communication is so I've opened up another ticket for this, I've been working on implementing DOM.getFlattenedDocument so that I can get the contents. However, this line and the socket returns are a bit awkward.

https://github.com/gsouf/headless-chromium-php/blob/f584792aced5033fda1968d8c666b5b91af57835/src/Communication/Connection.php#L259

An example response looks like;

[2017-12-06 23:10:43] DEBUG socket: |=> sending data:{"id":6,"method":"Target.sendMessageToTarget","params":{"message":"{\"id\":5,\"method\":\"DOM.enable\",\"params\":[]}","sessionId":"21dee360-7e84-45dd-b3ba-1573ab9f4e1b:1"}}
[2017-12-06 23:10:43] DEBUG socket: <=| receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"21dee360-7e84-45dd-b3ba-1573ab9f4e1b:1","message":"{\"id\":3,\"result\":{\"frameId\":\"26824.1\"}}","targetId":"21dee360-7e84-45dd-b3ba-1573ab9f4e1b"}}
[2017-12-06 23:10:43] DEBUG socket: <=| receiving data:{"id":6,"result":{}}
[2017-12-06 23:10:43] DEBUG socket: |=> sending data:{"id":8,"method":"Target.sendMessageToTarget","params":{"message":"{\"id\":7,\"method\":\"DOM.getFlattenedDocument\",\"params\":{\"depth\":-1}}","sessionId":"21dee360-7e84-45dd-b3ba-1573ab9f4e1b:1"}}
[2017-12-06 23:10:44] DEBUG socket: <=| receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"21dee360-7e84-45dd-b3ba-1573ab9f4e1b:1","message":"{\"id\":5,\"result\":{}}","targetId":"21dee360-7e84-45dd-b3ba-1573ab9f4e1b"}}
[2017-12-06 23:10:44] DEBUG socket: <=| receiving data:{"id":8,"result":{}}
[2017-12-06 23:10:44] DEBUG socket: <=| receiving data:{"method":"Target.receivedMessageFromTarget","params":{"sessionId":"21dee360-7e84-45dd-b3ba-1573ab9f4e1b:1","message":"{\"id\":7,\"result\":{\"nodes\":[{\"nodeId\":2,\"parentId\":1,\"backendNodeId\":16,\"nodeType\":10,\"nodeName\":\"html\",\"localName\":\"\",\"nodeValue\":\"\",\"publicId\":\"\",\"systemId\":\"\"},{\"nodeId\":5,\"parentId\":4,\"backendNodeId\":19,\"nodeType\":1,\"nodeName\":\"META\",\"localName\":\"meta\",\"nodeValue\":\"\",\"childNodeCount\":0,\"children\":[],\"attributes\":[\"content\",\"/images/branding/googleg/1x/googleg_standard_color_128dp.png\",\"itemp .... truncated ....

Problem here is, my response from the request, I guess it fulfilled the requirements of checkForResponse;

object(HeadlessChromium\Communication\Response)#103 (2) {
  ["message":protected]=>
  object(HeadlessChromium\Communication\Message)#98 (3) {
    ["id":protected]=>
    int(8)
    ["method":protected]=>
    string(26) "Target.sendMessageToTarget"
    ["params":protected]=>
    array(2) {
      ["message"]=>
      string(66) "{"id":7,"method":"DOM.getFlattenedDocument","params":{"depth":-1}}"
      ["sessionId"]=>
      string(38) "21dee360-7e84-45dd-b3ba-1573ab9f4e1b:1"
    }
  }
  ["data":protected]=>
  array(2) {
    ["id"]=>
    int(8)
    ["result"]=>
    array(0) {
    }
  }
}

I'm not sure if there's cases where you'd want to keep multiple responses for the same request (or if there's any requests that even do that) or, to say, filter out responses to requests that don't get immediate responses.

More than happy to take direction on this and implement some methods that manipulate the DOM, maybe a separate class that can be called with a Page object.

Thanks!

Redirect report

Is there any way how to track redirects? For example I load page domain.tld, it redirects to domain.com and it redirects to domain.com/en/ ...now I need to know all these url's where user will be redirected to reach final destination

How to use http proxy?

Thanks a lot!

Get response status code

I need to get response status code, what can I do?

Can I get page information like page size, number of css requests, number of js requests, etc?

I have a requirement to get all the information about a specific page. For example, I want to know how many total requests are required to load a page, how many total requests are static files, etc. Is it possible?

Send post message to page

Would it be possible to send a post message to a page?

Grabbing whole HTML

Hi there, I am trying to get pages html (whole). Is there an easy way to do this? When I am printing $page, it gives me more than I want.

Submit form and wait before getting the response

Hello,

With the help of gsouf I now have the following script:

$content = $page ->evaluate( '(() => { document.querySelector('#username').value = 'username'; document.querySelector('#password').value = 'password'; document.querySelector('#loginform').submit(); })()' ) ->getReturnValue();

Only when doing this the return value is empty.

Error in symphony

I'm trying to test your package and am getting the following error when trying to run your example...

Parse error: syntax error, unexpected '?', expecting variable (T_VARIABLE) in /var/www/html/scripts/browser/vendor/symfony/process/Process.php on line 140

The error seems to be triggered by this line...

$browserFactory = new BrowserFactory();

I've tracked the error down to...

$process = new Process($processString);

but on checking $processString just before the call it has content, so now I'm lost...

chrome --remote-debugging-port=0 --disable-background-networking --disable-background-timer-throttling --disable-client-side-phishing-detection --disable-default-apps --disable-extensions --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-translate --metrics-recording-only --no-first-run --safebrowsing-disable-auto-update --enable-automation --password-store=basic --use-mock-keychain --headless --disable-gpu --hide-scrollbars --mute-audio --user-data-dir=/tmp/chromium-php-ndLwqS

I've installed google-chrome-stable but I wonder if it's not finding it for some reason?

I've tried for hours to fix it with no luck. Does anybody have any ideas? I'm a newb when it comes to composer so I'm more than happy to believe I've done something wrong :)

UPDATE: I noticed I had two versions of PHP installed 7.0 and 7.2, and apache was using 7. I removed 7.2 and updated the one apache uses to 7.2 so hopefully, I only have one version now.

I managed to move past the last error but now there's a new one...

Fatal error: Uncaught RuntimeException: Chrome process stopped before startup completed. Additional info: sh: chrome: command not found in /var/www/html/scripts/browser/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php:353 Stack trace: #0 /var/www/html/scripts/browser/vendor/chrome-php/chrome/src/Utils.php(51): HeadlessChromium\Browser\BrowserProcess->HeadlessChromium\Browser\{closure}(Object(Symfony\Component\Process\Process)) #1 /var/www/html/scripts/browser/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php(388): HeadlessChromium\Utils::tryWithTimeout(30000000, Object(Generator)) #2 /var/www/html/scripts/browser/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php(127): HeadlessChromium\Browser\BrowserProcess->waitForStartup(Object(Symfony\Component\Process\Process), 30000000) #3 /var/www/html/scripts/browser/vendor/chrome-php/chrome/src/BrowserFactory.php(79): HeadlessChromium\Browser\BrowserProcess->start('chrome', Array) #4 /var/www/html/scripts/browser/index.php(14): HeadlessChromium\BrowserFactory->c in /var/www/html/scripts/browser/vendor/chrome-php/chrome/src/Browser/BrowserProcess.php on line 353

I'm sure it's the path not being set up correctly. I'll investigate further.

---

Well, I've finally fixed it... sort of, and as expected, it was my fault :(

The path to google-chrome isn't found so I added it manually just to test and it worked. Then I got an error that 'foo' couldn't be created, which I guess is a permissions error. I removed that and just had the script echo the page title and it worked as expected.

Not sure if it's worth leaving this here just in case somebody else has the same issues but I'll leave it up to the thread owner.

Thanks for a great piece of kit. It's going to make my job a LOT easier :)

Lux

Run browser as a service.

Is there a way to make the browser always running in background as a service and prevent calling it every time?

A portable standalone package including Chrome?

Hi! That looks great! I was looking for something like that

Is there any way to bundle it with Chrome to run on Linux? I'd like to create such a portable package

Thanks

Page.getLayoutMetrics

Hi.
I tried to take full-size screenshot of a page with this library. But noticed that the Page.getLayoutMetrics feature is not implemented.
So I was hoping a little help would be welcome. #43.

Ignore certificate issues?

So, I've installed "sudo apt-get install chromium-browser".

What I've got working so far, is reading the value from the "Search"-button https://www.google.nl/

require_once('vendor/autoload.php');
use HeadlessChromium\BrowserFactory;
// open browser
$factory = new BrowserFactory('chromium-browser');
//$browser = $factory->createBrowser();
$browser = $factory->createBrowser([
      'debugLogger'     => 'php://stdout'
]);

// navigate to a page with a form
$page = $browser->createPage();
$page->navigate('https://www.google.nl/')->waitForNavigation();
// get value in the new page
$value = $page->evaluate('document.querySelector("input[name=\"btnK\"]").value')->getReturnValue();
var_dump($value);

The problem I currently have, is with a web-gui from a vendor we have running internally.
There are multiple servers running the same web-gui, with each a self-signed certificate.
The servers are load-balanced by a virtual-ip. So the certificate is not only self-signed, it also contains the wrong server-name and server-ip address.
Of course, when using Google Chrome on my desktop, I get a certificate error.

How do I ignore the certificate errors with this script?
Because there are some statistics on that web-gui I'd like to grab.

Killed: 9

Running a scan across the full site eventually fires off Killed: 9

tried implementing a logger interface from Log::class, but getting nothing in the log, is the implementation for this correct? or is it implemented differently?

(using the built in Log class from laravel which impliments the PSR\LoggerInterface)

$browser = $browserFactory->createBrowser([
    'headless'        => true,         // disable headless 
    'debugLogger'     => Log::class // will enable verbose mode
]);