superbrothers/capturejs

Full webpage capture command-line tool with PhantomJS and NodeJS

CaptureJS

CaptureJS is full webpage capture command-line tool with PhantomJS.

Installation

First install PhantomJS.

$ npm install -g capturejs

Usage

Usage: capturejs [options]

Options:
  -u, --uri <value>                URI (required)
  -o, --output <value>             Output image file (required)
  -p, --ssl-protocol <value>       Sets the SSL protocol for secure connections (default is SSLv3) (sslv3|sslv2|tlsv1|any)
  -I, --ignore-ssl-errors          Ignores SSL errors (expired/self-signed certificate errors)
  -W, --web-security               Enables web security and forbids cross-domain XHR (default is true) (true|false|yes|no)
  -s, --selector <value>           CSS selector
  -A, --user-agent <value>         UserAgent
  -J, --javascript-file <value>    Inject external script code from the specified file into the page
  -j, --inject-script              Inject in-line custom scripts into the page
  -V, --viewportsize <value>       ViewPortSize {width}x{height}
  -C, --cliprect <value>           ClipRect {top}x{left}x{width}x{height} that will be rendered
  -c, --cookies-file <value>       Cookies file
  -T, --timeout <value>            HTTP Timeout (ms)
  -R, --renderdelay <value>        Render delay (ms)
  -w, --waitcapturedelay <value>   Capture delay (ms)
  -z, --zoomfactor <value>         Zoom Factor (default is 1.0, i.e. 100% zoom)

  -v, --version                    Show version number and exit
  -h, --help                       Show this message and exit

Quick Start

% capturejs --uri http://phantomjs.org/ \
            --selector '.header' \
            --viewportsize 1400x1400 \
            --output 'phantomjs.org.png'

% capturejs --uri http://phantomjs.org/ \
            --selector '.header' \
            --viewportsize 1400x1400 \
            --javascript-file ./hidelogo.js \
            --output 'phantomjs.org_hide_logo.png'

// hidelogo.js
document.querySelector('.header img').style.visibility = 'hidden';

% capturejs --uri http://phantomjs.org/ \
            --selector '.header' \
            --viewportsize 1400x1400 \
            --javascript-file ./hidelogo.js \
            --inject-script 'document.querySelector(".header").style.background = 'red';' \
            --output 'phantomjs_org_red.png'

Copyright

Copyright (c) 2012 Kazuki Suda. See LICENSE.txt for further details.

capturejs's Issues

testing

Change CUI command interface and to display generated file name

% capturejs http://phantomjs.org/
phantomjs.org-201306161603.png

I will change CUI command interface and to display generated image file name. Default image file name format is #{hostname}-%Y%m%d%H%M.png.

capturejs hangs

capturejs --uri http://tiqav.com/ \
            --selector 'header .wrapper' \
            --output tiqav_header.png

After successfully installing, and getting usage information, the system hangs on the above command, copied from the GitHub project homepage.

That is true on OS X SnowLeopard and Ubuntu.

Please let me know if I can provide more details.

express warning message

% capturejs --uri 'http://www.yahoo.co.jp' --output yahoojapan.png
Warning: express.createServer() is deprecated, express
applications no longer inherit from http.Server,
please use:

  var express = require("express");
  var app = express();

└─┬ [email protected] -> /Users/superbrothers/.nodebrew/node/v0.8.0/lib/node_modules/capturejs
  ├── [email protected] 
  └─┬ [email protected] 
    ├─┬ [email protected] 
    │ ├─┬ [email protected] 
    │ │ └── [email protected] 
    │ ├── [email protected] 
    │ ├── [email protected] 
    │ ├─┬ [email protected] 
    │ │ ├── [email protected] 
    │ │ └── [email protected] 
    │ └─┬ [email protected] 
    │   ├── [email protected] 
    │   ├── [email protected] 
    │   └── [email protected] 
    ├─┬ [email protected] 
    │ ├── [email protected] 
    │ └── [email protected] 
    └─┬ [email protected] 
      ├── [email protected] 
      ├─┬ [email protected] 
      │ ├── [email protected] 
      │ ├── [email protected] 
      │ ├── [email protected] 
      │ └── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      ├── [email protected] 
      └─┬ [email protected] 
        └── [email protected] 

Error on MacOS Big Sur Intel Machine

Node version: v12.18.3
Tried to run the following Command:

capturejs --uri http://phantomjs.org/ \
            --selector '.header' \
            --viewportsize 1400x1400 \
            --output 'phantomjs.org.png'

Got the error:
phantom stdout: NETWORK_ERR: XMLHttpRequest Exception 101: A network error occurred in synchronous requests.

abnormal phantomjs exit code: 2

On a FreeBSD 10.1 machine I get an abnormal phantomjs exit code: 2 with

$ uname -a
FreeBSD <servername> 10.1-RELEASE-p19 FreeBSD 10.1-RELEASE-p19 #0: Sat Aug 22 03:55:09 UTC 2015     [email protected]:/usr/obj/usr/src/sys/GENERIC  amd64

$ node --version
v0.12.7

$ npm list -g capturejs phantomjs phantomjs2
/usr/local/lib
├── [email protected] 
├── [email protected] 
└── [email protected]

This is what I get with phantomjs:

$ capturejs --uri http://phantomjs.org --output 'phantomjs.org.png'  
phantom stderr: /usr/local/lib/node_modules/phantomjs/lib/phantom/bin/phantomjs: 1: Syntax error: Unterminated quoted string  
/usr/local/lib/node_modules/phantomjs/lib/phantom/bin/phantomjs: 2: Syntax error: Error in command substitution  

{ [AssertionError: abnormal phantomjs exit code: 2]  
  name: 'AssertionError',  
  actual: false,  
  expected: true,  
  operator: '==',  
  message: 'abnormal phantomjs exit code: 2',  
  generatedMessage: false }

This is what I get with phantomjs2:

$ capturejs --uri http://phantomjs.org --output 'phantomjs.org.png'
phantom stderr: /usr/local/lib/node_modules/phantomjs2/lib/phantom/bin/phantomjs: �^�__ustring__TEXT��!V��!__info_plist__TEXT�!E�!qtmetadata__TEXT: not found
/usr/local/lib/node_modules/phantomjs2/lib/phantom/bin/phantomjs: h�0�: not found
/usr/local/lib/node_modules/phantomjs2/lib/phantom/bin/phantomjs: �����!�H__PAGEZERO�__TEXT�e�e__text__TEXTh�__stubs__TEXT4��@4��__stub_helper__TEXTt���: not found
/usr/local/lib/node_modules/phantomjs2/lib/phantom/bin/phantomjs: t���__gcc_except_tab__TEXTD׳XjD׳__cstring__TEXT�A��A�__const__TEXT�^�g�!@: not found
/usr/local/lib/node_modules/phantomjs2/lib/phantom/bin/phantomjs: 2: Syntax error: word unexpected (expecting ")")

{ [AssertionError: abnormal phantomjs exit code: 2]
  name: 'AssertionError',
  actual: false,
  expected: true,
  operator: '==',
  message: 'abnormal phantomjs exit code: 2',
  generatedMessage: false }

Any help is very much appreciated.

module化

var capturejs = require("capturejs");

support user name and password

Compressing option

If this library has compressing option, I will be ultra happy. For example:

Compress {width}
    --compress <value>

Travis CI integration

phantomjs-node in NPM is unmaintained

With phantomjs 1.7, when I attempt to run the capturejs example, it fails with this error:

$ capturejs --uri http://tiqav.com/ \
>             --selector 'header .wrapper' \
>             --output tiqav_header.png

phantom stdout: TypeError: 'undefined' is not a function (evaluating 'phantom.loadModuleSource('webpage')')

  /Users/rouge8/local/npm/lib/node_modules/capturejs/node_modules/phantom/shim.js:1573
  /Users/rouge8/local/npm/lib/node_modules/capturejs/node_modules/phantom/shim.js:1691
  /Users/rouge8/local/npm/lib/node_modules/capturejs/node_modules/phantom/shim.js:156

phantom stdout:   /Users/rouge8/local/npm/lib/node_modules/capturejs/node_modules/phantom/shim.js:7
  /Users/rouge8/local/npm/lib/node_modules/capturejs/node_modules/phantom/shim.js:1694

I found sebv/node-phantomjs-sync#3, which suggests using sebv/phantomjs-node rather than the NPM provided version. I tested this and it works for me.

Viewportsize parameter is ignored

I have discovered that some situations the --viewportsize option is ignored. For example if I run

capturejs --uri http://www.fosspatents.com/ \ 
            --viewportsize 1400x1000 \
            --output 'fosspatents.com.png'

all I get is a pretty large website. Maybe I get the meaning of this parameter wrong but I would expect that all the content that is not in the viewport is cut off.