Giter Site home page Giter Site logo

apricot's Introduction

Apricot

Apricot is a HTML / DOM parser, scraper for Nodejs. It is inspired by rubys hpricot and designed to fetch, iterate, and augment html or html fragments.

Install

npm install apricot

Getting Started

Apricot.parse("<p id='test'>An HTML Fragment</p>", function(err, doc) {
  // Do something awesome here..
});

// OR Open a remote website, or local file

Apricot.open("http://my_awesome_website.com", function(err, doc) {
  // Do something awesome here..
});    

// Experimental Live Code, third param.
// Live mode will evaluate all javascript in the context of the page.

Apricot.open( file, callback, live_mode)
Apricot.open("http://my_awesome_website.com", function(doc) {
  // Do something awesome here..
},true);

Packed with Awesomeness (API)

Parse and Open both return a Apricot Object, a HTML DOM, created by JSDOM, with all the power of the Sizzle Selector Engine, and XUI Framework for Augmentation.

Apricot.parse("<p id='test'>An HTML Fragment</p>", function(doc) {
  doc.find("selector");     // Populates internal collection, See Sizzle selector syntax (rules)
  doc.each(callback);       // Itterates over the collection, applying a callback to each match (element)
  doc.remove();             // Removes all elements in the internal collection (See XUI Syntax)

  doc.inner("fragment");    // See XUI Syntax
  doc.outer("fragment");    // See XUI Syntax    
  doc.top("fragment");      // See XUI Syntax
  doc.bottom("fragment");   // See XUI Syntax
  doc.before("fragment");   // See XUI Syntax
  doc.after("fragment");    // See XUI Syntax
  doc.hasClass("class");    // See XUI Syntax      
  doc.addClass("class");    // See XUI Syntax            
  doc.removeClass("class"); // See XUI Syntax            
  
  doc.toHTML;               // Returns the HTML
  doc.innerHTML;            // Returns the innerHTML of the body.
  doc.toDOM;                // Returns the DOM representation
  
  // Most methods are chainable, so this works
  doc.find("selector").addClass('foo').after(", just because");
  
});

Problems?

Apricot requires JSDom and htmlparser, these should be brought in via npm when you install apricot, but if you have problems

$ npm install jsdom
$ npm install htmlparser    

Contributors

  • alexkappa
  • lancefisher
  • brog45
  • vrutberg
  • chandlerkent
  • jamesaduncan

TODO

  • Cleanup Code, hide privates, etc.
  • Tests

LICENSE (MIT)

Copyright (c) 2010 Rob Ellis

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

apricot's People

Contributors

alexkwolfe avatar antani avatar barce avatar brog45 avatar chandlerkent avatar ebensing avatar indexzero avatar jamesaduncan avatar lancefisher avatar ryans avatar silentrob avatar skeggse avatar vorotech avatar vrutberg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

apricot's Issues

Examples are not working

Atleast when using npm the documentation says to use

Apricot.open("url", function(err,doc)

while it really only takes function(doc) in the release (fixed in trunk?)

x64 architecture unsupported?

When I run 'npm install apricot', I get message that x64 is not supported (though x86_64 is listed as supported). I'm guessing this may simply be a reporting issue, and a false positive? Other npm packages I've installed don't seem to have this issue. Are there any workarounds for me to install apricot on this machine?

Not compatible with your operating system or architecture: [email protected]

Valid OS:    linux,macos,win
Valid Arch:  x86,ppc,x86_64

Actual OS:   linux
Actual Arch: x64

Relative URLs that don't begin with a "/" replaced incorrectly

For a page with something like

<a href='abc.php'>

When I open it with the callback

doc.find("a").each(el) {
    console.log(el.href)
}

it shows /home/me/project-path/node-modules/apricot/lib/abc.php. I'd expect just abc.php there, or that you'd resolve it relative to the URL I passed to Apricot.open().

Interestingly, URLs that begin with a "/" are left as they are and no attempt is made to add the domain etc.

error { _code: 5 }

I get this error trying to parse an html file. Doc is underfined.

{ _code: 5 }

No other info is given. What does 5 indicate?

Finds title twice and includes linefeed

Apricot.open("http://audun.ytterdal.net/test.php", function(doc) {
    doc.find('title');
doc.each(function(el) {
    console.log('found "' + el.innerHTML + '"');
})
})

Produces

$ node examples.js 
found "
"
found "Tittelen på siden
"

Fails to parse HTML inside browser-specific directives

In the following test, only the link declared outside the IE directive can be found by Apricot:

var Apricot = require('apricot').Apricot,
    assert = require('assert');

var html = '<!--[if IE]><link href="http://foo" media="screen, projection" rel="stylesheet" type="text/css" /><![endif]-->' +
           '<link href="http://foo" media="screen, projection" rel="stylesheet" type="text/css" />'

Apricot.parse(html, function(err, doc) {
    if (err) throw err;
    var numLinks = 0;
    doc.find('link').each(function(link) {
        assert.equal('http://foo', link.href);
        numLinks++;
    });
    assert.equal(2, numLinks);
});

Unsupported Arch ia32

Trying to install Apricot on a Windows machine get this error message:

C:\node\crawler>npm install apricot -g
npm http GET https://registry.npmjs.org/apricot
npm http 304 https://registry.npmjs.org/apricot

npm ERR! Unsupported
npm ERR! Not compatible with your operating system or architecture: apricot@0.0.
5
npm ERR! Valid OS:    linux,macos,win
npm ERR! Valid Arch:  x86,ppc,x86_64
npm ERR! Actual OS:   win32
npm ERR! Actual Arch: ia32
npm ERR!
npm ERR! System Windows_NT 6.1.7601
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nod
ejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "apricot" "-g"
npm ERR! cwd C:\node\crawler
npm ERR! node -v v0.6.15
npm ERR! npm -v 1.1.16
npm ERR! code EBADPLATFORM
npm ERR! message Unsupported
npm ERR! errno {}
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     C:\node\crawler\npm-debug.log
npm not ok

Specifications:

Windows 7 Professional 32 bits
Service Pack 1
Amd Athlon II x2 270 Processor 3.4Ghz

error when attempting to parse anything?

I can require Apricot, but attempting to actually use it fails with a jsdom error. All I've installed on this box is node, npm, and apricot. All should be the newest versions, node and npm were cloned from git today, apricot was then installed via npm install apricot.

$ cat test.js
var apricot = require('apricot').Apricot;
console.log('apricot declared');
apricot.parse('

paragraph

', function() {
console.log('apricot callback');
});

$ node test.js
apricot declared

node.js:50
throw e;
^
Error: Cannot find module 'jsdom-0.1.2/jsdom/level1/core'
at loadModule (node.js:477:15)
at require (node.js:613:14)
at Object.Apricot (/usr/local/lib/node/.npm/apricot/0.0.4/package/lib/apricot.js:6:13)
at Function.parse (/usr/local/lib/node/.npm/apricot/0.0.4/package/lib/apricot.js:282:12)
at Object. (/home/geekdev/test.js:4:9)
at Module._compile (node.js:664:23)
at Module._loadScriptSync (node.js:671:10)
at Module.loadSync (node.js:540:12)
at Object.runMain (node.js:724:24)
at Array. (node.js:747:12)

$ npm ls installed
npm info it worked if it ends with ok
npm info version 0.2.1
[email protected] =silentrob active installed latest remote
[email protected] =tautologistics active installed latest remote
[email protected] =tmpvar active installed remote
[email protected] =tmpvar active installed remote stable latest
[email protected] =isaacs active installed latest remote
npm ok

NPM install fails

juuso@node:~/Dropbox/etsi$ npm install apricot
npm WARN package.json [email protected] No README.md file found!
npm http GET https://registry.npmjs.org/apricot
npm http 304 https://registry.npmjs.org/apricot
npm http GET https://registry.npmjs.org/jsdom
npm http GET https://registry.npmjs.org/node-sizzle
npm http 304 https://registry.npmjs.org/jsdom
npm http 304 https://registry.npmjs.org/node-sizzle
npm http GET https://registry.npmjs.org/libxmljs
npm http GET https://registry.npmjs.org/htmlparser
npm http GET https://registry.npmjs.org/request
npm http GET https://registry.npmjs.org/cssom
npm http GET https://registry.npmjs.org/contextify
npm http 304 https://registry.npmjs.org/libxmljs
npm http 304 https://registry.npmjs.org/contextify
npm http 304 https://registry.npmjs.org/htmlparser
npm http GET https://registry.npmjs.org/bindings/1.0.0
npm http 304 https://registry.npmjs.org/request
npm http 304 https://registry.npmjs.org/cssom
npm http GET https://registry.npmjs.org/bindings
npm http 304 https://registry.npmjs.org/bindings/1.0.0

> [email protected] install /home/juuso/Dropbox/etsi/node_modules/apricot/node_modules/node-sizzle/node_modules/libxmljs
> node-gyp rebuild

/bin/sh: 1: xml2-config: not found
Traceback (most recent call last):
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp_addon", line 38, in <module>
    rc = gyp.main(gyp_args)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/__init__.py", line 471, in main
    options.circular_check)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/__init__.py", line 111, in Load
    depth, generator_input_info, check, circular_check)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 2378, in Load
    depth, check)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 384, in LoadTargetBuildFile
    build_file_data, PHASE_EARLY, variables, build_file_path)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 1053, in ProcessVariablesAndConditionsInDict
    build_file)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 1068, in ProcessVariablesAndConditionsInList
    ProcessVariablesAndConditionsInDict(item, phase, variables, build_file)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 1027, in ProcessVariablesAndConditionsInDict
    ProcessConditionsInDict(the_dict, phase, variables, build_file)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 904, in ProcessConditionsInDict
    variables, build_file)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 1053, in ProcessVariablesAndConditionsInDict
    build_file)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 1072, in ProcessVariablesAndConditionsInList
    expanded = ExpandVariables(item, phase, variables, build_file)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 714, in ExpandVariables
    (contents, p.returncode))
Exception: Call to 'xml2-config --libs' returned exit status 127. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp_addon` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/nodejs/npm/node_modules/node-gyp/lib/configure.js:290:16)
gyp ERR! stack     at ChildProcess.EventEmitter.emit (events.js:91:17)
gyp ERR! stack     at Process._handle.onexit (child_process.js:674:10)
gyp ERR! System Linux 3.2.0-27-generic
gyp ERR! command "node" "/usr/lib/nodejs/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/juuso/Dropbox/etsi/node_modules/apricot/node_modules/node-sizzle/node_modules/libxmljs
gyp ERR! node -v v0.8.4
gyp ERR! node-gyp -v v0.6.1
gyp ERR! not ok 
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! `sh "-c" "node-gyp rebuild"` failed with 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the libxmljs package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls libxmljs
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 3.2.0-27-generic
npm ERR! command "nodejs" "/usr/bin/npm" "install" "apricot"
npm ERR! cwd /home/juuso/Dropbox/etsi
npm ERR! node -v v0.8.4
npm ERR! npm -v 1.1.45
npm ERR! code ELIFECYCLE
npm http 304 https://registry.npmjs.org/bindings

> [email protected] install /home/juuso/Dropbox/etsi/node_modules/apricot/node_modules/jsdom/node_modules/contextify
> node-gyp rebuild

Traceback (most recent call last):
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp_addon", line 38, in <module>
    rc = gyp.main(gyp_args)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/__init__.py", line 471, in main
    options.circular_check)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/__init__.py", line 111, in Load
    depth, generator_input_info, check, circular_check)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 2378, in Load
    depth, check)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 358, in LoadTargetBuildFile
    includes, True, check)
  File "/home/juuso/.node-gyp/0.8.4/tools/gyp/pylib/gyp/input.py", line 208, in LoadOneBuildFile
    raise Exception("%s not found (cwd: %s)" % (build_file_path, os.getcwd()))
Exception: binding.gyp not found (cwd: /home/juuso/Dropbox/etsi/node_modules/apricot/node_modules/jsdom/node_modules/contextify) while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp_addon` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/nodejs/npm/node_modules/node-gyp/lib/configure.js:290:16)
gyp ERR! stack     at ChildProcess.EventEmitter.emit (events.js:91:17)
gyp ERR! stack     at Process._handle.onexit (child_process.js:674:10)
gyp ERR! System Linux 3.2.0-27-generic
gyp ERR! command "node" "/usr/lib/nodejs/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/juuso/Dropbox/etsi/node_modules/apricot/node_modules/jsdom/node_modules/contextify
gyp ERR! node -v v0.8.4
gyp ERR! node-gyp -v v0.6.1
gyp ERR! not ok 
npm WARN optional dep failed, continuing [email protected]
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/juuso/Dropbox/etsi/npm-debug.log
npm ERR! not ok code 0

Elements within conditional comments are ignored.

Im trying to rewrite a bunch of asset urls in an html file.

All the script tags that are outside of an IE conditional are fine and are rewritten correctly when I modify the src but, the problem is anything within:

<!--[if lt IE 9]>
      <script src="bower_components/es5-shim/es5-shim.js"></script>
      <script src="bower_components/json3/lib/json3.min.js"></script>
<![endif]-->

isn't returned by find.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.