meanjs / mean-seo Goto Github PK

SEO Solution for MEAN.JS applications using PhantomJS.

JavaScript 97.59% HTML 2.41%

mean-seo's Introduction

Deprecated

Google has deprecated (10/2015) _escaped_fragment_ URL parameter and can now scrape javascript generated pages and thus this package isn't needed anymore at MEAN.js projects.

Beware that at the current state this package has some security issues via outdated dependencies.

MEAN-SEO

SEO Solution for MEAN.JS applications which forwards crawlers requests to a compiled HTML copy using PhantomJS.

Longer Version

If you ever tried to make your AngularJS application crawler friendly, you already know this is a bit of a headache. Part of evolving the MEAN.JS stack towards production ready state, the MEAN-SEO module makes it pretty simple to make sure your MEAN application is ready for crawlers requests.

When a crawler requests the page using the _escaped_fragment_, the module launches the PhantomJS headless-browser, which creates a copy of the page and stores it in cache for future requests.

Quick Install

First you'll need to install the MEAN-SEO module using npm:

npm install mean-seo --save

Then include in you express application:

var seo = require('mean-seo');

And finally, just before you require the app.router middleware add the following:

app.use(seo({
	cacheClient: 'disk', // Can be 'disk' or 'redis'
	redisURL: 'redis://:password@hostname:port', // If using redis, optionally specify server credentials
	cacheDuration: 2 * 60 * 60 * 24 * 1000, // In milliseconds for disk cache
	cacheBuffer: 200 * 1024 // The the largest amount of data allowed on stdout, used for PhantomJS. Increase this when receiving 'stdout maxBuffer exceeded' errors.
}));

// app.use(app.router) will be below this line

If you use HTML5 URL scheme then you should let the crawler know you're serving an AJAX application by adding the following to the HEAD tag of your page:

<meta name=”fragment” content=”!”>

Resources

License

(The MIT License)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

mean-seo's People

Contributors

Stargazers

Watchers

mean-seo's Issues

running on linux

Hi - fantastic module, thanks. We found a minor problem when running on a Linux box. Line 14 of lib/mean-seo.js is this:

Cache = require('./Cache');

On a mac that worked fine but when we uploaded the code to an EC2 box it failed to run. Changing the capitalization to this works:

Cache = require('./cache');

Can you amend it so an npm install on a fresh server works out of the box?

many thanks

Compatible with Express4?

Hi,
is this module compatible with Express without using Mean ?

escaped fragment should not add a slash

A pull request with a fix for this problem was merged to master, but later the bug was reintroduced. Could the redundant slash be removed again?

crawling a https website

how do i crawl a website with valid SSL certtificate. I updated the phantomjs to the latest version and added the required parameters in childArgs array in browser.js file like this-
childArgs[2] = 'ssl-protocol=tlsv1';
childArgs[3] = '--ssl-certificates-path=/path/to/SSL/certificate';
childArgs[4] = '--ignore-ssl-errors=true';
I am getting a 'unable to access network error'..
what am i doing wrong here?

Critical!!! Update dependencies. PhantomJs 2.x

Please update Phantom to 2.x cause Phantom 1.x doesn't have Function.prototype.bind so we cant run Angular 1.5+ with it

Error: [$injector:modulerr] Failed to instantiate module ng due to:
TypeError: 'undefined' is not an object (evaluating 'Function.prototype.bind.apply')

Get error on meanjs 0.5.0

TypeError: 'undefined' is not a function (evaluating 'this.window.console.error.bind(this.window.console)')

_escaped_fragment_ got deprecated.

_escaped_fragment_ option got deprecated:
https://webmasters.googleblog.com/2015/10/deprecating-our-ajax-crawling-scheme.html

How can we make it to pre-render for crawlers? (without the escaped_fragment)

Site doesn't render in Google Webmaster tools

Hi. I build a site with mean.js and use it mean-seo for google search optimisation. How can I check work of the mean-seo? What should I do? I try use Google Webmaster tools but have {{}} and not render page after request to '/'.

escapedFragment empty, mean-seo won't work

Hi,

I've been trying to install mean-seo and it never gets triggered. By digging into the code I found that

var escapedFragment = req.query.escaped_fragment;

is always empty.

In general, req.query is always empty.

Note: I put the app.use(seo...) at the right place, just before the routers initialization.

What seems to be the issue here in the middleware?

'Unterminated quoted string' when running in Docker

Error code is as follows:

at ChildProcess.exithandler (child_process.js:658:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Process.ChildProcess._handle.onexit (child_process.js:833:5)
Error: Command failed: /home/agepath/node_modules/mean-seo/node_modules/phantomjs/lib/phantom/bin/phantomjs: 4: /home/agepath/node_modules/mean-seo/node_modules/phantomjs/lib/phantom/bin/phantomjs: Syntax error: Unterminated quoted string

Waiting for a condition before render

Would be great to incorporate WaitFor.js (https://github.com/ariya/phantomjs/blob/master/examples/waitfor.js) into mean-seo.

This Phantom.js script waits until the test condition is true or a timeout occurs.
Useful for waiting on a server response or for a ui change (fadeIn, etc.) to occur.

I'm happy to look into this, but just making sure someone else is not already working on this?

could not understand where should I put app.use(...).

If I'm mistaken please correct me but isn't app.use(app.router) removed from Express 4.

https://github.com/visionmedia/express/wiki/Migrating-from-3.x-to-4.x

And if I'm not mistaken again, MEAN is using Exp. 4
Therefore there is no line of app.use(app.router)

then where should I put app.use(seo...)?

app.use(seo({
    cacheClient: 'disk', // Can be 'disk' or 'redis'
    cacheDuration: 2 * 60 * 60 * 24 * 1000, // In milliseconds for disk cache
}));

// app.use(app.router) will be below this line

Pushstate instead go hash bang ?

Is it possible to use Pushstate instead go hash bang ?

partials code not showing

when i go to url
http://localhost:3000/?_escaped_fragment_=/#!/signin

the resulting page source is showing

<section style="min-height:500px"> <section class="container">          
    <!-- uiView:  --><section data-ui-view="" class="ng-scope"><section data-ng-controller="HomeController" class="ng-scope"> 
    <span ng-show="false" class="ng-hide">homepage content</span> 
</section>

and not showing signin page content . please correct me if i am going wrong.

`escapedFragment` is falsey if it's an empty string

Maybe I'm not reading the spec correctly, but it says that for HTML5 pushstate URLs you get pinged with "?escaped_fragment=". Notice that the get parameter has no value, it's just an empty string.

As such, when I was testing this out on my app, I tried this:

$ curl "http://localhost:3000?_escaped_fragment_="

But this didn't trigger the SEO middleware.

Digging into the source, I found that you do this in lib/mean-seo.js:39:

return function SEO(req, res, next) {
  var escapedFragment = req.query._escaped_fragment_;

  //If the request came from a crawler
  if (escapedFragment) {
    /* do SEO stuff */
  } else {
    next();
  }
}

For my CURL, that comes up as falsey.

Shouldn't the if conditional rather test for escapedFragment !== undefined?

Critical!!! Doesn't pass params to Phantom

For example crawler requests page /categories?sort=name&_escaped_fragment_= but Mean Seo passes to Phantom /categories.
As result wrong rendered page!

Server error

In my local development environment, i can see the cache folder being created in mean-seo folder.

But on production when i try to hit the url eg, xyz.com/?escaped_fragment=about i get server error page.

What is wrong?