Comments (10)
I created a hotfix using chromedriver on the selenium branch, phantomjs driver was giving me some issues. I'll continue taking a look at phantomjs, but this should work for now.
- Tested on Sierra 10.12.5, Python 3.6.1
from 9anime-scraper.
Yea 9anime started using more anti bot measures, one of them was setting a reqkey cookie. They have been changing the way that reqkeys are generated weekly... So I'm not fast enough on reversing their generators. Basically you just have to change the reqkey variable in constants with one manually obtained. Feel free to use this one acccf12490f93045a066052213d910d6
But they are reset daily.
from 9anime-scraper.
I was thinking of doing a full rewrite but this time using selenium, which will allow for executing the javascript reqkey generators. But selenium has a much higher overhead than basic webscraping. If you think of other methods feel free to contribute.
from 9anime-scraper.
I'm getting the same error with this one.
I'm not sure in what capacity you'd use selenium. Is it because it has form filling capabilities?
what about it allows you to execute the generators?
from 9anime-scraper.
I'll do a bit more digging around over the weekend, it's also bugging out for me even with the updated reqkey. So they probably changed the system again. Basically what they did before was send a token file that would be parsed by a really absurdly annoying javascript parser that would then set the result as a reqkey cookie.
I was planning on using selenium because it's able to fully emulate a browser and therefore, execute the javascript token parser automatically.
However, taking a quick look at 9anime again. It seems that they changed even more things.
I might be able to get a workaround going again by Monday, depending on how busy the weekend gets.
from 9anime-scraper.
http://stackoverflow.com/questions/13287490/is-there-a-way-to-use-phantomjs-in-python#15699761
doesn't seem unreasonable
from 9anime-scraper.
does this still work?
from 9anime-scraper.
@sanidhyajain1 it worked briefly a long time ago. just use https://github.com/anime-dl/anime-downloader instead
from 9anime-scraper.
hmm i want to build something like this should i use a library or something to bypass cloudflare?
im using puppeteer js btw
from 9anime-scraper.
dead project
from 9anime-scraper.
Related Issues (7)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 9anime-scraper.