Giter Site home page Giter Site logo

something wrong now? about 9anime-scraper HOT 10 CLOSED

jqwotos avatar jqwotos commented on September 27, 2024
something wrong now?

from 9anime-scraper.

Comments (10)

jQwotos avatar jQwotos commented on September 27, 2024 2

I created a hotfix using chromedriver on the selenium branch, phantomjs driver was giving me some issues. I'll continue taking a look at phantomjs, but this should work for now.

  • Tested on Sierra 10.12.5, Python 3.6.1

from 9anime-scraper.

jQwotos avatar jQwotos commented on September 27, 2024

Yea 9anime started using more anti bot measures, one of them was setting a reqkey cookie. They have been changing the way that reqkeys are generated weekly... So I'm not fast enough on reversing their generators. Basically you just have to change the reqkey variable in constants with one manually obtained. Feel free to use this one acccf12490f93045a066052213d910d6

But they are reset daily.

from 9anime-scraper.

jQwotos avatar jQwotos commented on September 27, 2024

I was thinking of doing a full rewrite but this time using selenium, which will allow for executing the javascript reqkey generators. But selenium has a much higher overhead than basic webscraping. If you think of other methods feel free to contribute.

from 9anime-scraper.

danielb2 avatar danielb2 commented on September 27, 2024

I'm getting the same error with this one.

I'm not sure in what capacity you'd use selenium. Is it because it has form filling capabilities?
what about it allows you to execute the generators?

from 9anime-scraper.

jQwotos avatar jQwotos commented on September 27, 2024

I'll do a bit more digging around over the weekend, it's also bugging out for me even with the updated reqkey. So they probably changed the system again. Basically what they did before was send a token file that would be parsed by a really absurdly annoying javascript parser that would then set the result as a reqkey cookie.

I was planning on using selenium because it's able to fully emulate a browser and therefore, execute the javascript token parser automatically.

However, taking a quick look at 9anime again. It seems that they changed even more things.

I might be able to get a workaround going again by Monday, depending on how busy the weekend gets.

from 9anime-scraper.

danielb2 avatar danielb2 commented on September 27, 2024

http://stackoverflow.com/questions/13287490/is-there-a-way-to-use-phantomjs-in-python#15699761

doesn't seem unreasonable

from 9anime-scraper.

sanidhya711 avatar sanidhya711 commented on September 27, 2024

does this still work?

from 9anime-scraper.

danielb2 avatar danielb2 commented on September 27, 2024

@sanidhyajain1 it worked briefly a long time ago. just use https://github.com/anime-dl/anime-downloader instead

from 9anime-scraper.

sanidhya711 avatar sanidhya711 commented on September 27, 2024

hmm i want to build something like this should i use a library or something to bypass cloudflare?
im using puppeteer js btw

from 9anime-scraper.

danielb2 avatar danielb2 commented on September 27, 2024

dead project

from 9anime-scraper.

Related Issues (7)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.