Giter Site home page Giter Site logo

helmssyss / xenforo_image_scraper Goto Github PK

View Code? Open in Web Editor NEW
0.0 1.0 1.0 42 KB

Uploads images of specified topics opened on the Xenforo based web site to Google Drive.

License: MIT License

Python 100.00%
google-drive-api-v3 google-drive-uploader xenforo google-drive google googleapi googleauth

xenforo_image_scraper's Introduction

Xenforo_Image_Scraper

TR

Türk Hack Team(https://www.turkhackteam.org/) forum sitesinde, açılmış herhangi bir forum linkinde www.hizliresim.com 'sitesi üzerinden atılmış bütün resimleri alır google drive'a yükler.

Yükleme işlemi, gidilen saf resim linkinden resimin sayfa içeriğini alır bunu byte formatına çevirir ve bu çevrilen formatta drive'a yollar. Drive tarafında bunlar işlenip png uzantılı resimler haline dönüşür. Böylece lokalde tekrardan indirmeye gerek kalmayarak hızdan ve yerden tasarruf etmiş oluruz.

Başlamadan önce

>> pip install -r requirements.txt

Google Drive'dan API'yi aktif etmeniz, aktif ettikten sonra da size verilen json dosyasını indirmeniz gerekli. İndirilen json dosyasının adını "credentials.json" olarak değiştirmeniz gerekli.

Daha sonra Foruma gidip ilgili cookieleri edinmeniz gerekli. Bunları da edinip, ".cfg" dosyasındaki değerlere yazmalısınız.

Çalıştırmak için

>> python app.py

link_manage.json adında json dosyası bulunuyor.. script çalıştğında bu dosya yoksa oluşturur, varsa bu dosyayı okur. Siz bir veya birden fazla link girilmesini istediğinizde, bunları all_link adındaki anahtarın barındırdığı listeye eklemeniz gerekli. Ekledikten sonra kayıt edebilirsiniz..

EN

Türk Hack Team (https://www.turkhackteam.org/) takes all the pictures posted on the forum site, www.hizliresim.com, on any forum link that has been opened, and uploads them to google drive.

The upload process takes the page content of the image from the pure image link, converts it to byte format and sends it to the drive in this converted format. On the Drive side, these are processed and converted into png images. Thus, we save speed and space by not having to download it again locally.

before you start

>> pip install -r requirements.txt

You need to activate the API from Google Drive and download the json file given to you after activating it. You need to rename the downloaded json file to "credentials.json".

Then you need to go to the Forum and obtain the relevant cookies. You should also obtain these and write them to the values ​​in the ".cfg" file.

To make it work

>> python app.py

There is a json file named link_manage.json. When the script runs, it creates this file if it does not exist, reads this file if it exists. When you want to enter one or more links, you need to add them to the list hosted by the key named all_link. You can save after adding.

Fenrirsoftware Helmsys

xenforo_image_scraper's People

Contributors

fenrirsoftware avatar helmssyss avatar

Watchers

 avatar

Forkers

fenrirsoftware

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.