Giter Site home page Giter Site logo

iceflowre / unidown-mr_de Goto Github PK

View Code? Open in Web Editor NEW
0.0 0.0 1.0 215 KB

Downloads all available german ebooks from MobileRead (https://mobileread.com). -- Lädt alle verfügbaren deutschen Bücher von MobileRead (https://mobileread.com/) herunter.

License: GNU General Public License v3.0

Python 100.00%

unidown-mr_de's People

Contributors

dependabot-preview[bot] avatar dependabot-support avatar iceflowre avatar

Watchers

avatar avatar avatar

Forkers

deumus74

unidown-mr_de's Issues

Fix html5 parsing 

2018.11.15 11:44:42.931 | ERROR - root | manager.run: Plugin mr_de crashed.
Traceback (most recent call last):
  File "c:\python37\lib\site-packages\unidown\core\manager.py", line 146, in run
    download_from_plugin(plugin)
  File "c:\python37\lib\site-packages\unidown\core\manager.py", line 93, in download_from_plugin
    plugin.update_download_links()
  File "c:\python37\lib\site-packages\unidown\plugin\a_plugin.py", line 187, in update_download_links
    self._download_data = self._create_download_links()
  File "c:\python37\lib\site-packages\unidown_mr_de\plugin.py", line 46, in _create_download_links
    self.download(wiki_thread_dic, self.threads_path, 'Downloading threads', 'thread')
  File "c:\python37\lib\site-packages\unidown\plugin\a_plugin.py", line 310, in download
    download_without_errors.append(job.result())
  File "c:\python37\lib\concurrent\futures\_base.py", line 425, in result
    return self.__get_result()
  File "c:\python37\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "c:\python37\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "c:\python37\lib\site-packages\unidown\plugin\a_plugin.py", line 250, in download_as_file
    with self._downloader.request('GET', url, preload_content=False, retries=urllib3.util.retry.Retry(3)) as reader:
  File "c:\python37\lib\site-packages\urllib3\request.py", line 68, in request
    **urlopen_kw)
  File "c:\python37\lib\site-packages\urllib3\request.py", line 89, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "c:\python37\lib\site-packages\urllib3\connectionpool.py", line 550, in urlopen
    if assert_same_host and not self.is_same_host(url):
  File "c:\python37\lib\site-packages\urllib3\connectionpool.py", line 436, in is_same_host
    host = _ipv6_host(host, self.scheme)
  File "c:\python37\lib\site-packages\urllib3\connectionpool.py", line 902, in _ipv6_host
    if host.startswith('[') and host.endswith(']'):
AttributeError: 'NoneType' object has no attribute 'startswith'

branch: https://github.com/IceflowRE/unidown-mr_de/tree/wip2

Add custom file checking

Need custom checking if a good file is downloaded.
Or add after and before methodes?

with filename.open(mode='r', encoding="utf-8") as reader:
  check = reader.readline(14)
    if check == "<!DOCTYPE html":  # if only the html file "Invalid Attachment specified." was downloaded
      download_failed_list.append(at_id)
    else:
      download_succeed_list.append(at_id)

IceflowRE/unidown#27

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.