Comments (3)
Provide some used code or log details.
from pywebcopy.
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/robots.txt
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri
webpage - INFO - Starting save_complete Action on url: ['http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri']
parsers - INFO - Parsing tree with source: <<urllib3.response.HTTPResponse object at 0x11AF7E90>> encoding and parser <<lxml.etree.HTMLParser object at 0x11AEF030>>
webpage - INFO - Starting save_assets Action on url: 'http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri'
webpage - Level 100 - Queueing download of <29> asset files.
webpage - INFO - Starting save_html Action on url: 'http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri'
webpage - INFO - WebPage saved successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.writeopinions.com\c69aeed5__abu-abd-al-rahman-ibn-aqil-al-zahiri.html
config - INFO - Got response <Response [200]> from http://t0.gstatic.com/images?q=tbn:ANd9GcQEWTHxP9VC7soNKgEZpIegGh8zvZV1h1ABtQR5Gpt8V85Kqfx8ITUVv3A
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t0.gstatic.com\9b67b38a__images.jpg
elements - INFO - File of type .jpg written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t0.gstatic.com\9b67b38a__images.jpg
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/logo.png
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.writeopinions.com\a0cc724d__logo.png
config - INFO - Got response <Response [200]> from http://t2.gstatic.com/images?q=tbn:ANd9GcQmJSdsNcOqsYnCA8qmUtwhfvmzo-VC6-jow3XF3Hq2EyW4N1vstbrNG_D6
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t2.gstatic.com\9a3a25ef__images.jpg
elements - INFO - File of type .png written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.writeopinions.com\a0cc724d__logo.png
elements - INFO - File of type .jpg written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t2.gstatic.com\9a3a25ef__images.jpg
config - INFO - Got response <Response [200]> from http://connect.facebook.net/en_US/all.js#xfbml=1
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\connect.facebook.net\en_US\2beb0397__all.js
config - INFO - Got response <Response [200]> from http://t3.gstatic.com/images?q=tbn:ANd9GcRNmVn169BsI2COBefR47BN6eDbuKZ18B4AXdm-5un12uldcw4d6k9dwc0
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t3.gstatic.com\11d6e711__images.jpg
elements - INFO - File of type .jpg written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t3.gstatic.com\11d6e711__images.jpg
config - INFO - Got response <Response [200]> from http://t3.gstatic.com/images?q=tbn:ANd9GcRH8DNu8Q0BHGrdQW2vWtFRhWmhAurISHkI1LrA3MnalPVkPJ-FU97HHOw
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t3.gstatic.com\084a7c15__images.jpg
elements - INFO - File of type .jpg written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\t3.gstatic.com\084a7c15__images.jpg
config - INFO - Got response <Response [301]> from http://www.google.com/trends/embed.js?hl=en-US&q=abu%20abd%20al%20rahman%20ibn%20aqil%20al%20zahiri&content=1&cid=TIMESERIES_GRAPH_0&export=5&w=500&h=330
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/es.gif
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.writeopinions.com\f1643c95__es.gif
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.writeopinions.com\c69aeed5__abu-abd-al-rahman-ibn-aqil-al-zahiri.pwc
elements - INFO - File of type .gif written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.writeopinions.com\f1643c95__es.gif
elements - INFO - File of type .htm written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.writeopinions.com\c69aeed5__abu-abd-al-rahman-ibn-aqil-al-zahiri.pwc
elements - INFO - File of type .js written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\connect.facebook.net\en_US\2beb0397__all.js
config - INFO - Got response <Response [200]> from https://trends.google.com/trends/embed.js?hl=en-US&q=abu%20abd%20al%20rahman%20ibn%20aqil%20al%20zahiri&content=1&cid=TIMESERIES_GRAPH_0&export=5&w=500&h=330
elements - INFO - Writing file at location C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.google.com\trends\c98a0889__embed.js
elements - INFO - File of type .js written successfully to C:\work\www_ZNZRAC_ABC_1to500\URLs_cache\ZN_101772\www.google.com\trends\c98a0889__embed.js
The log output stops there when in a thread.
In a self-standing python interpreter:
kwargs = {'project_name' : 'dummy'}
pywebcopy.save_webpage(url='http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri', project_folder='c:\work', **kwargs)
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/robots.txt
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri
webpage - INFO - Starting save_complete Action on url: ['http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri']
parsers - INFO - Parsing tree with source: <<urllib3.response.HTTPResponse object at 0x045AD270>> encoding and parser <<lxml.etree.HTMLParser object at 0x044609F0>>
webpage - INFO - Starting save_assets Action on url: 'http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri'
webpage - Level 100 - Queueing download of <29> asset files.
webpage - INFO - Starting save_html Action on url: 'http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri'
webpage - INFO - WebPage saved successfully to c:\work\dummy\www.writeopinions.com\c69aeed5__abu-abd-al-rahman-ibn-aqil-al-zahiri.html
config - INFO - Got response <Response [200]> from http://t3.gstatic.com/images?q=tbn:ANd9GcRNmVn169BsI2COBefR47BN6eDbuKZ18B4AXdm-5un12uldcw4d6k9dwc0
elements - INFO - Writing file at location c:\work\dummy\t3.gstatic.com\11d6e711__images.jpg
elements - INFO - File of type .jpg written successfully to c:\work\dummy\t3.gstatic.com\11d6e711__images.jpg
config - INFO - Got response <Response [200]> from http://t0.gstatic.com/images?q=tbn:ANd9GcQEWTHxP9VC7soNKgEZpIegGh8zvZV1h1ABtQR5Gpt8V85Kqfx8ITUVv3A
elements - INFO - Writing file at location c:\work\dummy\t0.gstatic.com\9b67b38a__images.jpg
elements - INFO - File of type .jpg written successfully to c:\work\dummy\t0.gstatic.com\9b67b38a__images.jpg
config - INFO - Got response <Response [200]> from http://t3.gstatic.com/images?q=tbn:ANd9GcRH8DNu8Q0BHGrdQW2vWtFRhWmhAurISHkI1LrA3MnalPVkPJ-FU97HHOw
config - INFO - Got response <Response [200]> from http://t2.gstatic.com/images?q=tbn:ANd9GcQmJSdsNcOqsYnCA8qmUtwhfvmzo-VC6-jow3XF3Hq2EyW4N1vstbrNG_D6
elements - INFO - Writing file at location c:\work\dummy\t3.gstatic.com\084a7c15__images.jpg
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/logo.png
elements - INFO - Writing file at location c:\work\dummy\t2.gstatic.com\9a3a25ef__images.jpg
elements - INFO - Writing file at location c:\work\dummy\www.writeopinions.com\a0cc724d__logo.png
elements - INFO - File of type .png written successfully to c:\work\dummy\www.writeopinions.com\a0cc724d__logo.png
elements - INFO - File of type .jpg written successfully to c:\work\dummy\t3.gstatic.com\084a7c15__images.jpg
elements - INFO - File of type .jpg written successfully to c:\work\dummy\t2.gstatic.com\9a3a25ef__images.jpg
config - INFO - Got response <Response [301]> from http://www.google.com/trends/embed.js?hl=en-US&q=abu%20abd%20al%20rahman%20ibn%20aqil%20al%20zahiri&content=1&cid=TIMESERIES_GRAPH_0&export=5&w=500&h=330
config - INFO - Got response <Response [200]> from http://connect.facebook.net/en_US/all.js#xfbml=1
elements - INFO - Writing file at location c:\work\dummy\connect.facebook.net\en_US\2beb0397__all.js
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/es.gif
elements - INFO - Writing file at location c:\work\dummy\www.writeopinions.com\f1643c95__es.gif
config - INFO - Got response <Response [200]> from http://www.writeopinions.com/abu-abd-al-rahman-ibn-aqil-al-zahiri
elements - INFO - File of type .gif written successfully to c:\work\dummy\www.writeopinions.com\f1643c95__es.gif
elements - INFO - File of type .js written successfully to c:\work\dummy\connect.facebook.net\en_US\2beb0397__all.js
config - INFO - Got response <Response [200]> from https://trends.google.com/trends/embed.js?hl=en-US&q=abu%20abd%20al%20rahman%20ibn%20aqil%20al%20zahiri&content=1&cid=TIMESERIES_GRAPH_0&export=5&w=500&h=330
elements - INFO - Writing file at location c:\work\dummy\www.google.com\trends\c98a0889__embed.js
elements - INFO - File of type .js written successfully to c:\work\dummy\www.google.com\trends\c98a0889__embed.js
elements - INFO - Writing file at location c:\work\dummy\www.writeopinions.com\c69aeed5__abu-abd-al-rahman-ibn-aqil-al-zahiri.pwc
elements - INFO - File of type .htm written successfully to c:\work\dummy\www.writeopinions.com\c69aeed5__abu-abd-al-rahman-ibn-aqil-al-zahiri.pwc
core - INFO - Saved the Project as ZIP archive at c:\work\dummy.zip
core - INFO - Downloaded Contents Size :: 23 KB's
from pywebcopy.
from pywebcopy.
Related Issues (20)
- how to pass cookies with V7.0 HOT 5
- UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position 8: ordinal not in range(128) HOT 3
- Fails on tel links HOT 6
- Cannot download a website if it has invalid SSL certificate HOT 3
- How to change session header settings when getting a 403 Forbiden error HOT 1
- Relative Path Support
- Testing current full web crawling functionality
- Only Downloads HTML and Nothing Else HOT 4
- Inconsistency between image filename saved to disk and link in source file HOT 1
- "Blocked resource" error HOT 1
- Problem with paths with _ on them HOT 1
- UTF-8 encoding issues
- [WinError 3] The system cannot find the path specified: 'E:\\' HOT 2
- Pywebcopy save HOT 1
- Unreliable iterator based incremental parsing HOT 6
- Login before save HOT 14
- How to set cookies on a url HOT 3
- Incomplete Read HOT 13
- Fatal error when running the demo script HOT 1
- Bot being detected HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pywebcopy.