abiyani / automate-save-page-as Goto Github PK
View Code? Open in Web Editor NEWAutomate browser's "Save Page As" operation
License: Apache License 2.0
Automate browser's "Save Page As" operation
License: Apache License 2.0
Is it possible to perform actions that ASPA needs to do without needing this function (so without asking to focus a window or to know which one is focused)? Is it possible to create an argument to skip this step? Because I don't think X11 installed on WSL2 support EWMH. Or maybe you have an idea for this WM to support it?
In an already running browser, after the newly created window is closed, a Downloads list is still left across the bottom of the screen.
So (optionally) zapping that is also needed, for a perfect script.
The "X" on its right side needs to be clicked (in chromium at least).
There is a problem when the destination file already exists.
To avoid handling a popup dialog, I suggest to download to temprorary name first, then move to the destination just by mv
.
When Chrome is used on KDE, the 'save as' popup is a different file-selection-widget and doesn't have default focus on the 'filename' (like gnome/gtk)
Any pointers to make this tool work in KDE environment is appreciated.
When running the script under Ubuntu 16.04 running Unity the chrome browser opens with the URL. The save dialog box appear. But then the "Type your command" popup appears as if ALT
key is pressed. So the full path gets added to the command box instead of the save dialog and the dialog does not close. It's caused by the fix done for issue #1.
If the above issue is fixed, after saving the page, when the tab close (ALT+F4
) is invoked, it is changing to text mode as if (CTRL+ALT+F4
) is pressed.
I have fixed both the issues but tested only in Ubuntu 16.04. Will send the PR.
what is that sync option here
browser_wid="$(xdotool search --sync --onlyvisible --class "${browser}" | head -n 1)"?
it just never worked with that option. Is it important? Please explain.
thanks for the nice tool.
Hi, I am using Firefox Save as on MAC, and I was wondering how would I edit the script to implement Save As Complete Website?
thanks!
Brian
KUbuntu 18.
During the first run in a session, it appears as though the Firefox browser is first warming up. It registers the "index.html" as the name of the page to be saved, which is wrong. I click on cancel and close the browser myself. Later on, it works well automatically.
Hi, can this program not work with modern pages, where the pages are loaded gradually via scrolling? I would like to use this tool for a page of this type. The page reads only what is visible and scrolls down through AJAX to generate additional parts of the source code. So after loading, the page has only 40 lines, this tool sees it. Really, this page has, after scrolling down, for example, 10762 lines. How can I get a whole page of 10762 lines?
Thank you all for the tips.
I added --dialog-wait-time.
(In fact you could consider not hard wiring all the --delays too...)
I am using curl to login and then after login I am trying to save page. Since I can't get wget to replicate FireFox Save As Website Complete, I am hoping to use this Save Page As script.
thanks!
Brian
First and foremost, Thank you for writing this program. It saved my bacon when i needed to save about 250 pages and I can't thank you enough.
Secondly I use Firefox developer branch and the program didn't want to use it. It's an easy fix as i just linked firefox-developer to Firefox and noticed no issues. Personally i'd like to see this fixed from inside the program as it is more elegant.
ps: firefox branches: firefox-developer,firefox-beta,firefox-nightly,(deprecated firefox-aurora).
pps:I'm in the middle of my exams right now and don't have the time to fix it, but if there's no-one else I definitely will.
I'm running the script like this:
./save_page_as -d /tmp -b chromium-browser --load-wait-time 1.5 --save-wait-time 3 "$URL"
Sometimes, while the script is saving the file, it shows &tmp on the save dialog. Sometimes it works correctly. Im also using another script to call save_page_as with several different urls.
What could be the problem?
when i do ctrl+S to overwrite existing file - it fails....
thanks
//Close the browser tab/window (Ctrl+w for KDE, Ctrl+F4 otherwise)
This is the KDE desktop, but the Firefox is never closing by itself at the end of the operation.
Other small issues.
I noticed that the script does not work for a freshly installed Firefox - only the first time.
I noticed also, that the script did not work today only on the first execution of this test run:
./save_page_as -b firefox -d tmp "https://www.bing.com"
Might be helpful when used in scripting scrapes. I suspect Firefox will not hold the 250 pages I'm trying to scrape. =)
Feature suggestion:
Add an option to save the rendered HTML from Firefox's Gecko's webengine (Inspector's "Copy Outer HTML").
When i start the script i get then Error messages
Error: Can't open display: (null) Failed creating new xdo instance
I am running it with firefox on Ubunut 14.04
Thanks for this! Real handy tool.
It's defaulting to google-chrome, but if google-chrome isn't installed it just runs indefinitely.
Building an error message off the exit status of "which $browser" might be an easy fix.
./save_page_as -b firefox --load-wait-time 240 https://trade.aliexpress.com/order_detail.htm?orderId=nnn
Page loads completely but the following message is issued
XGetWindowProperty[_NET_WM_DESKTOP] failed (code=1)
Ubuntu 16.04 MATE 1.12.1 Firefox 64.0 (64-bit) xdotool 1:3.20150503.1-2 (Ubuntu native)
I hope this can help and will repay, thanks for sharing
Hello,
Thanks for the script :)
I use Debian 9 stretch, I have installed xdotools.
Whether with firefox or chrome, the page opens, but there are no catches.
In logs, I have this:
Aug 8 09:19:21 xxxxxxxxxxxxxxx kerrnel: [ 1258.634385] xdotool[1922]: segfault at 20 ip 00007face0fcb346 sp 00007ffd3d6ac490 error 4 in libxdo.so.3[7face0fc4000+b000]
Aug 8 09:25:08 xxxxxxxxxxxxxxxxxxxx kernel: [ 1605.317681] xdotool[2194]: segfault at 20 ip 00007f3cc379f346 sp 00007ffdbb266c20 error 4 in libxdo.so.3[7f3cc3798000+b000]
Aug 8 09:26:28 xxxxxxxxxxxxxxxxxx kernel: [ 1684.747747] xdotool[2323]: segfault at 20 ip 00007f1acba36346 sp 00007ffe012508c0 error 4 in libxdo.so.3[7f1acba2f000+b000]
Do you have any idea of ββthe problem ?
thank you
Really wish there was a way to tell it that on Debian chromium is called chromium, without needing to edit the source.
SPA.diff.txt
When I run:
./save_page_as 'https://www.tradingview.com/symbols/ETHUSD/' --destination /tmp/eth.html
I get:
XGetWindowProperty[_NET_WM_DESKTOP] failed (code=1)
I do have xdotool installed. Please advise...
Many thanks!
Maybe add a note to --help:
Note: Save to "Webpage Complete" or "HTML Only": the browser (at least
chromium) remembers the choice made last time you saved something...
So you might need to "prep" your browser by saving something by hand first,
if you don't like its current choice.
when i send several Tab keys after page is loaded - it works ok. But my enter key does not work....
Anything i am missing? thanks
Hi! Downloaded latest save_as... with xdotool-2.20110530.1 and my charset is UTF-8. Great tool but my first output of it is as follows:
save_page_as "www.facebook.com" --destination "/tmp/facebook_home_page.html" ERROR: Either -- destination ('/tmp/facebook_home_page.html') or --suffix ('') contains a non ascii or non-printable ascii character(s). 'xdotool' does not mingle well with non-ascii characters (https://code.google.com/p/semicomplete/issues/detail?id=14).
!!!! Will NOT proceed !!!!
is it a bug or something with my .bashrc ? Thanks
After downloading the script and finding it works great,
one wants to contact the authors and congratulate them etc.
However unless one remembers where one downloaded it from,
or searches for patterns on Google, all knowledge of where the script came from is lost.
I would put a
in the --help message.
The script is working perfectly fine (Opens FF, Saves, Closes FF) when I run it manually on terminal but I need it to run without human intervention, so I set a cron job to save the page everyday.
Unfortunately when the computer is locked, the script only opens the page on FF and does not save or close it. Im not sure if its an Ubuntu problem or a script problem (perhaps the keyboard shortcuts dont work when the computer is not logged in?), but I just wanted to check if anybody else has encountered and has a workaround.
Oh and I'm running Ubuntu 14.04LTS / Firefox. Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.