palewire/news-homepages

An open-source archive that gathers, saves, shares and analyzes news homepages

An open-source archive that gathers, saves, shares and analyzes news homepages

Links

• Task runner: github.com/palewire/news-homepages-runner
• Internet Archive: archive.org/details/news-homepages
• Mastodon mastodon.palewi.re/@newshomepages
• Telegram: t.me/newshomepages
• Documentation: palewi.re/docs/news-homepages/
• Packaging: pypi.org/project/newshomepages
• How to add a site: palewi.re/docs/news-homepages/adding.html

news-homepages's Issues

Find some way to link the wayback URLs with the screenshot images

Create a utils function for getting the javascript

Figure out how to set the title attribute of the archive.org file

Add readability.js job that is pulling out the content of the page

Add utils module autodoc to the docs site

commit the latest images to the repo on each run

Fix @USATODAY: Masthead ad and cookies footer bar

Could we block ads? Should we block ads?

Abstract back Internet Archive action into a reusable app for the Marketplace

Tweet Internet Archive link

Handle TooManyRequests error in wayback.py

Example here

Fix @newsobserver and @theobserver: Blocked

Install an ad blocker

Save a folder to the artifacts so that we can start introducing more files

Fix @segodnya_online: Cloudflare blocker

Add README instructions on how to add a site

Fix @vcstar: Popover

Add Discord to the README

Fix @abqbizfirst: Blocked

cc @chrislkeller

Documentation Error

I was testing something, and I found this documentation error. In the documents, under Test the screenshot, the command to test should be pipenv run python -m newshomepages.shoot single your-sites-handle instead of pipenv run python -m newshomepages.tweet single your-sites-handle

I tried updating adding.md file, but my blacken-docs tests are failing. I'm not sure why, but I am debugging. I just wanted to create an issue in case someone else got confused with the command.

Add a Sphinx documentation site

Split out a utils file with common functions

Introduce system to link multiple sites to a single javascript file

Create a Twitter workflow for bundles

Add alt text to the images

Add `bundle` subcommands to the docs

Automate a status report

Total sites
Total screenshot
Date range

Add outputs to the actions that return the URLs they create

Fix @postcrescent and other chain papers: Popup

More seen here in Wisconsin

Sort Twitter handles alphabetically for the list

Add step that uploads images to the Internet Archive

Fix @thetimes: Popover

This still shows up for me, despite having a JavaScript include that works in my browser console.

document.querySelectorAll(
  '.message-container'
).forEach(el => el.remove())

Add ability to customize width and height by site

Add a paragraph and link to the docs explaining where to get Discord token

Add archive.py support for bundles

Add user-agent to savepagenow hits

Add timestamp in local time to tweets

This will require adding the timezone to each row in the sources sheet

Add telegram to the README

Add a new command to scrape all of the hyperlinks from each site

Fix @kpcc: Popover

Create a system for bundling multiple screenshots into a single tweet

Add tests

Explain bundles in the docs

Group all the commands in a module folder

Fix @htrnews: Popover

Gonna pull this one from the routine until we get the popup handled. cc @chrislkeller

Take full page shot as well

Discord bundle command isn't working

Write out a YAML file on the fly so that we can use the `multi` command in shot-scraper

Add support for a threaded bundle with more than four sources

Autoformat a table of all the bundles and sites for the docs