httpreserve/workbench

Client app for httpreserve pkg that generates CSV, JSON, HTTP, and BoltDB

httpreserve workbench

UPDATE: A cli-only version of this tool is now available, see httpreserve-linkstat

Bulk analysis tool for the preservation of web links inside documentary heritage.

More information on the project can be found via the Open Preservation Foundation: http://openpreservation.org/blog/2017/05/19/links-all-of-the-way-down-httpreservation-of-documentary-heritage/

Outputs:

• JSON
• CSV (Example: Million Dollar Webpage)
• BoltDB (embedded key-value database)
• webapp (pictured)

Builds for different platforms will be available soon. Otherwise, install Go, run Go Get, and Go Build.

Archiving Weblinks

• Find and Connect Project: Nicola Laurent on the impact of broken links.
• Binary Trees? Automatically Identifying the links between born digital records: I write about hyperlinks as a public record in own right when submitted as part of a documentary heritage.
• HiberActive Pilot A scholarly publishing tool that extracts URLs, returns both the original URL and a perma-link.

License

GNU General Public License Version 3. Full Text

workbench's Issues

Research: Internet Archive Links

https://archive.org/search.php?query=%22http%3A%2F%2F%22&sin=TXT&page=2

Research: No Robot links such as Jezebel

Goroutine issues (Share by communicating)

Have managed to get control of some of the goroutines being used in the code but i am using buffering to manipulate lists at runtime. Mainly in webapp.go

This seems to be counter to share by communicating ethos.

Have a look at ways this can be fixed.

ip method for resetting linkstats delivery

responseWriter... r.RemoteAddr