raviqqe/liche

Fast Link Checker for Markdown and HTML in Go

DEPRECATION NOTICE

Sorry this project is not actively maintained anymore! 😢 Please consider migrating to one of the alternatives listed below.

Alternatives

• muffet
  • Fast website link checker in Go
• lychee
  • A glorious link checker
  • This tool supports testing links both in local files and on websites.
• hyperlink
  • Checks folder of HTML for relative/internal links (no markdown or external websites)

Why is it not maintained anymore?

It's because we found several problems with the goals of the project and the amount of work it needs.

The goal of this software was originally to check links in Markdown files which are compiled into HTML files and served via HTTP servers. But that raises the following problems.

• We cannot test links which do not exist in the Markdown files.
  • For example, some markdown-based static site generators generate links at compile time.
  • e.g. automatic generation of table of contents
• We cannot test the behaviour of HTTP servers.
  • Different HTTP servers handles URLs differently.
  • e.g. trailing slashes, inference of page file extensions, ...

It needs a lot of work to support all these different use cases. In short, we need to emulate different Markdown file compilers and HTTP servers as well as web browsers.

But I still want this...

If you think this software is still valuable for you even in comparison with the alternatives listed above and want it to be maintained, please let us know by posting a new issue.

liche

liche is a command to check links' connectivity in Markdown and HTML files. It checks all a and img tags in specified files.

Installation

go get -u github.com/raviqqe/liche

• requires Go Modules

Usage

> liche --help
Link checker for Markdown and HTML

Usage:
	liche [-c <num-requests>] [-d <directory>] [-r] [-t <timeout>] [-x <regex>] [-v] <filenames>...

Options:
	-c, --concurrency <num-requests>  Set max number of concurrent HTTP requests. [default: 512]
	-d, --document-root <directory>  Set document root directory for absolute paths.
	-r, --recursive  Search Markdown and HTML files recursively
	-t, --timeout <timeout>  Set timeout for HTTP requests in seconds. Disabled by default.
	-x, --exclude <regex>  Regex of links to exclude from checking.
	-v, --verbose  Be verbose.

Examples

> liche file.md
> liche file1.md file2.md
> liche file.html
> liche file1.html file2.html
> liche -r directory # Search all Markdown and HTML files recursively.

Supported properties

• File extensions: .md, .html, .htm
• HTML tags: a, img
• HTML attributes: href, src
• URL schemes: http, https

Also supports relative and absolute paths. (Absolute paths need --document-root option.)

License

MIT

liche's Issues

Proxy support

Seems like the environment variables http_proxy, https_proxy and no_proxy are ignored by this tool

Feature request: --exclude-private-hosts

Thanks for this tool. We are using this tool with Github actions to make sure our docs don't have invalid urls.

One of the common pattern for us is that our docs have localhost addresses and domains with non-public TLD. Currently I am using --exclude pattern to skip those. But this making the --exclude pattern very long and hard to read. I think it will be very handy to add a --exclude-private-hosts to avoid private domains and private ips.

If you are open to this, I can send a pr.

go get fails.

Hello.

Seems like go get fails on go1.12.5 version. Is this expected?

go get github.com/raviqqe/liche:

go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/go-openapi/swag v0.19.4
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/mattn/go-colorable v0.1.2
go: finding github.com/valyala/fasthttp v1.4.0
go: finding github.com/prometheus/common v0.6.0
go: finding github.com/gogo/protobuf v1.1.1
go: finding github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4
go: finding github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: finding github.com/go-stack/stack v1.8.0
go: finding github.com/fatih/color v1.7.0
go: finding k8s.io/api v0.0.0-20190722141453-b90922c02518
go: finding github.com/stretchr/testify v1.3.0
go: finding github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815
go: finding k8s.io/apimachinery v0.0.0-20190719140911-bfcf53abc9f8
go: finding github.com/pkg/errors v0.8.0
go: finding gopkg.in/yaml.v2 v2.2.2
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a
go: finding github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90
go: finding github.com/golang/protobuf v1.3.1
go: finding github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223
go: finding github.com/kr/pretty v0.1.0
go: finding github.com/klauspost/compress v1.4.0
go: finding golang.org/x/net v0.0.0-20190613194153-d28f0bde5980
go: finding github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e
go: finding gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
go: finding github.com/go-logfmt/logfmt v0.4.0
go: finding github.com/stretchr/objx v0.1.0
go: finding github.com/raviqqe/liche v0.0.0-20190130153545-3b20c094c539
go: finding golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: finding github.com/julienschmidt/httprouter v1.2.0
go: finding github.com/sirupsen/logrus v1.2.0
go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f
go: finding github.com/prometheus/client_golang v1.0.0
go: finding golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: finding golang.org/x/net v0.0.0-20180911220305-26e67e76b6c3
go: finding github.com/davecgh/go-spew v1.1.0
go: finding github.com/golang/protobuf v1.2.0
go: finding gopkg.in/inf.v0 v0.9.0
go: finding github.com/elazarl/goproxy v0.0.0-20170405201442-c4fc26588b6e
go: finding sigs.k8s.io/yaml v1.1.0
go: finding github.com/google/uuid v1.1.1
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding github.com/spf13/pflag v1.0.3
go: finding gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: finding github.com/gogo/protobuf v1.0.0
go: finding github.com/hashicorp/golang-lru v0.5.0
go: finding github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: finding k8s.io/kube-openapi v0.0.0-20190709113604-33be087ad058
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/valyala/bytebufferpool v1.0.0
go: finding github.com/google/go-cmp v0.3.0
go: finding github.com/onsi/gomega v1.5.0
go: finding github.com/google/gofuzz v1.0.0
go: finding github.com/stretchr/objx v0.1.1
go: finding github.com/prometheus/procfs v0.0.2
go: finding github.com/beorn7/perks v1.0.0
go: finding gopkg.in/yaml.v2 v2.2.1
go: finding github.com/go-kit/kit v0.9.0
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633
go: finding github.com/prometheus/common v0.4.1
go: finding github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63
go: finding github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9
go: finding github.com/mattn/go-isatty v0.0.8
go: finding k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6
go: finding sigs.k8s.io/structured-merge-diff v0.0.0-20190525122527-15d366b2352e
go: finding github.com/googleapis/gnostic v0.0.0-20170426233943-68f4ded48ba9
go: finding golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f
go: finding github.com/fsnotify/fsnotify v1.4.7
go: finding github.com/onsi/ginkgo v1.8.0
go: finding github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96
go: finding github.com/go-openapi/spec v0.0.0-20160808142527-6aced65f8501
go: finding github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910
go: finding github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf
go: finding golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e
go: finding golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223
go: finding github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a
go: finding github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: finding github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c
go: finding github.com/kr/text v0.1.0
go: finding github.com/modern-go/reflect2 v1.0.1
go: finding k8s.io/klog v0.3.1
go: finding golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5
go: finding github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d
go: finding github.com/evanphx/json-patch v4.2.0+incompatible
go: finding golang.org/x/text v0.3.0
go: finding github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2
go: finding github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0
go: finding github.com/google/gofuzz v0.0.0-20161122191042-44d81051d367
go: finding github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973
go: finding github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1
go: finding github.com/davecgh/go-spew v0.0.0-20151105211317-5215b55f46b2
go: finding github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc
go: finding github.com/json-iterator/go v0.0.0-20180612202835-f2b4162afba3
go: finding github.com/onsi/ginkgo v1.6.0
go: finding golang.org/x/tools v0.0.0-20181011042414-1f849cf54d09
go: finding k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92
go: finding github.com/go-logfmt/logfmt v0.3.0
go: finding github.com/PuerkitoBio/purell v1.0.0
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea
go: finding golang.org/x/net v0.0.0-20180906233101-161cd47e91fd
go: finding golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4
go: finding golang.org/x/text v0.3.2
go: finding github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
go: finding github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87
go: finding github.com/go-kit/kit v0.8.0
go: finding gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
go: finding github.com/kr/pty v1.1.1
go: finding gopkg.in/fsnotify.v1 v1.4.7
go: finding github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f
go: finding github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d
go: finding github.com/stretchr/testify v0.0.0-20151208002404-e3a8ff8ce365
go: finding github.com/json-iterator/go v1.1.6
go: finding golang.org/x/text v0.0.0-20160726164857-2910a502d2bf
go: finding github.com/prometheus/client_golang v0.9.1
go: finding github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c
go: finding github.com/hpcloud/tail v1.0.0
go: finding golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
go: finding golang.org/x/net v0.0.0-20181114220301-adae6a3d119a
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding github.com/modern-go/reflect2 v0.0.0-20180320133207-05fbef0ca5da
go: finding github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d
go: finding github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46
go: finding golang.org/x/net v0.0.0-20170114055629-f2499483f923
go: finding github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff
go: finding github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: downloading github.com/raviqqe/liche v0.0.0-20190130153545-3b20c094c539
go: extracting github.com/raviqqe/liche v0.0.0-20190130153545-3b20c094c539
go: downloading github.com/kr/text v0.1.0
go: downloading github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815
go: downloading github.com/fatih/color v1.7.0
go: downloading golang.org/x/net v0.0.0-20190613194153-d28f0bde5980
go: downloading github.com/valyala/fasthttp v1.4.0
go: extracting github.com/kr/text v0.1.0
go: extracting github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815
go: extracting github.com/valyala/fasthttp v1.4.0
go: downloading github.com/klauspost/compress v1.4.0
go: downloading github.com/valyala/bytebufferpool v1.0.0
go: extracting github.com/valyala/bytebufferpool v1.0.0
go: extracting github.com/fatih/color v1.7.0
go: downloading github.com/mattn/go-isatty v0.0.8
go: downloading github.com/mattn/go-colorable v0.1.2
go: extracting github.com/mattn/go-isatty v0.0.8
go: downloading golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f
go: extracting github.com/mattn/go-colorable v0.1.2
go: extracting golang.org/x/net v0.0.0-20190613194153-d28f0bde5980
go: extracting golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f
go: extracting github.com/klauspost/compress v1.4.0
go: finding gopkg.in/russross/blackfriday.v2 v2.0.1
go: downloading github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e
go: extracting github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e
go: downloading gopkg.in/russross/blackfriday.v2 v2.0.1
build github.com/raviqqe/liche: cannot load gopkg.in/russross/blackfriday.v2: cannot find module providing package gopkg.in/russross/blackfriday.v2

Regex not excluding internal links

Hey!

Thank you for this amazing tool!

I've tried using it to do the link verification of a static site generator package (packit.dev), but I want to exclude all internal links (eg, ones beginning with "/" or "{"). Here's the example: https://regex101.com/r/De199r/1

I've tried the regex in all sorts of ways, but it doesn't ever seem to exclude them, from my understanding this might be due to the time of evaluation (it checks the internal before it checks if it should exclude)

You can see all my attempts to get this working in this repo https://github.com/JackMcKew/packit.dev and subsequent github actions fired with the errors https://github.com/JackMcKew/packit.dev/actions

Maybe I'm making a silly error, but I'd love to be able to use this package for this purpose

Thank you!

Question: Too many redirects detected

I have a link which gives the following error:

ERROR	https://forum.unity.com
		Too many redirects detected when doing the request

Is there a solution to this?

Support HTML files

Getting "Too Many Requests (HTTP error 429)" Error

I am getting this "Too Many Requests (HTTP error 429)" Error in the output file of my Link Checker when calling to external sites as shown below.

@raviqqe - Is there a way to get this resolved or to skip displaying these in the output file using Liche arguments?

Markdown link is truncated mistakenly

Hi, thanks for writing this action. I met the following error today:

But the markdown link I wrote was:

The link was mistakenly truncated with ) missing so the action reported a 404.

Could you please take a look at it?

Release tag.

Hey,

Thanks for great tool!
Can you tag some version? That would be bit easier to reference.

Check internal links against external host

It would be great if it was possible to prefix any relative and absolute paths for internal links with an external host to check against. E.g. our website is only partly published on GitHub while other parts a forum and a downloads directory structure, which cannot be reasonably hosted on GitHub as well. However, links from our main website to those are either relative or absolute paths and it would be great if liche was able to prefix them with our host/domain to check links against the live website.

Support crawling a website via HTTP

Hi,

thanks for publishing liche! Would you consider adding support for crawling a website via HTTP? I'd like to check the website for restic (a backup program written in Go) using something else than linkchecker. We have some rewriting magic going on via the webserver on https://restic.net, so checking the local files does not work completely.

I'd like to be able to do this (with liche):

$ linkchecker --check-extern --no-robots http://localhost:4000

Where on localhost on port 4000 I have an instance of Jekyll running, serving the files.

What's your take on that? I'd be willing to submit a PR if you're interested.

liche reports 404 links as OK

~/go/src/github.com/raviqqe/liche$ cat a.md
# Test
A [404](http://www.google.com/idontexist.md) link.
~/go/src/github.com/raviqqe/liche$ liche -v a.md 
a.md
        OK      http://www.google.com/idontexist.md

It's expected that a non-existing file fails the link verification.

Can you add [v] prefix to tag?

Go mod does not consider a tag valid without v prefix. Can you please also retag the last release as v0.1.1 ?

Make usable as a package

Hey,

Would you accept a change to make this CLI usable as a package?

Bad dependency

$ go get -u github.com/raviqqe/liche
package github.com/russross/blackfriday/v2: cannot find package "github.com/russross/blackfriday/v2" in any of:
	/home/go/src/github.com/russross/blackfriday/v2 (from $GOROOT)
	/home/ccallegar/go/src/github.com/russross/blackfriday/v2 (from $GOPATH)

See russross/blackfriday#565

Go Get fails on blackfriday module

I saw a similar issue (#25), but think this might be slightly different.

My Travis-CI jobs recently started failing using Go 1.12.1 due to blackfriday/v2 modules not found? Could this be a Travis-CI issue instead for their Go environment? I did see recent changes to use modules instead: 1c8cb96

I can reproduce locally on go1.13.1 linux/amd64
From clean environment:

$ go version 
go version go1.13.1 linux/amd64
$ go get -u github.com/raviqqe/liche
package github.com/russross/blackfriday/v2: cannot find package "github.com/russross/blackfriday/v2" in any of:
	/usr/local/go/src/github.com/russross/blackfriday/v2 (from $GOROOT)
	/home/nlynchjo/go/src/github.com/russross/blackfriday/v2 (from $GOPATH)

And my Travis-CI logs:

$ go version
go version go1.12.10 linux/amd64
go.env
$ go env
9.89s$ go get -u github.com/raviqqe/liche
package github.com/russross/blackfriday/v2: cannot find package "github.com/russross/blackfriday/v2" in any of:
	/home/travis/.gimme/versions/go1.12.10.linux.amd64/src/github.com/russross/blackfriday/v2 (from $GOROOT)
	/home/travis/gopath/src/github.com/russross/blackfriday/v2 (from $GOPATH)
The command "go get -u github.com/raviqqe/liche" failed and exited with 1 during .

Parse # in the URLs

URLs with # returns 400

$ liche docs/README.md 
docs/README.md
	ERROR	https://docs.openshift.com/container-platform/4.1/cli_reference/administrator-cli-commands.html#create-kubeconfig
		Bad Request (HTTP error 400)

Invalid go.mod

After latest update to go.mod (#38), getting the following error:

go get -u github.com/raviqqe/liche
go: github.com/raviqqe/liche upgrade => v0.0.0-20200922051910-3354ae108473
go get: github.com/raviqqe/[email protected] requires
        gopkg.in/russross/[email protected]: parsing go.mod:
        module declares its path as: github.com/russross/blackfriday/v2
                but was required as: gopkg.in/russross/blackfriday.v2

Add retry parameter

sometimes http links fail for no particular reason and a retry would allow this to succeed. For example, it's kind of annoying when this fails in CI because one http link returns a 50x for no particular reason and returns a 200 the next run.

It would be great to add a -r (or similar) flag that adds retries to http[s]:// links, disabled by default to not change any behavior.

Happy to work on this if it's a desired feature!

Readme should show wildcard usage too

The ascii-cinema vid shows it, but it not apparent in the README

mode of operation for relative links needed

Re ![](/images/foo.png)

It could be that that image is available locally in ../images/ (Jekyll is laid out like this with posts in _posts/).

Re ![](../another_image_root/foo.png)

It could be that that image is relatively available online, say "http://some.site/another_image_root" and not at all in the clone/checkout (as the Jekyll case above)

Small read buffer with some sites

Using liche to check links in a set of markdown file documentation. A link to cuda is causing trouble. Not sure of a workaround, however this issue over in muffet seems useful but unsure: raviqqe/muffet#37

docker run -it --rm -v ${PWD}:/src raviqqe/liche:latest --concurrency 1 -v /src/test-links.md
/src/test-links.md
	ERROR	https://developer.nvidia.com/cuda-toolkit
		Error when reading response headers: small read buffer. Increase ReadBufferSize. Buffer size=4096, contents: "HTTP/1.1 200 OK
        Age: 553
        Cache-Control: no-cache
Cache-Control: no-store
Content-Language: en
Content-Security-Policy-Report-Only: default-src 'self' 'unsafe-eval' 'unsafe-inline' assets.adobedtm"..."developer.nvidia.com forums.developer.nvidia.com www.youtube.com 20762408p.rfihub.com a.rfihub.com platform.twitter.com s.dca0.com bid.g.doubleclick.net nvidia.demdex.net syndication.twitter.com stati"

The response header is about 5.8kb:

curl -D header_size.txt https://developer.nvidia.com/cuda-toolkit

ls -l header_size.txt
5844 Jun  8 14:10 header_size.txt

small read buffer error with current ReadBufferSize

Certain links cause fasthttp to return ErrSmallBuffer

To reproduce:

echo "[AWS Console](https://console.aws.amazon.com/support/v1#/case/history)" > test.md
liche test.md

Output:

test.md
        ERROR   https://console.aws.amazon.com/support/v1#/case/history
                Error when reading response headers: small read buffer. Increase ReadBufferSize. Buffer size=4096, contents: "HTTP/1.1 200 OK\r\nServer: Server\r\nDate: Fri, 15 Feb 2019 13:30:23 GMT\r\nContent-Type: text/html;charset=UTF-8\r\nContent-Length: 1004\r\nConnection: keep-alive\r\nCache-Control: private\r\nExpires: Thu, 01 Jan "..........cloudfront"

Looks like fasthttp has a setting ReadBufferSize which will need to be set.

bug: `Too Many Requests (HTTP error 429)` Error when visiting github.com

Thank you for your work! This is so helpful!

Here is the problem. When running liche on github actions, visiting github.com would return Too Many Requests (HTTP error 429), even when concurrency is set to 1.

Github Actions screenshot: https://github.com/chaos-mesh/chaos-mesh/pull/987/checks?check_run_id=1172843506#step:5:648

Github Rate Limited

I would expect that liche is commonly used on GH projects. I was looking to use it for an open source project.... however I keep running into GH rate limits. I understand that nonauth rate limits are 60/hr for OAuth it is 5000/hr. details: https://developer.github.com/v3/#rate-limiting

I think we need liche to support GH rate limit controls to be useful.

Invalid link can not be picked up

What happened:

Invalid link can not be picked up, https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/1.10/color/certified-kubernetes-1.10-color.png。

The output as follows:
[root]# liche -v design_v1.10.md
design_v1.10.md
...
OK https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
OK https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/1.10/color/certified-kubernetes-1.10-color.png
ERRORhttps://API.AdvertiseAddress:API.BindPort
Strconv.Atoi: parsing "api.bindport": invalid syntax

The link is invalid, by curl:
[root]# curl https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/1.10/color/certified-kubernetes-1.10-color.png
404: Not Found

The link in markdown file as follows:
kubeadm init and kubeadm join together provides a nice
user experience for creating a best-practice but bare Kubernetes cluster from scratch.
However, it might not be obvious how kubeadm does that.

Add --skip-tls-verification option

Hi!

Could you please consider to add --skip-tls-verification as it's done in https://github.com/raviqqe/muffet

Thanks

400 Bad Response

Hello!

Our team has started to notice failures when checking links on twitter.com. E.g.:

        ERROR   https://twitter.com/ashleymcnamara
                Bad Request (HTTP error 400)

When I access the above link in my browser it appears to be fine. Doing some research this might be related to legacy twitter being removed https://screenrant.com/twitter-legacy-nintendo-3ds-shut-down-date-december-2020/

Thanks,
Jake & @Callisto13

How to exclude multiple directories?

I've been trying to run liche in a my local project directory (.) in CI tests, and exclude multiple directories, such as vendor/ and some others. How can I match more than one dir prefix with exclude?

Thanks!

Publish binary releases for easier usage

Please consider uploading binary releases to GitHub Releases, so that people can quickly use this tool without installing a Go toolchain (which can be helpful in continuous integration). This is made easy thanks to tools like GoReleaser 🙂

Links to https://crates.io returning 404

This is probably down to rust-lang/crates.io#788 ; I guess adding support for custom headers to liche would help, but I am not sure whether this should rather be fixed on the crates.io side instead.

Add concurrent-reqs option

Lint referenced anchors

Right now, liche seems to only check that referenced documents are accessible.
It would be nice if Liche could also verify that referenced anchors actually exist in the referenced documents.

Add an option to specify document root

Provide a way to exclude links

Thank you for a great tool!
Could you please provide a way to exclude links from checking by providing a list of patterns, e.g. --exclude=localhost,EXAMPLE_DOMAIN*, or using a config file?

Wrong output

Hi,

I'm using liche to check recursively a folder ./_site/ with this command:

liche -r ./_site/

Sometimes it works fine, it prints only the results of the check

But others starts printing the following until I have to use ^C to stop it

Both cases are with the same command

Thanks in advance

Add -q | --quiet mode

For some of our repos the list gets very long. I would like to just print out the errors. I think the -q | --quiet mode could do that.