ipfs/helia-http-gateway

Run Helia NodeJS in a Docker Container.

helia-http-gateway

HTTP IPFS Gateway implemented using Helia

About

A Dockerized application that implements the HTTP IPFS-gateway API spec and responds to the incoming requests using Helia to fetch the content from IPFS.

Run from the github container registry

$ docker run -it -p 8080:8080 ghcr.io/ipfs/helia-http-gateway:latest

See https://github.com/ipfs/helia-http-gateway/pkgs/container/helia-http-gateway for more information.

Run Using Docker Compose

$ docker-compose up

Run Using Docker

Build

$ docker build . --tag helia-http-gateway:local

Pass the explicit platform when building on a Mac.

$ docker build . --platform linux/arm64 --tag helia-http-gateway:local-arm64

Running

$ docker run -it -p 8080:8080 -e DEBUG="helia-http-gateway*" helia-http-gateway:local # or helia-http-gateway:local-arm64

Run without Docker

### Build

$ npm run build

Running

$ npm start

Supported Environment Variables

Variable	Description	Default
DEBUG	Debug level	''
FASTIFY_DEBUG	Debug level for fastify's logger	''
PORT	Port to listen on	8080
HOST	Host to listen on	0.0.0.0
USE_SUBDOMAINS	Whether to use origin isolation	true
METRICS	Whether to enable prometheus metrics. Any value other than 'true' will disable metrics.	true
USE_BITSWAP	Use bitswap to fetch content from IPFS	true
USE_TRUSTLESS_GATEWAYS	Whether to fetch content from trustless-gateways or not	true
TRUSTLESS_GATEWAYS	Comma separated list of trusted gateways to fetch content from	Defined in Helia
USE_LIBP2P	Whether to use libp2p networking	true
ECHO_HEADERS	A debug flag to indicate whether you want to output request and response headers	false
USE_DELEGATED_ROUTING	Whether to use the delegated routing v1 API	true
DELEGATED_ROUTING_V1_HOST	Hostname to use for delegated routing v1	https://delegated-ipfs.dev
USE_DHT_ROUTING	Whether to use @libp2p/kad-dht for routing when libp2p is enabled	true
USE_SESSIONS	If true, use a blockstore session per IPFS/IPNS path	true

See the source of truth for all process.env.<name> environment variables at src/constants.ts.

You can also see some recommended environment variable configurations at:

• ./.env-all
• ./.env-delegated-routing
• ./.env-gwc
• ./.env-trustless-only

Running with custom configurations

Note that any of the following calls to docker can be replaced with something like MY_ENV_VAR="MY_VALUE" npm run start

Disable libp2p

$ docker run -it -p $RPC_PORT:5001 -p $HTTP_PORT:8080 -e DEBUG="helia-http-gateway*" -e USE_LIBP2P="false" helia

Disable bitswap

$ docker run -it -p $RPC_PORT:5001 -p $HTTP_PORT:8080 -e DEBUG="helia-http-gateway*" -e USE_BITSWAP="false" helia

Disable trustless gateways

$ docker run -it -p $RPC_PORT:5001 -p $HTTP_PORT:8080 -e DEBUG="helia-http-gateway*" -e USE_TRUSTLESS_GATEWAYS="false" helia

Customize trustless gateways

$ docker run -it -p $RPC_PORT:5001 -p $HTTP_PORT:8080 -e DEBUG="helia-http-gateway*" -e TRUSTLESS_GATEWAYS="https://ipfs.io,https://dweb.link" helia

E2E Testing

We have some tests enabled that simulate running inside of ProbeLab's Tiros, via playwright. These tests request the same paths from ipfs.io and ensure that the resulting text matches. This is not a direct replacement for gateway conformance testing, but helps us ensure the helia-http-gateway is working as expected.

By default, these tests:

1. Run in serial
2. Allow for up to 5 failures before failing the whole suite run.
3. Have an individual test timeout of two minutes.

Run e2e tests locally

$ npm run test:e2e # run all tests
$ npm run test:e2e -- ${PLAYWRIGHT_OPTIONS} # run tests with custom playwright options.

Get clinicjs flamecharts and doctor reports from e2e tests

$ npm run test:e2e-doctor # Run the dev server with clinicjs doctor, execute e2e tests, and generate a report.
$ npm run test:e2e-flame # Run the dev server with clinicjs flame, execute e2e tests, and generate a report.

Metrics

Running with METRICS=true will enable collecting Fastify/libp2p metrics and will expose a prometheus collection endpoint at http://localhost:8080/metrics

js-libp2p metrics are collected by default, but can be disabled by setting USE_LIBP2P_METRICS=false. Find out more details at https://github.com/libp2p/js-libp2p/tree/main/packages/metrics-prometheus

Viewing metrics

See Metrics config for more information on how to view the generated metrics.

Install

$ npm i helia-http-gateway

License

Licensed under either of

• Apache 2.0, (LICENSE-APACHE / http://www.apache.org/licenses/LICENSE-2.0)
• MIT (LICENSE-MIT / http://opensource.org/licenses/MIT)

Contribute

Contributions welcome! Please check out the issues.

Also see our contributing document for more information on how we work, and about contributing in general.

Please be aware that all interactions related to this repo are subject to the IPFS Code of Conduct.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

helia-http-gateway's Issues

bug: requesting `ipfs/<CIDv0>` path breaks with error

example url: http://localhost:8888/ipfs/QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D

redirected to url http://qmsnuwmxptjzdljpkrarxbms2ju2oanvrgbr2xwbie9b2d.ipfs.localhost:8888/

Error

{
  "statusCode": 500,
  "error": "Internal Server Error",
  "message": "To parse non base32 or base58btc encoded CID multibase decoder must be provided"
}

more info: https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway

bug: /ipns/strn.network request results in invalid request

I'm not sure which request is kicking off that is causing this (I assume it's not the primary call, and one of the children calls) but I have consistently reproduced this error in the tests i have been doing over at #18 (comment)

[11:27:26.544] ERROR (94466): helia-http-gateway:fastifyPath: /ipfs/ is not valid, provide path as /ipfs/<cid> or /ipns/<path>
    reqId: "req-z"
    req: {
      "method": "GET",
      "url": "/ipns/strn.network",
      "hostname": "localhost:8080",
      "remoteAddress": "127.0.0.1",
      "remotePort": 51797
    }
    res: {
      "statusCode": 500
    }
    err: {
      "type": "Error",
      "message": "Path: /ipfs/ is not valid, provide path as /ipfs/<cid> or /ipns/<path>",
      "stack":
          Error: Path: /ipfs/ is not valid, provide path as /ipfs/<cid> or /ipns/<path>
              at HeliaFetch.parsePath (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-http-gateway/dist/src/heliaFetch.js:69:19)
              at HeliaFetch.fetchPath (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-http-gateway/dist/src/heliaFetch.js:108:63)
              at HeliaFetch.fetchIpns (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-http-gateway/dist/src/heliaFetch.js:151:21)
              at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
              at async HeliaFetch.fetch (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-http-gateway/dist/src/heliaFetch.js:91:28)
              at async HeliaServer._fetch (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-http-gateway/dist/src/heliaServer.js:116:35)
              at async HeliaServer.fetchWithoutSubdomain (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/helia-http-gateway/dist/src/heliaServer.js:105:13)
    }

feat: helia-http-gateway as a service worker (phase 2)

This issue is intended to pull out work mentioned in ipfs/helia#275 and required for IPFS dApps working group into it's own epic:

Tasks

Beta Give feedback

1. Create helia-http-gateway-sw: a service worker version of all the helia-http-gateway logic
2. Create helia-service-worker browser gateway that can work with probelab's tiros

Some important links to previous discussions:

• ipfs/helia#275 (comment)

feat: improve configuration options

see comment from @achingbrain at #63 (comment):

I wonder if we couldn't just merge the USE_BITSWAP and USE_LIBP2P vars to just be use @helia/http or use helia?

Like, USE_HTTP_GATEWAYS_ONLY or something to select @helia/http.

---

I think the original intent was to surface metrics for the following:

1. helia with libp2p & bitswap + trustless gateways + delegated routing
2. helia with libp2p and delegated routing only (no bitswap)
3. helia via trustless gateways only.

so we can see where fetching is the least performant and target improvements there. ipns://probelab.io/websites/#websites-http-comparison-ttfb-p90 has some explainers.

fix: helia-docker container dies when running in tiros

We need to move away from using an in-memory data-store because there is no way to limit the amount of memory used. @wemeetagain mentioned that Lodestar uses an in-mem datastore that flushes to file-system. The code for that is https://github.com/ChainSafe/lodestar/blob/unstable/packages/beacon-node/src/network/peers/datastore.ts#L71

If lodestar has a datastore we can use, we should use that, otherwise we should migrate to use file-system directly and then update to use lodestar strategy.

from #18 (comment)

---

log-events-viewer-result.csv

Action items before close

Beta Give feedback

1. ensure the helia-http-gateway version deployed on Tiros is the latest version
2. Use file-based datastore
3. Use mem->file flushing datastore & blockstore
4. Investigate potentially hanging promises for helia-all & helia-delegated-routing? tiros jobs not finishing: https://github.com/plprobelab/probelab-infra/issues/87

fix: remove duplicate ECHO_HEADERS check?

helia-http-gateway/src/index.ts

Lines 298 to 317 in b4531c0

	if ([ECHO_HEADERS].includes(true)) {
	app.addHook('onRequest', async (request, reply) => {
	if (ECHO_HEADERS) {
	log('fastify hook onRequest: echoing headers:')
	Object.keys(request.headers).forEach((headerName) => {
	log('\t %s: %s', headerName, request.headers[headerName])
	})
	}
	})
	app.addHook('onSend', async (request, reply, payload) => {
	if (ECHO_HEADERS) {
	log('fastify hook onSend: echoing headers:')
	const responseHeaders = reply.getHeaders()
	Object.keys(responseHeaders).forEach((headerName) => {
	log('\t %s: %s', headerName, responseHeaders[headerName])
	})
	}
	return payload
	})

feat: METRICS env var should enable libp2p promclient metrics?

helia-http-gateway/src/constants.ts

Lines 7 to 12 in bce0558

	/**
	* If set to any value other than 'true', we will disable prometheus metrics.
	*
	* @default 'true'
	*/
	export const METRICS = process.env.METRICS ?? 'true'

we may want a separate env var for this

came up during https://pl-strflt.notion.site/2023-10-26-Helia-Working-Group-ac880b0ddb244f64bb224367167e2ad7?pvs=4

Backpressure is ignored while writing responses

The return value of reply.raw.write is ignored so there's no way to react to backpressure.

https://github.com/ipfs/helia-http-gateway/pull/63/files#r1552002067

feat: add openTelemetry instrumentation

We should enable openTelemetry instrumentation via an environment variable

Resources

• https://github.com/open-telemetry/opentelemetry-js
• https://opentelemetry.io/docs/instrumentation/js/exporters/#prometheus

Add helia maintainers to this repo

We can't all push branches to this repo. @whizzzkid please can you open a PR to https://github.com/ipfs/github-mgmt giving us all perms on this repo.

Failed to install `helia-http-gateway` using NPM

Seems that the app is not yet published on NPM, so npm i helia-http-gateway fails.
https://www.npmjs.com/search?q=helia-http-gateway

The Docker installation though works but it doesn't fit my needs. Is there a way to install the package as NPM dependency? What do you suggest?

Rename this package (?), clarify what's in scope, and describe how to configure

Naming

Is the focus of this repo really around "Docker images for Helia"?

From my poking, it seems like the real value of this repo is that it implements some of the HTTP Gateway APIs and uses Helia behind the scenes. One of the ways we package/distribute this is via Docker.

As a result, maybe call this helia-http-gateway ?

(This is similar to https://github.com/ipfs/bifrost-gateway. It has Docker distribution, but the defining attribute is that it provides "a lightweight IPFS Gateway daemon backed by a remote data store."

Scope

1. Can we denote how much of https://specs.ipfs.tech/http-gateways/ we're attempting to implement currently?
2. Lets also call out what of https://docs.ipfs.tech/reference/kubo/rpc/ we're implementing
3. (I thought I saw it mentioned somewhere, but can't find it now.). Can we hook up the HTTP Gateway Conformance test suite to this? That is the canonical way to show how much this package supports. (Lets create a tracking issue for this if we don't have one.)

Configurability

How does someone customize the Helia config that is used? I have no expertise on how much configurability should be allowed outside of the docker image build vs. asking users to create their own docker image. That said, we're definitely going to want configurability here (e.g., run this with "trustless gateway fallback" enable and disabled.)

Tasks

Beta Give feedback

1. Rename to helia-http-gateway
2. docs: Denote ipfs-http-gateway specs coverage. #55
   +0
3. Make helia configurable using env vars. #19
   +0

docs: Denote ipfs-http-gateway specs coverage.

Related work: #57

feat: enable gateway-conformance tests

similar to https://github.com/ipfs/bifrost-gateway/blob/70f10e7b9688cd504f881d57ff3fa71215813e16/.github/workflows/gateway-conformance.yml

Currently passing tests on #57 (last updated 2023-12-04):

`subdomain-ipns-gateway`

PASS: TestMetadata (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28direct_HTTP%29/Status_code (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28HTTP_proxy%29/Status_code (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28direct_HTTP%29#01/Status_code (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28HTTP_proxy%29#01/Status_code (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28direct_HTTP%29#02/Status_code (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28HTTP_proxy%29#02/Status_code (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28direct_HTTP%29#03/Status_code (0.00s)
PASS: TestGatewaySubdomainAndIPNS/request_for_%7BCIDv1-libp2p-key%7D.ipns.%7Bgateway%7D_returns_expected_payload_%28HTTP_proxy%29#03/Status_code (0.00s)

`subdomain-ipfs-gateway`

PASS: TestMetadata (0.00s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28direct_HTTP%29/Status_code (0.00s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28direct_HTTP%29/Body (0.00s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28direct_HTTP%29 (0.99s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28HTTP_proxy%29/Status_code (0.00s)
PASS: TestGatewaySubdomains/valid_file_and_subdirectory_paths_in_directory_listing_at_%7Bcid%7D.ipfs.example.com_%28direct_HTTP%29/Status_code (0.00s)
PASS: TestGatewaySubdomains/valid_file_and_subdirectory_paths_in_directory_listing_at_%7Bcid%7D.ipfs.example.com_%28HTTP_proxy%29/Status_code (0.00s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28direct_HTTP%29#01/Status_code (0.00s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28direct_HTTP%29#01/Body (0.00s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28direct_HTTP%29#01 (0.00s)
PASS: TestGatewaySubdomains/request_for_%7BCID%7D.ipfs.example.com_should_return_expected_payload_%28HTTP_proxy%29#01/Status_code (0.00s)
PASS: TestGatewaySubdomains/valid_file_and_subdirectory_paths_in_directory_listing_at_%7Bcid%7D.ipfs.example.com_%28direct_HTTP%29#01/Status_code (0.00s)
PASS: TestGatewaySubdomains/valid_file_and_subdirectory_paths_in_directory_listing_at_%7Bcid%7D.ipfs.example.com_%28HTTP_proxy%29#01/Status_code (0.00s)

Tasks

Beta Give feedback

1. Figure out how to load fixtures (need to do via new api-endpoint if we're using in-memory block/data stores)
2. Address failure for http://localhost:8090/ipns/?uri=ipns%3A%2F%2Fdnslink-subdomain-gw-test.example.org requests
3. fix http://localhost:8090/ipns/dnslink-subdomain-gw-test.example.org/wiki
4. fix `http://k51qzi5uqu5dk3v4rmjber23h16xnr23bsggmqqil9z2gduiis5se8dht36dam.ipns.localhost:8090
5. fix http://localhost:8090/ipns/QmVujd5Vb7moysJj8itnGufN7MEtPRCNHkKpNuA4onsRa3
6. fix IPNS resolution using IPNS fixtures loaded in kubo

fix: express-session should not be used in production

From docker logs:

2023-10-23 19:04:11 Tue, 24 Oct 2023 02:04:11 GMT express-session deprecated undefined resave option; provide resave option at file:/app/dist/src/index.js:14:9
2023-10-23 19:04:11 Tue, 24 Oct 2023 02:04:11 GMT express-session deprecated undefined saveUninitialized option; provide saveUninitialized option at file:/app/dist/src/index.js:14:9
2023-10-23 19:04:11 Warning: connect.session() MemoryStore is not
2023-10-23 19:04:11 designed for a production environment, as it will leak
2023-10-23 19:04:11 memory, and will not scale past a single process.

Make helia configurable using env vars.

A few points of interest:

1. Kubo has experience with ENV_VARs and years of development that we can benefit from
2. Helia has no CLI (currently, empty repo exists at https://github.com/ipfs/helia-cli) so.. we don't have this yet.

A few potential paths forward

• We may just add ENV_VAR support to all config options directly in Helia
• May be useful to create a package @helia/env-vars-to-config type package that could be consumed by helia-cli or any other users of helia for populating their config, that consolidates all our environment variables

Env vars should be able to configure a few things:

• trustless gateway urls to use with blockBrokers
• libp2p transports?
• others

fix: uncaughtException needs more best practices

Tasks

Beta Give feedback

1. default ALLOW_UNHANDLED_ERROR_RECOVERY to false
2. Add warning in readme
3. Add default known-recoverable uncaughtExceptions
4. Add listener to req & resp for ERR_STREAM_PREMATURE_CLOSE

helia-http-gateway/src/index.ts

Lines 273 to 280 in 6f10038

	const uncaughtHandler = (error: any): void => {
	log.error('Uncaught Exception:', error)
	if (ALLOW_UNHANDLED_ERROR_RECOVERY && (RECOVERABLE_ERRORS === 'all' || RECOVERABLE_ERRORS.includes(error?.code) || RECOVERABLE_ERRORS.includes(error?.name))) {
	log.trace('Ignoring error')
	return
	}
	void closeGracefully('SIGTERM')
	}

related:

• #18
• #102 (comment)

cc @achingbrain

bug: replace setTimeout with AbortSignal.timeout

#102 (comment)

helia-http-gateway/src/helia-server.ts

Lines 270 to 273 in 6f10038

	setTimeout(() => {
	this.log.trace('request timed out for url "%s"', url)
	opController.abort()
	}, timeout)

feat: implement health-check

related:

• https://github.com/libp2p/js-libp2p-amino-dht-bootstrapper/blob/3a66d65110275ec5de99d0eeeff0f99560b7fb89/src/health-check.ts
• https://github.com/plprobelab/probelab-infra/pull/75/files#r1365542171

subdomain gateways

While working on minimal http gateway integration with tiros, a weird quirk of browsers shows when accessing IPNS links. e.g: http://localhost:8080/ipns/probelab.io

The requests are relative to /ipns/<dns-link>/asset but the browser considers it /asset so a redirect has to be made to the right asset link. However, it's up to the browser to follow the redirect or just display asset not found (broken image link). This problem is solved in subdomain gateways as the url transforms into a subdomain and that makes everything relative to the host load correctly.

This issue will focus on loading content using subdomain gateways.

bug: `/ipns/en.wikipedia-on-ipfs.org` fails with subdomain redirects

http://localhost:8888/ipns/en.wikipedia-on-ipfs.org ends up redirecting to http://wiki.ipns.localhost:8888/ and erroring, because that is invalid.

debugging: fs.ls directory contents for more debugging information

helia-http-gateway/src/heliaFetch.ts

Lines 193 to 195 in bce0558

	const path = this.sanitizeUrlPath(`${directoryPath}/${file}`)
	this.log('Trying to get root file:', { file, directoryPath })
	const stats = await this.fs.stat(cid, { path })

came up during https://pl-strflt.notion.site/2023-10-26-Helia-Working-Group-ac880b0ddb244f64bb224367167e2ad7?pvs=4