The first place to go in order to find a CID stored in Filecoin
This repo provides an indexer implementation that can be used to index data stored by a range of participating storage providers.
Released for production: The current production release is running at https://cid.contact
This project and is currently under active development ๐ง.
This assumes go is already installed.
Install storetheindex:
go get github.com/filecoin-project/storetheindex
Initialize the storetheindex repository and configuration:
storetheindex init
Optionally, edit the configuration
edit ~/.storetheindex/config
To run storetheindex as a service, run the daemon
command. The service watches for providers to index, and exposes a query / content routing client interface.
The daemon is configured by the config file in the storetheindex repository. The config file and repo are created when storetheindex is initialized, using the init
command. This repo is located in the local file system. By default, the repo is located at ~/.storetheindex. To change the repo location, set the $STORETHEINDEX_PATH
environmental variable.
After a configured amount of time without any updates from a provider (PollInterval
), the indexer will poll the provider at its last know publisher address, for any index updates. If there is no response from the provider after at least one attempt to poll, then the provider is considered inactive and is not returned in any find results. The indexer will continue polling on an interval (PollRetryAfter
) until a time limit (PollStopAfter
) is reached. If there is still no response to the poll attempts after this time limit is reached, then the provider is removed from the indexer and its records are garbage-collected and will need to be refetched.
The configuration values that control this are documented here, and their default values are specified here. A custom polling configuration may be applied for specific providers using the PollOverrides
configuration value to specify per-provider Polling configuration.
There are a number of client commands included with storetheindex. Their purpose is to perform simple indexing and lookup actions against a running daemon. These can be helpful to test that an indexer is working. These include the following commands:
Informational:
find
Find value by CID or multihash in indexerproviders
Show information about providers known to the indexerget
Get information about a specified providerlist
List the known providers
Administrative:
admin
Perform admin activities with an indexerallow
Allow advertisements and content from peerblock
Block advertisements and content from peerimport-providers
Import provider information from another indexerreload-config
Reload various settings from the configuration filesync
Sync indexer with provider
init
Initialize or upgrade indexer node config file
Testing:
import
Imports data to indexer from different sourcesregister
Register provider information with an indexersynthetic
Generate synthetic load to import in indexer
To see a list of available commands, see storetheindex --help
. For help with command usage, see storetheindex <command> --help
.
The storetheindex config file documentation