spicehq/docs

Spice documentation, built into docs.spice.ai

description
Welcome to the Spice.ai Cloud Platform documentation!

Welcome to Spice.ai

The Spice.ai Cloud Platform is a data and AI application platform; an AI-backend-as-a-service comprising of composable, ready-to-use data and AI building blocks including federated SQL query, machine learning, and cloud-scale, managed Spice.ai OSS.

{% hint style="info" %} This is the Spice.ai Cloud Platform documentation. If you are looking for the Spice.ai OSS project documentation please visit docs.spiceai.org.

{% endhint %}

Operating data and AI infrastructure can be painful.

😓 Query data from many sources and backends.

😓 Maintain expensive and often error prone ETL, data, query, and ML infrastructure.

😓 Constantly monitor data, training, and inferencing for errors, correctness, changes, and bottlenecks.

😓 Comply with governance, compliance, and privacy concerns.

Spice.ai makes it easy.

With Spice.ai, you can:

• Query real-time and historical data using SQL in seconds and fetch results in JSON or Apache Arrow for use with applications, ML, or libraries like NumPy and Pandas.
• Collaborate on datasets and ML models in one place through the new community-centric developer hub.
• Create, fork, and curate your own hosted datasets and ML models to share and build with the community indexed by the Spice.ai package manager, spicerack.org.
• Define and manage custom datasets in GitHub along with your application code.
• Bring and federate your own data or leverage 100TB+ of preloaded community data.

Spice.ai is preloaded with community data from ecosystems including Ethereum, Bitcoin, and EigenLayer along with enriched datasets for NFTs, DeFi, DEXs, ENS, Asset Prices, and more. Most datasets are open-source and available at github.com/spiceai/datasets.

Building blocks available in the Spice.ai platform today include:

• Petabyte-scale Cloud Data Platform
• Datasets and Views
• Spice Firecache (managed Spice.ai OSS)
• ML Models (in Preview)
• AI Predictions (in Preview)
• Community Data including Blockchain Data

Take it for a spin by starting with the getting started guide.

{% hint style="info" %} Feel free to ask any questions or queries to the team in Discord. {% endhint %}

docs's Issues

Release notes for January

Meaning of some columns not explained in the docs

Meaning of reserve0,reserve1,price0,price1 and price_cumulative not explained in the docs for pool_stats. Similar 0/1 columns in other dex tables.

GO SDK: Code snippet for first query is wrong

SELECT * FROM eth.recent_blocks ORDER BY block_number LIMIT 10 is the provided query but the column name is number not block_number

Release notes for February

Add notes and instructions for Regenerating API key

Update docs for API section.

Issues

https://docs.spice.xyz/api/sql-query

• "Ethereum (EVM) Queries" link doesn’t work
• "Ethereum Beacon queries" points to bitcoin

https://docs.spice.xyz/api/sql-query/http-api

• Tables in "See Tables for a list" is a broken link.
• Link for “Use the Apache Arrow Flight API to” is broken
• Response payload comment could be better than

{
  // Response
}

• v1/sql/[org_name]/[app_name]/sql is wrong
• Should have better example, include firesql
• Javascript example should use spice.js

Async HTTP API - Spice.ai Documentation

• “See Tables for a list of” is broken link
• “ Get in touch on Discord” Should this be to slack now
• “Requests are limited to 90 seconds.” Isn’t it 5 minutes now

Apache Arrow Flight API - Spice.ai Documentation

• Firecache documentation is link to YAML reference, not great link
• ”Use basic authentication”: username needs to be appId

https://docs.spice.xyz/api/ethereum/beacon-http-api

• Change Beacon node API specification link to: https://ethereum.github.io/beacon-APIs/#/Beacon

https://docs.spice.xyz/api/ethereum/blocks

• Parameters/Query: change key to api_key

https://docs.spice.xyz/api/ethereum/gas-fees

• Parameters/Query: change key to api_key

https://docs.spice.xyz/api/goerli/beacon-http-api

• Change Beacon node API specification link to: https://ethereum.github.io/beacon-APIs/#/Beacon

https://docs.spice.xyz/api/goerli/blocks

• Parameters/Query: change key to api_key

https://docs.spice.xyz/api/goerli/gas-fees

• Parameters/Query: change key to api_key

Update `Reference` section of docs

https://docs.spice.xyz/reference/sql-reference/all-functions

• Do we want to keep all of this, or just reference elsewhere

https://docs.spice.xyz/reference/sql-reference/sql-commands

• No description for SHOW

https://docs.spice.xyz/reference/sql-reference/sql-commands/show

• Should make sure table names referenced are ones we have

https://docs.spice.xyz/reference/sql-query-tables/ethereum

• Uniswap V2 Tables link points to Uniswap V3

https://docs.spice.xyz/reference/sql-query-tables/ethereum/token-tables

• Links under link should reference [ERC-20](https://ethereum.org/en/developers/docs/standards/tokens/erc-20/) Fungible tokens not just ERC-20
• Inconsistency between having a separate table page for recent_

https://docs.spice.xyz/reference/sql-query-tables/goerli/token-tables-1
- Links point to Ethereum, not goerli

https://docs.spice.xyz/reference/samples-and-examples/example-ethereum-evm-sql-queries

• Examples are all empty

https://docs.spice.xyz/reference/samples-and-examples/example-bitcoin-queries

• Just broken

https://docs.spice.xyz/reference/samples-and-examples/example-bitcoin-queries/blocks

• Just broken

https://docs.spice.xyz/reference/samples-and-examples/arrow-flight-samples

• token_pair = client.authenticate_basic_token('', apiKey) is wrong should be
  • token_pair = client.authenticate_basic_token(‘appId’, apiKey)

Document the `sort` fields for each table

What

Document which filters can be used by queries to improve the speed of queries. If a query doesn't need to process the entire blockchain's history, then adding a filter to a specific time range cuts down on the amount of data the query needs to process and returns the result set much faster.

Why

This improves the customer experience of using our platform by allowing them to run queries even faster.

See https://github.com/spicehq/data-platform/issues/695 for more context on why this is important.

Merge query docs with mainsite docs

Add notes and instructions on in-app Monitoring

including for usage monitoring and API request logs

GO SDK: Code snippet for instantiating gospice is incorrect

Currently it is:

spice := NewSpiceClient()
defer spice.Close()

but should be:

spice := gospice.NewSpiceClient()
defer spice.Close()

Release notes for April

Docs: High quality combined documentation

Add documentation for `v1/prices`

Docs: High quality SQL reference documentation

Promote our compatibility with ANSI SQL

Docs for token specific tables

Release notes for March

Node.JS SDK `getPrices` and `getPrice` not documented

Write model train + inference tutorial

Add notes and instructions on deleting app

Release notes for May

Docs: Current Limitations

Standalone page in docs (under reference)

Tokens getting started tutorial

ETH Core: Indexed columns are out of data, missing most tables

https://docs.spice.xyz/reference/sql-query-tables/sql-query-tables/token-tables#improving-query-performance-indexed-columns

GO SDK: Docs should say supports v1.19+ not just v1.19

DEX data docs

Improve M1 instructions

M1 instructions at https://docs.spice.xyz/api/sql-query-api/apache-arrow-flight-api#apple-silicon-arm64-support are still not clear enough.

1. Provide no context on why this is required. Should probably be under "getting started (M1)" rather than troubleshooting which people would skip. Aim to be preventive here.
2. Doesn't mention "M1" at all.
3. Misses the Conda environment creation and activation steps

E.g.

Create Best Practices section in docs

• Use Arrow API
• Limits
• Offsets and pagination

Unclear on how to sue the JSON-RPC for Polygon with web3.py

Portal mentions that you can use polygon via the JSON-RPC endpoint using libraries such as web3.py however looking at web3.py it doesn't mention Polygon and so isn't obvious how to get the Polygon data out.
Using the .eth.XXX methods produces error so clearly isn't the way (or if it is then that's a bug)

ETH Core: Indexed columns is out of date, missing most tables

https://docs.spice.xyz/reference/sql-query-tables/sql-query-tables/nft-tables

ETH Core: Wallet balances still in docs even though it is turned down

Fix docs.spiceai.io CNAME/link

GO SDK: Docs should have step for importing SDK

Docs should contain step for adding

import (
	"github.com/spiceai/gospice"
)

Add notes and instructions for connecting app to Github Repo

Step by step instructions for user to connect app to Github Repo
Location: should be listed as sub page under Portal-> apps

e.g.
create a repo in the org thats mapped to your spice account
with the same name as one of your spice apps
then in spice app settings there's an option to install the spice app to github
then you can connect that repo to the app

Adding a pair to the Historical Prices snippet breaks the URL

https://data.spiceai.io/v0.1/prices/eth-btc?preview=true?api_key= ->
https://data.spiceai.io/v0.1/prices/eth-btc?preview=true&api_key=

Create docs for `/eth/v0.1/sql` API