lbryio/spec

LBRY protocol specification

LBRY Protocol Specification

LBRY is a protocol for accessing and publishing digital content in a global, decentralized marketplace.

This repo contains the official protocol specification for LBRY. It is built with Jekyll.

Usage

The spec is available at https://spec.lbry.com.

To view the content locally, you must have Ruby and bundler installed. Then run bundle install and bundle exec jekyll serve, and open localhost:4000 in your browser.

Contributing

To edit the spec:

• run bundle exec jekyll serve --watch
• make changes to the markdown
• refresh localhost:4000 in your browser to see the effect of your changes.

Contributions to this project are welcome, encouraged, and compensated. For more details, click here.

License

This project is MIT licensed. For the full license, see LICENSE.

Security

We take security seriously. Please contact [email protected] regarding any security issues. Our PGP key is here if you need it.

Contact

The primary contact for this project is @lyoshenka ([email protected]).

spec's Issues

Metadata signing spec is wrong

@tiger5226 commented on Wed Jun 12 2019

I implemented this outside of the SDK in Chainquery. There were a number of things wrong or missing.

The most important thing missing is the fact that the outpoint hash is used in the claim signing process and is needed to validate the signature too.

specifics include:

• How to get outpoint hash

  • reverse bytes of the txid of the first input of the transaction containing the claim.
  • convert the vout to a little endian uint32 4 byte array.
  • append them both into a byte array
  • hex encode to string and return it

• signature creation

  • in between the version and the certificate id the outpoint hash in bytes needs to be inserted.

• signature validation

• the outpoint hash needs to be inserted in the same place to validate the signature.

• signature format

  • 64 byte concatenation of the unsigned 32 byte R value and unsigned 32 byte S value correlating to the ECDSA x,y values.

• public key format

  • the public key format is encoded to ans1 ECDSA DER
    • encoding the special format of the public key is non-trivial and a reference implmentation is a good idea. There is one in the lbryschema.go repo.
  • the decoding requires specific ans1 object id values
    • ans1 encoding oid for ecdsa public key = 1, 2, 840, 10045, 2, 1
    • asn1 encoding oid for secp256k1 = 1, 3, 132, 0, 10
    • asn1 marshalling structure contains just the algorithm, rawContent and the publickey as an ans1 formatted bitstring.

spec should talk about the fact that updating a claim may add/remove/change a claim's channel

Add Justification Section

Presumably to the Introduction

Typo. "chuck" instead of "chunk". Line 765. 2021-08-25 Thursday 25. August

Typo. "chuck" instead of "chunk".

Line 765. Todays date: 2021-08-25 Thursday 25. August

add google analytics tag to spec

I'm not sure how to add the GA tag since this is markdown.

Here is the new ID for spec.lbry.io:

UA-60403362-10

Here is the full tag for reference

<script async src="https://www.googletagmanager.com/gtag/js?id=UA-60403362-10"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'UA-60403362-10'); </script>

proposal: metadata extensions

LBRY has a base schema for claim metadata. This schema is very general. Different file types and communities will want to include metadata that is specific to them. It would be good to have a mechanism to suppor this.

One idea for this is to allow publishing of metadata schemas to LBRY itself. This would be a new claim type (Schema). The schema would be a protobuf file (or similar, if protobufs cannot be compiled live), plus other information that hubs need (e.g. which fields to index). Then a claim can reference a published schema to tell hubs how to decode extra metadata contained in the claim.

Schema claim type

• schema structure
• how to specify indexes
• schema versioning?
• can schemas be changed? that complicates things
• limit who can create schemas at first, for testing?

Stream and Channel claim changes

• add a way to add arbitrary data
• add a way to specify the schema for the arbitrary data

SDK search changes

The SDK would switch from using flags for search to using a simple DSL. This way the SDK does not need to know all possible metadata fields in advance.

• what DSL to use for search?
• how to reference fields in search?

Concerns

• this is an avenue for protocol users to inject code into hubs and/or control their behavior. It's possible to DDoS hubs or impose unsafe or expensive computations on them (eg indexing fields for search), esp in combination with user-created claims.
• i doubt that protobufs can be compiled on the fly (at least in Go). will we need another schema description language?

Finish It

Single line horizontally scrolling code is hard to read

It is tricky to grab and scroll horizontally, plus the scroll bar obscures the code so I can't read it.

Example:

Add more details about blob exchange

This can be either in the spec itself, an appendix, or a resources article.

This is one of the few feedback items I've been unable to address.

generate pdf version of spec

Explore definition of effective amount

reposting from Slack:

Going back to the discussing around effective amount (lbryio/lbrycrd#203), do we want to consider only effective amounts for claims that are active, or all claims? I really don't see the use case for considering it for only active ones (assuming you are already taking into account any activation delay) especially when it relates to the new discovery tools we have in the app (i.e. sorting something by Top bids - I don't care if the claim is in takeover or not) . The spec does say it's supposed to be 0 when a claim is not active.

On a side note, the implementation also doesn't match the spec if I post a claim to lbry://one today for 1 LBC..that claim should be active right away because a takeover can't happen (If the stake is an update to an active claim, is the only accepted non-abandoned claim for a name, or does not cause a change in which claim is controlling the name, the activation delay is 0 (i.e. the stake becomes active immediately).) but we set the takeover height/effective amount to 0. If effective amount isn't the right term in this case, we should reconsider it.

Incorporate feedback from L. C.

@lyoshenka and I have this document in our email

be more explicit about how stream keys are stored and exchanged

we've received feedback that several people thought the key is passed directly from the publisher to the viewer. need to explain how it works.

clarify a few points about updates

update LBRY spec to clarify that an update to a claim does not reset the activation delay. right now an update is treated as a new claim and resets the activation delay counter.

update spec to clarify that an update cannot change a name (and post-fork, a normalized name). updates must keep the name consistent.

Many dead links off the spec.

https://lbry.tech/spec?_ga=2.13516727.1296393475.1587778860-519824460.1581147894 has many dead links.

• https://github.com/lbryio/lbrycrd/blob/master/src/chainparams.cpp
• https://github.com/lbryio/lbrycrd/blob/master/src/main.cpp#L1594
• https://github.com/lbryio/lbrycrd/blob/master/src/hash.cpp#L18

consider specifying how effective_amount works for claims in a channel

Today, the spec says that each claim has a single global effective_amount. However, a claim in a channel competes for it's name globally AND within the channel. In both cases, the effective amount is the same. This leads to surprising behavior when the in-channel activation delay does not match the global activation delay.

This happens only when a channel has multiple claims for the same name, a state that does not make much sense if we assume each channel has only one owner. However, this assumption may change in the future, and even if it doesn't, we should be clear about the behavior we want.