opentimestamps/python-opentimestamps

python-opentimestamps

Python3 library for creating and verifying OpenTimestamps proofs.

Installation

From the PyPi repository:

pip3 install opentimestamps

Structure

Similar to the author's python-bitcoinlib, the codebase is split between the consensus-critical opentimestamps.core.* modules, and the non-consensus-critical opentimestamps.* modules. The distinction between the two is whether or not changes to that code are likely to lead to permanent incompatibilities between versions that could lead to timestamp validation returning inconsistent results between versions.

Unit tests

python3 -m unittest discover -v

Additionally Travis is supported.

SSL Root Certificates

On some MacOS setups SSL certificates may be missing. The following commands could be of use to resolve this error (the below example assumes a user is running Python "3.7", and is using Certifi package):

cd /Applications/Python\ 3.7
Install\ Certificates.command

python-opentimestamps's Issues

Complete release of version 0.2.1

Version 0.2.1 contains an import segwit fix and is already released on pypi.
Completing the release by updating __version__ and adding a Git tag would be a great help for packaging the library. (In my case: I'd like to add it to nixpkgs, along with the client and server.)

You service is sh*t for Bitcoin community :-E

Hi,

You service is sh*t for Bitcoin community :-E
You use UTXO for proof and a database of UTXO permanently grows from your proof transactions!
You could use OP_RETURN as it do https://proofofexistence.com/

It's right way!

Add Support For BLAKE3

BLAKE3 is very new, shiny, and fast!

Python is planning to add it to hashlib: https://bugs.python.org/issue39298

I'm trying to quickly timestamp a few TB hard drives worth of files and the speed of BLAKE3 would be very nice to have.

Create Code of Conduct

Maybe a Code of Conduct might be nice. What do you all think?

OpPad?

Is concatenation of message chunks - separating by junk padding - a valid/useful commitment operation? Specifically, imagine our commitment operation takes a message, and a list of padding chunks as the argument:

<delta idx 1> <padding chunk 1> ... <delta idx n> <padding chunk n>

and produces the result by taking the message, and inserting the padding chunks appropriately.

This should still be a commitment operation, as for a given set of padding and a given message, the result will always be the same; it's essentially a very fancy OpAppend.

OpReverse allows the creation of cycles in a timestmap graph

Suppose we create a timestamp on 'abcd' with the following operations:

OpReverse -> 'dcba' -> OpReverse -> 'abcd'

...and we're back where we started.

Now granted, this will work just fine in the opentimestamps client, just it's an unexpected property that adds complexity in some cases. For instance, the current calendar implementation in the opentimestamps server would probably fail if you ever added a timestamp containing two reverses in a row to it.

Thoughts? Might be enough to just put a warning on it and provide mechanisms to prevent OpReverse being used in places where it can cause issues, like the server.

CC: @RCasatta

reading from stdin is broken in linux

Well, this doesn't work. (Ubuntu linux, bash, python 3.5)

$ echo $LANG
en_US.UTF-8

$ echo "test" | ots verify test.ots -f -
Traceback (most recent call last):
  File "/home/xxx/.local/bin/ots", line 11, in <module>
    sys.exit(main())
  File "/home/xxx/.local/lib/python3.5/site-packages/otsclient/ots.py", line 34, in main
    args.cmd_func(args)
  File "/home/xxx/.local/lib/python3.5/site-packages/otsclient/cmds.py", line 482, in verify_command
    actual_file_digest = detached_timestamp.file_hash_op.hash_fd(args.target_fd)
  File "/home/xxx/.local/lib/python3.5/site-packages/opentimestamps/core/op.py", line 301, in hash_fd
    hasher.update(chunk)
TypeError: Unicode-objects must be encoded before hashing

A naive fix would be to change that line in op.py where hash_fd is to: hasher.update(chunk.encode('utf8')), but then it's broken when reading from a file. I have the feeling there could be a common solution to that, but I don't know it.

Scrypt Op

Even if no current OTS implementations support Litecoin (LTC), Scrypt should at least be in the .ots vocabulary.

In order for a proof to be verified with just a trusted block header, the .ots files need to contain a Merkle brach of the blockchain used. Since LTC uses Scrypt as its hash function, the .ots needs to convey the Scrypt hash function if LTC is used.

Development on LTC support by a third party can't start until there is a consensus on how to serialize/deserialize Scrypt.

make a readme like JAVA and Javascript "From Code"

upgrade readme explanation for use OpenTimestamps from code like other languages.

Add comment explaining why there isn't a testnet attestation

See #19

Timestamp Serialization Format

Based on the ots command line program, the Stamp command returns a timestamp formatted like this:

{
    'txid':b2lx(txid),
    'nonce':b2x(nonce),
    'addr':str(addr)
}

The Complete command takes that timestamp and if the block is complete, returns a timestamp that looks like this

[
    [merkle_path...],
    [timestamp header]
]

The disparity between the two formats should be reconciled. If we want to create a function to read a timestamp, it doesn't seem good to first check if it is an array or an object, etc. Maybe the timestamp returned by the Stamp command can match the array-style and just put the txid, addr, in the header area.

operation to commit to secp256k1 points?

Given a commit c, the operation c → P + H(P||c)G is a valid commitment operation, where P is an arbitrary given curvepoint and G is a standard generator for the curve. It does not depend on any properties of the curve except that G have large order (and this is a mathematical fact about crypto groups, not a security assumption), only on the security of the hash function H.

There are unfortunately a lot of variants here -- H can be any second-preimage-resistant hash function, the curve can be any elliptic curve or even a non-EC group, etc. But the parameters secp256k1, SHA256, would allow opentimestamps to support pay-to-contract and sign-to-contract on Bitcoin (and Ethereum, and grin).

This would require any OT library to support elliptic curve math, but they don't have to do a good job of it, only a correct job. Sidechannel resistance for example is unnecessary.

Properly sanity check aggregator/calendar responses

Currently the only sanity check that RemoteCalendar does is to reject responses larger than 10KB.

For submission, we should verify/filter the following:

• PendingAttestation matches expected whitelist (URL is unauthenticated!)
• Number of PendingAttestation's is reasonable
• Hash operations are sane (exclusively SHA256?)
• Proof size is appropriate for merkle tree aggregation
• All attestations are known (maybe not a good idea for upgradability? e.g. a future trusted signature scheme could return valid attestations directly in response to a submission)

And when getting timestamps:

• Response does not include any PendingAttestation's (allowing loops greatly complicates the logic)
• Redundant attestations (e.g. two different bitcoin blocks)

`sdist` does not include `README.md` which is required by `setup.py`

Question about data length encoding

I’m trying to understand the serialisation format of an ots file. As I understand it commitment operations are encoded like

1 Byte Tag | variable bytes data length | variable bytes operation payload

For example

F0 10 0e 9b 49 ab 01 34 50 80 35 ce 26 97 f1 ad 09 85

can be read as append (F0) the next sixteen (10) bytes.

As I can see in your code data length (10 in this case) is encoded as a signed integer stored in big endian encoding. When I compare this to the Bitcoin wiki I see that there it is the exact opposite, unsigned integer stored in little endian.

https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer

Of course, both ways are valid, but as your implementation is very close to the Bitcoin protocol I want to understand the reason why you are deviating here.

The same is true for the block height of the Bitcoin Attestation. But in this case, there seems to be no “official” way to encode it.

Crash on signing large code rebase

Since it's been a couple of years since I hooked up OTS, this could very likely be an already fixed issue, in which case I'll close it ofc, but just thought I should drop this traceback in case it isn't a known issue:

Traceback (most recent call last):
  File "/home/adam/.local/bin/ots-git-gpg-wrapper", line 11, in <module>
    sys.exit(main())
  File "/home/adam/.local/lib/python3.5/site-packages/otsclient/git_gpg_wrapper.py", line 108, in main
    signed_commit_timestamp = Timestamp(hash_signed_commit(git_commit, gpg_sig))
  File "/home/adam/.local/lib/python3.5/site-packages/otsclient/git.py", line 27, in hash_signed_commit
    return OpSHA256()(OpSHA256()(git_commit) + OpSHA256()(gpg_sig))
  File "/home/adam/.local/lib/python3.5/site-packages/opentimestamps/core/op.py", line 133, in __call__
    raise MsgValueError("Message too long; %d > %d" % (len(msg), self.MAX_MSG_LENGTH))
opentimestamps.core.op.MsgValueError: Message too long; 4133 > 4096
error: gpg failed to sign the data
fatal: failed to write commit object

Could not apply a166035f6046aa0df194f12272f40090a6c9bc5f... fix wallet sync test bug

Failed to create timestamp: need at least 2 attestations but received 0 within `timeout`

I am getting this error and wondering if the service is still available or if this error is related to a problem locally on my end:

Submitting to remote calendar https://a.pool.opentimestamps.org Submitting to remote calendar https://b.pool.opentimestamps.org Submitting to remote calendar https://a.pool.eternitywall.com Submitting to remote calendar https://ots.btc.catallaxy.com Failed to create timestamp: need at least 2 attestations but received 0 within timeout`