lightning / bolts Goto Github PK
View Code? Open in Web Editor NEWBOLT: Basis of Lightning Technology (Lightning Network Specifications)
BOLT: Basis of Lightning Technology (Lightning Network Specifications)
according to BIP141,
If the version byte is 0, and the witness program is 32 bytes:
- It is interpreted as a pay-to-witness-script-hash (P2WSH) program.
- The witness must consist of an input stack to feed to the script, followed by a serialized script (witnessScript).
- The witnessScript (≤ 10,000 bytes) is popped off the initial witness stack. SHA256 of the witnessScript must match the 32-byte witness program.
- The witnessScript is deserialized, and executed after normal script evaluation with the remaining witness stack (≤ 520 bytes for each stack item).
- The script must not fail, and result in exactly a single TRUE on the stack.
the last item of the witness data should be witnessScript, i.e. <2 <key1> <key2> 2 OP_CHECKMULTISIG> in serialized form.
The requirements for the data part of the payment invoice reads:
"A writer MUST set signature to a valid 512-bit secp256k1 signature of the double SHA2 256-bit hash of the Human Readable Part concatenated with the Data Part and zero bits appended to the next byte boundary, with a trailing byte containing the recovery ID (0, 1, 2 or 3)."
However, trying to decode the examples created by @rustyrussell, I can only make it verify by taking a single hash of the data. What is correct?
(As discussed in PR #119 )
This nicely reduces the requirements if nodes vanish before getting far into channel setup. The only issue is that if one side has sent funding_signed and the other hasn't received it: in this case the re-transmission of funding_signed will cause an error.
Since maximum number of htlc in fly is limited (by tx-size) number 432 it is possible and costless to congest payment channel:
lets consider topology A-B-C-A.
Node A can easily DDOS B-C channel by sending 432 (or max_htlc_in_fly
) htlc-payments and not automatically resolving it (or resolving it with considerable delay).
As I understand there are no ways to stop this attack with current specification, since each node can't defend itself: blocking by ip, or blocking specific node is ineffective due to onion routing (attacking node may not interact directly with B or C). Thus this issue is different from #122.
Since this attack is critical for Lightning Network proper work and since I can't imagine mitigating of this attack without adding new messages or significant changes in existing I think this issue cant wait till v1.1 .
I figured out one possible way to avoid this issue: non-refundable prepaid fee for htlc.
I think the current construction of channel ids might be hard to extend beyond the bitcoin blockchain.
Maybe the current information could be hashed and a prefix of this hash used as transaction id, so that later an additional blockchain identifier could be added.
We should make all fields consistent: I initially used - instead of _ because _ causes emphasis in markdown. But since we're supposed to surround all references with `` we should be OK.
Currently we have a nasty mix; I prefer _ because I can use those names exactly in the source code.
I've not made a PR, as I don't want to interfere with Real Work.
When nodes reconnect/restart, they need some way to see what was received by the other end to re-sync state. For normal (or shutting down state) this naturally follows batching done by commit_signature/revoke_and_ack messages.
The c-lightning prototype used a scheme where init messages contained a single counter: the total number of commit-signature and revoke_and_ack messages it had received. On disconnect, it would also forget any updates which it had not received commit_signature for.
In Milan, @adiabat argued simply retransmitting and discarding duplicates, rather than an explicit ack number. More recently, @pm47 asked to avoid the compulsory discard, and require an exact retransmission of previous messages; @rustyrussell instead asked for a strict superset. But further consideration has raised issues with these approaches.
update1
commitment_signed
disconnectupdate1
. Has one update pending.update1
and commitment_signed
. Has sent revoke_and_ack
.Now, when A reconnects, it does an exact retransmission:
update1
commitment_signed
next_per_commitment_point
it sent in revoke_and_ack
.There is also the case where A adds another change (eg. feechange, or another update).
Possible solutions:
It's important that an optimal implementation only be required to remember state at the minimal number of points, as a robust implementation will need to synchronously write to disk(s). A node must remember when it receives revoke_and_ack
(to create penalty transactions later), and when it sends commitment_signed
(as it is committed to the HTLCs at that point, so it must remember them), so these are the minimal "sync" points possible.
Thus, requiring a node to persistently remember updates it has sent but not yet committed to is a poor idea. However, this can be reconstructed: we have to remember incoming HTLCs or fulfill/fails which were going to the reconnecting peer anyway, we can just re-send them. However we would not normally remember fee changes we have not committed to: requiring this to be recorded on sending update_fee adds a disk sync. Nor would we normally remember the order in which we sent the updates, which is imperative for the update_add_htlc id
fields to match.
ECLAIR seems to require remembering the state and not rolling back. c-lightning (old, pre-Milan daemon) used reconstruction on reconnect/restart, but assumed the other side would roll back and used a total counter, and thus didn't have an issue if order or fees changed. lnd goes even further, and doesn't even remember id
across reconnections: HTLCs are implicitly renumbered from 0 at that point. I don't find this 8-byte ondisk saving convincing: once HTLCs are no longer in the commitment transaction the ID can indeed be forgotten, but so can the amount and routing information: only the cltv and RIPEMD of the payment hash need to be remembered for creating the penalty transaction.
DDoSing lightning network node is very critical attack and mitigation method is needed. For example, node discovery should not rely on ip address, and nodes should be able to relay packets behaving as proxy for other nodes.
Clearly defined invoice format is crucial for compatibility of different LN software. Thus it should be added to LN specification.
As I understand lnd, eclair and lightning-c already propose invoice formats, so I encourage @pm47, @cdecker and @rustyrussell to colloborate and create unified standart.
[1:realm][8:amt_to_forward][2:cltv_value][9:unused_with_v0_version_on_header]
There was discussion on why we want to duplicate CLTV here (after all, it's in the HTLC itself); ISTR it was to avoid probe attacks using differing HTLC values?
Consider this setup:
A 2-----0 B 2-----0 C 0-----2 D
2 0
| |
| |
0 2
E 2-----0 F
If A wants to pay D 1 BTC, a simple Dijsktra will return the route A-B-C-D, but only the route A-B-C-E-F-D would work. I believe this kind of scenario could be pretty common until we use dual-funded channels, and assuming that typical_htlc_amount << typical_channel_capacity might not be enough.
We don't currently advertise the amount a node is willing to forward to an outgoing channel. This could have been done by adding a field in the channel_update
message, and would have allowed us to easily filter out channels a priori when calculating a route, but IIRC we discarded this possibility for privacy reasons.
The other way I think could be to filter out channels a posteriori by adding a specific error code in BOLT 4, but there isn't any. Is that on purpose?
ElementsProject/lightning#210 has a discussion about the issues with partial payments.
The requirement of all payments needing an exact match invoice creates problems in many use cases and a solution would enable a lot more flexibility.
Not sure if my idea of using the recipient rhash as a generator for the actual payment rhashes that are derived via hash(sharedsecret + timestamp + origrhash + smalleramount) will be sufficient, but hopefully close enough that someone who fully understands this can fix
BOLT 1 describes the error message and the optional data field
A receiving node SHOULD only print out data verbatim if it is a valid string.
How does one determine if the data is a valid string?
Besides a string, it's not obvious what this optional data field may contain or how it should be parsed.
I appreciate that this is probably going to be spec'd out in future revisions.
Thanks,
Donal
For both onion and transport as per discussion on #43
Suppose there is a following route:
A
(public ->) B
(non-public ->) C
(non-public ->) D
In which order should node D
include B -> C
and C -> D
in it's payment requests? Is it B -> C
, C -> D
or C -> D
, B -> C
?
There is seemingly a problem with WatchTower incentives for storing a revoked commit txs as there could potentially be billions of records to keep which would require a reasonable infrastructure while WatchTower's reward for catching a channel breach is currently nothing.
Plus, channel breaches are not expected to happen too often so even if every revoked tx contained a small WatchTower reward still it does not seem like any substantial amount could be made.
Here I propose a concept of special disposable "storage token" which is supposed to be provided by a WatchTower and later used by LN wallet client to store one revoked tx.
This works as follows:
(clear token, clear sig)
tuples locally.(clear token, clear sig)
attached.I've used this paper for blind signature scheme: https://arxiv.org/pdf/1304.2094.pdf
An implementation which uses secp256k1 curve: https://github.com/btcontract/olympus/blob/master/src/main/scala/com/lightning/olympus/crypto/ECBlind.scala
An implementation of client-side state machine which automatically handles all the steps outlined above: https://github.com/btcontract/lnwallet/blob/master/app/src/main/java/com/lightning/wallet/lncloud/LNCloud.scala
What this gives us:
I'd like to hear your thoughts on this.
In order to identify the next hop, we currently use a node-address
defined as hash160(node-id)
. Using a channel-id
would work the same way, and use 8 bytes instead of 20 bytes.
Not only is channel-id
smaller than node-id
, but it also carries more information because it identifies the next node and the exact channel to use. If we allow several channels between the same two nodes (I don't remember what we decided in Milan), we need to know what channel to use when forwarding the htlc because the fees/expiry-delta might not be the same. So even if we need to use a hash for some other security reason, using hash160(channel-id) instead of hash160(node-id) would still be useful.
Every failure should come down to one of three things:
(1) it violates requirements set out in channel_update (eg. amount too low, fee too low), OR
(2) it's using an old channel_update, OR
(3) this channel is transiently limited (in which case we need a signed channel update which says that: TODO)
Eventually, we'll want a robust chain-of-custody for these messages, to ensure they don't get corrupted (and if they do, we can prove it so that node is blacklisted).
But ignoring that, we don't want nodes to be able to publish a channel_update and not honor it.
I'm currently integrating the onion reply into our client and I stumbled across an inconsistency between the spec and the test vectors. The spec says we ought to use 32 byte untruncated HMACs in the reply, while the test vectors are using 20 byte truncated HMACs.
How do we resolve this inconsistency? Shall we use the truncated ones or the full HMACs? I'd prefer using the non-truncated ones since that matches all the other HMACs in the spec, and we removed the truncated ones everywhere else.
type: 36 (MSG_FUNDING_LOCKED)
data:
[8:temporary-channel-id]
[8:channel-id]
[32:next-key-offset]
[33:next-revocation-halfkey]
I have two questions about the channel-id field:
min_conf
value of 0 or 1 could lead to each party computing a different channel-id
. Easiest way is probably to just enforce a minimum value for min_conf? Yes a MIN_MIN_CONF
!MIN_MIN_CONF
high enough so that the probability of computing a different channel-id
is extremely low, why do we even need to specify the channel-id
in this message? If we prefer to keep it here, then we need to specify what happens when they are not the same (probably just close the connection). I'd rather remove it, since we still need to handle the general case where we receive messages with unknown channel ids (e.g. in htlc messages).Cheers,
Pierre
Current local channel state may be completely lost at any moment, this is most relevant to mobile devices (destruction, loss, etc) but can happen on any setup. It would be nice to have a way to claim at least some of the remaining channel funds in such a situation.
Having current perCommitmentPoint
included in ChannelReestablish
would allow for the following scheme:
ChannelReestablish
, then send and Error
and wait for a current commit to be published, then use perCommitmentPoint
and saved static channel parameters to claim a P2WPKH output.HTLC outputs can't be claimed but this may be okay in a majority of cases as mobile device is much more likely to get lost when it's in a steady state (has no in-flight HTLCs) anyway.
For future upgradability and backwards compatibility would it not be wise to include a version number in the message format? I know there is a type field, but it makes sense to have them both otherwise you will run out of types fields quickly (in the future).
What if two peers who want to open a channel with each other sets minimum-depth
to 1 and there are two blocks created simultaneously (same height) with the funding transaction in both of them, and each peer receives a different block. They will send the funding_locked
message with (most likely) different channel-id
, and they must now fail the channel according to the spec. Should the protocol not handle this case?
Instead, use key recovery based on the signatures (like in #183) to save space.
We would save 4*33 - 1 byte (4 * two "recovery bits") = 131 bytes.
While in Requirements we have sentence:
The sender SHOULD set
minimum-depth
to an amount where the sender considers reorganizations to be low risk.
message description actually doesn't contain minimum-depth
field.
I propose to add minimum-depth
field to the end of message, thus it should not broke current implementation since basic policy for reading message: "read exactly number of bytes you need and drop residue". However it will be kind of ugly.
local_funding_key: 30ff4956bbdd3222d44cc5e8a1261dab1e07957bdac5ae88fe3261ef321f374901
remote_funding_key: 1552dfba4f6cf29a62a0af13c8d6981d36d0ef8d61ba10fb0fe90da7634d7e1301
local_revocation_key: 131526c63723ff1d36c28e61a8bdc86660d7893879bbda4cfeaad2022db7c10901
local_payment_key: e937268a37a774aa948ebddff3187fedc7035e3f0a029d8d85f31bda33b02d5501
remote_payment_key: ce65059278a571ee4f4c9b4d5d7fa07449bbe09d9c716879343d9e975df1de3301
None of these are valid keys; they should start with 03 or 02, and why do they all end in 01?
The names also don't match the names in the rest of the document, but I was hoping to figure that out from the test vectors and submit a patch...
Since both nodes may choose a different value for dust-limit-satoshis
(say dust-limit-a
and dust-limit-b
), I think we need to specify which one is to be used for the closing transaction in a mutual close scenario.
This is definitely a corner case, but it still can happen with channel-reserve-satoshis
>> dust-limit-satoshis
if we close a channel shortly after creation.
I think using dust-limit-a
for the output going to A and dust-limit-b
for the output going to B would make sense, but it might be easier to just use min( dust-limit-a
, dust-limit-b
) or max( dust-limit-a
, dust-limit-b
).
What do you guys think?
Cheers,
Pierre
Currently the recipient can be guessed by the second to last hop due to the short timelock on the HTLC, i.e., it is not plausible that this is just an intermediate hop since the timelock cannot further be reduced. We discussed adding a random offset to the CLTV timelocks in order to avoid this sort of detection. This can further be strengthened by computing an appendix/shadow route to the actual route of random length, and offsetting all CLTVs by the sum of CLTV values. It is still unclear whether the sender or the recipient should be the one computing the shadow route.
I assumed that type was unique to each message, but now I see that both ping and final_incorrect_cltv_expiry have the same type.
We currently have no good way of forgetting really old channels/nodes. By having nodes forget channels that they've not heard about in the last n
hours (or similar) we can prune the channel graph easily, by putting a little bit more load on the network. I'd suggest having nodes issue a channel_update
n-1
hours after the last update, which should provide ample time for the update to propagate. If nothing changed the update should just resend the previous update, with the current timestamp.
The current LN protocol appears to assume every address is a single party address. However, I believe with a small change, multisig addresses could be supported.
That would allow LN payments sent to a group controlled address and disbursement requiring signatures from MofN parties.
Not sure what changes other than having a multisig redeemscript is needed at the protocol level. At the implementation level, coordinating the MofN nodes would need to be solved, which might require opening up a signature round in the protocol.
I apologize that I am not better versed in the protocol to make more accurate suggestions.
Opening a non-public channel without initial routing sync seems to be an optimal option for a bandwidth and computation constrained Lightning node primairly focused on payments rather than payment routing (a lightweight mobile wallet for example). Let's suppose such a minimal node A
is connected to a public node B
via a non-public channel, this implies that node A
needs to add a B -> A
routing tag to it's payment requests which means it needs to have an access to B
's latest fee
and cltv_expiry_delta
.
Currently localfeatures
allows for all or nothing initial routing sync options but what if a feature bit could be added which would instruct a node B
to only send an info sufficient for A
to add a routing tag to it's payment requests and nothing else?
I can't find any document relating to how route discovery will happen? Onion routing can be used once a source has the information necessary to construct a packet with the desired route, but how does a source discover the routes it needs to choose the optimal route?
Suppose A opens a channel with B. funding-locked messages have been sent and received ad there are no pending acks. Now suppose A restarts, and tries to reconnect to B and restore the channel. How does B know which channel the new incoming connection is for ?
If we assume there is a single connection between peers and channel messages are multiplexed over it, then it's easy but what if we have one connection per channel ?
A mandatory retransmit message that includes the channel id might be helpful here.
ps. I remember that multiplexing channel messages over single connections was discussed but I don't remember that a choice had been made, so the 'one connection per channel' model should also be valid ?
ps2: I had a discussion over this with Christian on #lightning-dev
The requirements of the open_channel message describe how the receiver may/should fail the channel in certain cases. For example, if the funding-satoshis field is too low.
Should there be another optional message reject_channel (Similar to TCP's NAK). This message would contain a failure reason(s) so the sender side can readjust the parameters (If it is possible to).
I am trying to picture this from a UI/UX perspective. A user wishes to open a channel with a party and presumably enters a funding amount but the party's LN client rejects it. It's difficult to imagine how this information would be propagated back to the user without some sort of error reason.
Thanks,
Donal
CC-BY?
GNU FDL?
WTFPL?
We should be able to request a refund address in BOLT 11, and put a payment request into the onion (requires onion expansion).
See: https://lists.linuxfoundation.org/pipermail/lightning-dev/2017-August/000751.html
Add some text field to allow sending some additional information with payment.
Currently, this suggestion is not very useful because some communication between sender and receiver is
required to share rhash. May became useful for onion encoded payments when there are no
prior communication between sender and receiver.
We would like to propose a small modification of the commiment transaction HTLC scripts, which would make it possible, when the other party publishes a revoked commit transaction, to create penalty transactions that spend the HTLC outputs. With the current design the penalty transaction spends the second-stage HLTC-timeout and HTLC-success transactions.
This does not change the way external channel monitors would work, but for node which do their own monitoring this has several advantages:
The resulting transaction tree would be:
+------------+
| funding tx |
+------------+
|
| +-------------+
\--------| commit tx B |
+-------------+
| | | |
| | | | A's main output
| | | \------------------ to A
| | |
| | |
| | | ,-- to B (& delay)
| | | B's main output /
| | \----------------<
| | \
| | `-- to A (& revocation key)
| |
| | ,-- to B (& delay)
| | +-----------------+ /
| | ,--| HTLC-timeout tx |---<
| | HTLC offered by B / +-----------------+ \
| \-------------------< (after timeout) `-- to A (& revocation key)
| \
| `-- to A (& payment preimage)
| \
| `- to A (& revocation key)
|
| ,-- to B (& delay)
| +-----------------+ /
| ,--| HTLC-success tx |---<
| HTLC received by B / +-----------------+ \
\----------------------< (w/ payment preimage) `-- to A (& revocation key)
\
`-- to A (after timeout)
\
`- to A (& revocation key)
The solution we propose is to use a multisig 2-of-3 with localkey
, remotekey
and revocationkey
for the Offered HTLC Output script:
<remotekey> OP_SWAP
OP_SIZE 32 OP_EQUAL
OP_NOTIF
# To me via HTLC-timeout transaction (timelocked) or to you with revocation key.
OP_DROP 2 OP_SWAP <localkey> <revocationkey> 3 OP_CHECKMULTISIG
OP_ELSE
# To you with preimage.
OP_HASH160 <ripemd-of-payment-hash> OP_EQUALVERIFY
OP_CHECKSIG
OP_ENDIF
This output can be spent, as before, with the remote key and payment preimage, or with the remote key and local key,
but it can be spent also be spent with the remote key and revocation key, with the following witness script:
0 <remote-sig> <revocation-sig> 0
We use additonal IF branch (there is probably a better way to do this?):
<remotekey> OP_SWAP
OP_SIZE 32 OP_EQUAL
OP_IF
# To me via HTLC-success transaction.
OP_HASH160 <ripemd-of-payment-hash> OP_EQUALVERIFY
2 OP_SWAP <localkey> 2 OP_CHECKMULTISIG
OP_ELSE
OP_SIZE 0 OP_EQUAL
OP_IF
# To you after timeout.
OP_DROP <locktime> OP_CHECKLOCKTIMEVERIFY OP_DROP
OP_CHECKSIG
OP_ELSE
# To you if you have the revocation key
OP_SWAP 2 OP_SWAP <revocationkey> 2 OP_CHECKMULTISIG
OP_ENDIF
OP_ENDIF
This output can be spent, as before, with the payment preimage and the remote and local keys, with the remote key after a delay,
but it can also be spent with the remote key and revocation key, with the following witness script:
0 <remote-sig> <revocation-sig>
The RFC perhaps in the introduction section should contain a high level design overview and statement of the intended properties of the system.
It's pretty difficult to analyze if they specification achieves it's goal without this.
Is BOLT 4 channel_id
(type: per_hop
) same as BOLT 7 short_channel_id
?
We added the extraction tool as a travis-ci config, but we still need a repo admin to flip the switch:
https://travis-ci.org/profile/lightningnetwork
That should get us feedback on the extractor tool for all pull requests and allow formatting checks in the future.
Currently it is left unspecified how many routing fields are allowed. I remember Tadge and Joseph disagreeing about what the typical channel length would be, but they both claimed a number under 10. So that might be a mandatory minimum requirement.
OTOH, as I read it, it is possible to make an invoice with 10000 nodes in it. Should that really be allowed?
<RIPEMD160(revocationkey)>
--> <HASH160(revocationkey)>
or <RIPEMD160(SHA256(revocationkey))>
?
Currently the test vectors are in-line with the specification description in a format optimized for human readability (the comments, etc). I propose we instead move to a format optimized for machine readability (JSON, or the like strikes a nice balance). This would allows maintainers of implementations to simply sync a file/directory with all the test vectors into their project, giving them a seamless way to keep up with the latest test vectors.
Coming from the Bitcoin side, Bitcoin Core has a set of reference tests encoded in JSON files. As a btcd
developer this format is delightful as I can simply copy-pasta the latest set of JSON files if they're modified, re-run our tests (which parse the JSON test cases) to ensure we're still in-sync and complaints (at least from the PoV of the tests).
HKDF(salt,ikm): a function is defined in 5, evaluated with a zero-length info field.
while there are only 3 listed.
References
https://tools.ietf.org/html/rfc7539
http://noiseprotocol.org/noise.html
https://tools.ietf.org/html/rfc5869
Andreas Schildbach mentioned this on the mailing list, there were no replies.
It sounds like a good idea to me.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014472.html
Feel free to close, just please tell me why
I think we should set a max allowed length for the description field, as it is now not specified. As of now we must support strings of any size, which is probably not necessary.
I suggest 1024 characters/bytes, I think that should be enough.
I think there might be a chicken-and-egg issue in the way we exchange announcement signatures with funding_locked
messages.
Each party can compute its own announcement-bitcoin-signature
, but it needs the other party's announcement-bitcoin-signature
in order to compute the announcement-node-signature
the way it is currently specified. So we can't actually build valid funding_locked
messages.
Maybe we can work around this issue by not including the other node's announcement-bitcoin-signature
in our announcement-node-signature
(channel id needs to be included)? It would still prove that each node owns their respective bitcoin address, which can be linked to an actual tx, and that both nodes agreed on the announcement.
+-------+ +-------+
| |--(1)---- add_htlc ------>| |
| |--(2)---- add_htlc ------>| |
| |<-(3)---- add_htlc -------| |
| | | |
| |--(4)---- commit ------>| |
| A | | B |
| |<-(5)--- revoke_and_ack-----| |
| |<-(6)---- commit -------| |
| | | |
| |--(7)--- revoke_and_ack---->| |
+-------+ +-------+
If the receiver A had given senderB (4)commit, and B drop the connection on purpse. Then sender B can have an commitment that can't be revoked. Even sender have the new commitment causing it to lose token, sender B can have a snapshot of current state. If they exchange commitment after that, and B has send more token to receiver A, B can then sign and publish the commit mentioned above to cheat.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.