tendermint / rust-abci Goto Github PK
View Code? Open in Web Editor NEWA rust implementation of the ABCI protocol for tendermint core
License: Apache License 2.0
A rust implementation of the ABCI protocol for tendermint core
License: Apache License 2.0
I'm running on Tendermint 0.25 with this patched version: 90dad48
With too many/large transactions, rust-tsp will panic:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: WriteZero, error: StringError("failed to write whole buffer") }', libcore/result.rs:1009:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: std::sys_common::backtrace::print
at libstd/sys_common/backtrace.rs:71
at libstd/sys_common/backtrace.rs:59
2: std::panicking::default_hook::{{closure}}
at libstd/panicking.rs:211
3: std::panicking::default_hook
at libstd/panicking.rs:227
4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
at libstd/panicking.rs:477
5: std::panicking::continue_panic_fmt
at libstd/panicking.rs:391
6: std::panicking::try::do_call
at libstd/panicking.rs:326
7: core::ptr::drop_in_place
at libcore/panicking.rs:77
8: core::alloc::Layout::repeat
at libcore/macros.rs:26
9: <bytes::bytes::BytesMut as core::convert::AsRef<[u8]>>::as_ref
at libcore/result.rs:808
10: <protobuf::cached_size::CachedSize as core::default::Default>::default
at ~/.cargo/git/checkouts/rust-tsp-6c1140b7f202a3ea/90dad48/src/server.rs:150
11: abci::server::respond
at ~/.cargo/git/checkouts/rust-tsp-6c1140b7f202a3ea/90dad48/src/server.rs:118
12: abci::server::handle_stream
at ~/.cargo/git/checkouts/rust-tsp-6c1140b7f202a3ea/90dad48/src/server.rs:62
13: abci::server::serve::{{closure}}
at ~/.cargo/git/checkouts/rust-tsp-6c1140b7f202a3ea/90dad48/src/server.rs:34
Add a link or badge to https://crates.io/crates/abci in the readme
There are several (outdated or broken) published crates related to this repository:
e.g. https://crates.io/crates/rust-abci and https://crates.io/crates/tsp
To avoid confusion it would be cool to add a link to the actual crate: https://crates.io/crates/abc
Add a badge (e.g. like in the kms repo: https://github.com/tendermint/kms)
It may require switching to machine executor: https://discuss.circleci.com/t/cargo-tarpaulin-fails/30215
The other option is kcov
: https://abronan.com/building-a-rust-project-on-circleci/
Currently, there are only printlns -- https://github.com/tendermint/rust-abci/blob/master/src/server.rs#L75
It would be nice use e.g. env_logger
instead
As mentioned in: #21 (comment)
Currently, there's one mutex and synchronous IO, but not much reason for that. Mempool/info connections may not need to mutate the app state + The Tendermint ABCI specs say:
Thus, DeliverTx and CheckTx messages are sent asynchronously, while all other messages are sent synchronously.
It should have the merkle proof as in: https://github.com/davebryson/rust-abci/blob/da_protobufs/abci-protobuf/src/abci.rs#L5571
Currently, there's a new stable release 0.3: https://github.com/rust-lang-nursery/futures-rs/blob/master/CHANGELOG.md#030---2019-11-5
the current code uses 0.1
Currently, there's 0.4.0: https://crates.io/crates/abci
The latest master is 0.4.1 -- it'd be good if it was auto-published on crates (e.g. on release tags?)
Currently, we use circleci for our ci. I would like to move this to github actions as they have very nice support for rust.
error[E0053]: method `decode` has an incompatible type for trait
--> src/codec.rs:23:5
|
23 | fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<Request>, Box<dyn Error>> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `bytes::bytes::BytesMut`, found struct `bytes::BytesMut`
|
= note: expected type `fn(&mut codec::ABCICodec, &mut bytes::bytes::BytesMut) -> std::result::Result<std::option::Option<messages::abci::Request>, std::boxed::Box<(dyn std::error::Error + 'static)>>`
found type `fn(&mut codec::ABCICodec, &mut bytes::BytesMut) -> std::result::Result<std::option::Option<messages::abci::Request>, std::boxed::Box<(dyn std::error::Error + 'static)>>`
error[E0053]: method `encode` has an incompatible type for trait
--> src/codec.rs:42:5
|
42 | fn encode(&mut self, msg: Response, buf: &mut BytesMut) -> Result<(), Box<dyn Error>> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `bytes::bytes::BytesMut`, found struct `bytes::BytesMut`
|
= note: expected type `fn(&mut codec::ABCICodec, messages::abci::Response, &mut bytes::bytes::BytesMut) -> std::result::Result<(), std::boxed::Box<(dyn std::error::Error + 'static)>>`
found type `fn(&mut codec::ABCICodec, messages::abci::Response, &mut bytes::BytesMut) -> std::result::Result<(), std::boxed::Box<(dyn std::error::Error + 'static)>>`
error[E0277]: the trait bound `&std::vec::Vec<u8>: bytes::Buf` is not satisfied
--> src/codec.rs:52:17
|
52 | buf.put(&varint);
| ^^^^^^^ the trait `bytes::Buf` is not implemented for `&std::vec::Vec<u8>`
error[E0599]: no method named `writer` found for type `&mut bytes::BytesMut` in the current scope
--> src/codec.rs:53:38
|
53 | msg.write_to_writer(&mut buf.writer())?;
| ^^^^^^ help: there is a method with a similar name: `iter`
|
= help: items from traits can only be used if the trait is in scope
= note: the following trait is implemented but not in scope, perhaps add a `use` for it:
`use crate::bytes::buf::BufMutExt;`
error: aborting due to 4 previous errors
the project doesn't exist on travis and the travis.yml probably needs an update
Currently, the version compatibility has been non-systematically tracked in README + version.txt
It would be good to have something similar to https://github.com/jTendermint/jabci#compatibility
(probably not possible to reconstruct it in such a detail, but just to have something to start with)
tendermint/tendermint#3643
this is now in the latest release and it changes ABCI
small ABCI type changes;
PR could also include a starting compatibility version table
#58
update protobufs
Unable to compile a simple example due to erros of type:
error[E0425]: cannot find value VERSION_2_8_0
in module protobuf
--> /.../.cargo/registry/src/github.com-1ecc6299db9ec823/abci-0.6.1/src/messages/abci.rs:27:49
|
27 | const _PROTOBUF_VERSION_CHECK: () = ::protobuf::VERSION_2_8_0;
| ^^^^^^^^^^^^^ help: a constant with a similar name exists: VERSION_2_8_1
github.com-1ecc6299db9ec823/abci-0.6.1
Just trying to setup the framework with the example from https://docs.rs/abci/0.6.1/abci/
Would be good to check at least that each request type gets back a corresponding response type
Currently travis fails because build.rs
can't find src/types.proto
. travis needs to be able to find that file and compile it.
Upgrade protobuf
version to latest (2.10.0
)
Shouldn't we rather use the latest release (2.6.2) instead (and regenerate the rust code using that release)?
Regarding the lock file, https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html states that
If you’re building a non-end product, such as a rust library that other rust packages will depend on, put Cargo.lock in your .gitignore. If you’re building an end product, which are executable like command-line tool or an application, or a system library with crate-type of staticlib or cdylib, check Cargo.lock into git.
rust-abci is more of a library and not an end-product. At the same time, I think adding the cargo lock file would increase reproducibility. Which seems important for rust-abci IMHO. Let's discuss this in a separate issue.
Issue #10 is still open not to mention that 0.21 is the current version.
fee6aa0#diff-4ce93534efc34e923ce01e975eb7ed80R18
since rust-abci
is a library, it shouldn't initialize the logger, as that should be initialized by applications that use it
Cannot compile using a stable release due to:#![feature(ptr_internals)]
line 2 of lib.rs
error[E0554]: #![feature] may not be used on the stable release channel --> /..../rust-tsp-5c599386875a2e57/39d001d/src/lib.rs:2:1\ | 2 | #![feature(ptr_internals)] | ^^^^^^^^^^^^^^^^^^^^^^^^^^
This is on OS X using rustc 1.28.0
I think this is due to the use of std::ptr::Unique
in tcpserver which is a nightly only feature. Note std::ptr::Unique does not even appear in the 1.28 docs.
This is needed because the precompiled types.rs file doesn't seem to work across platforms.
The default features of the tokio
crate bring quite a few tokio-*
crates that are most likely unused (e.g. tokio-fs
). It'll be good to reduce the number of these crates with a more explicit dependency in Cargo.toml:
tokio = { version = "0.1", default-features = false, features = ["codec", ....] }
where in features
, one would only list the used / needed features of tokio.
There are a few protocol assumptions that are currently unchecked / trusted, e.g. the block execution state machine:
BeginBlock -> [DeliverTx]* -> EndBlock -> Commit -> BeginBlock -> ...
This could be encoded in session types: https://github.com/Munksgaard/session-types
there are 3 connections:
messages exchanges in info and mempool connections probably do not require to borrow ABCI applications as mutable, so it may be possible to rearchitect ABCI without having the mutex on the entire application (so that, for example, CheckTX/Info/Query can be processed while consensus operations are going)
One thing a library never can have too many of is examples! Right now, our only example is a bare-bones implementation of an Application
. Now that #13 is merged, we'll actually be able to create a mutable-state application as an example. A 'counter' application would probably be a good and simple example of an ABCI app, so I was thinking that would be a good example to add to the library. I know we had one before, but that one didn't actually function as a counter app.
from here
Some of the message types generated by protobuf have a bit imprecise types -- for example, the block height is returned as i64
even though it should never be negative
It supports Rust.
I guess it'll make sense after #63
error: failed to select a version for `tokio`.
... required by package `abci v0.6.4 (/Users/markobaricevic/code/abak0/CoTend/rust-abci)`
versions that meet the requirements `^0.2` are: 0.2.1, 0.2.0
the package `abci` depends on `tokio`, with features: `rt-full, codec, io` but `tokio` does not have these features.
failed to select a version for `tokio` which could resolve this conflict
&&
➜ cargo build
Compiling abci v0.6.4 (/Users/markobaricevic/code/abak0/CoTend/rust-abci)
error[E0432]: unresolved import `tokio::codec`
--> src/codec.rs:6:12
|
6 | use tokio::codec::{Decoder, Encoder};
| ^^^^^ could not find `codec` in `tokio`
error[E0432]: unresolved import `tokio::codec`
--> src/server.rs:7:12
|
7 | use tokio::codec::Decoder;
| ^^^^^ could not find `codec` in `tokio`
error[E0432]: unresolved import `tokio::net::TcpListener`
--> src/server.rs:9:5
|
9 | use tokio::net::TcpListener;
| ^^^^^^^^^^^^^^^^^^^^^^^ no `TcpListener` in `net`
error[E0412]: cannot find type `Result` in module `io`
--> src/server.rs:18:50
|
18 | pub fn serve<A>(app: A, addr: SocketAddr) -> io::Result<()>
| ^^^^^^ not found in `io`
|
help: possible candidates are found in other modules, you can import them into scope
|
1 | use core::fmt::Result;
|
1 | use core::prelude::v1::Result;
|
1 | use core::result::Result;
|
1 | use futures::core_reexport::fmt::Result;
|
and 8 other candidates
error[E0425]: cannot find function `spawn` in crate `tokio`
--> src/server.rs:45:20
|
45 | tokio::spawn(writes.then(|_| Ok(())))
| ^^^^^ not found in `tokio`
|
help: possible candidate is found in another module, you can import it into scope
|
1 | use std::thread::spawn;
|
warning: unused import: `tokio::prelude::*`
--> src/server.rs:10:5
|
10 | use tokio::prelude::*;
| ^^^^^^^^^^^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
error: aborting due to 5 previous errors
Some errors have detailed explanations: E0412, E0425, E0432.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `abci`.
It should be without any complications -- it's just put as an issue, because the automated dependabot PRs don't seem to commit the re-generated files (and one would like to verify that)
related to #38 because i went to manually publish the crate and running cargo build
updated some files that I think should be already updated (still figuring out rust)
currently, only TCP is supported
It appears mockstream
is no longer used. But it's still in the deps list in cargo.toml
.
Due to tendermint/abci#205, this implementation has become incompatible with Tendermint v0.16 and newer.
The breakage was caused by how varints are now encoded.
Instead of a custom encoding, Tendermint now uses the protobuf default approach of ZigZag encoding.
rust-protobuf seems to support that encoding (see doc).
The update likely also requires syncing src/types.proto to the one in tendermint/abci (as far as I understand, it needs to be v1.10+). I already tried that (manually with protoc
on the cmdline), but didn't yet manage to compile the included https://github.com/tendermint/abci/blob/master/types/types.proto such that the types KVPair
and KI64Pair
get properly included and namespaced.
Btw. the README lacks the information that protobuf support needs to be installed on the system.
For some reason, the old test capturing this bug was removed: #20
There are no tests for the "killer" feature which is that query and mempool connections don't block progress on the consensus connection
Recently, while trying to run this basic block-increment test:
extern crate abci;
use abci::*;
struct EmptyApp { block: i64 }
impl Application for EmptyApp {
// Info/Query connection
fn info(&mut self, req: &RequestInfo) -> ResponseInfo {
let mut r = ResponseInfo::new();
r.set_last_block_height(self.block);
r
}
fn set_option(&mut self, req: &RequestSetOption) -> ResponseSetOption {
ResponseSetOption::new()
}
fn query(&mut self, p: &RequestQuery) -> ResponseQuery {
ResponseQuery::new()
}
// Mempool connection
fn check_tx(&mut self, p: &RequestCheckTx) -> ResponseCheckTx {
ResponseCheckTx::new()
}
// Consensus connection
fn init_chain(&mut self, p: &RequestInitChain) -> ResponseInitChain {
ResponseInitChain::new()
}
fn begin_block(&mut self, p: &RequestBeginBlock) -> ResponseBeginBlock {
ResponseBeginBlock::new()
}
fn deliver_tx(&mut self, p: &RequestDeliverTx) -> ResponseDeliverTx {
ResponseDeliverTx::new()
}
fn end_block(&mut self, p: &RequestEndBlock) -> ResponseEndBlock {
ResponseEndBlock::new()
}
fn commit(&mut self, p: &RequestCommit) -> ResponseCommit {
self.block += **1;**
ResponseCommit::new()
}
}
fn main() {
static APP: EmptyApp = EmptyApp { block: 0 };
let addr = "127.0.0.1:26658".parse().unwrap();
server::new(addr, &APP);
}
I got a rather nasty segfault at self.block += 1;
. Running with valgrind
produced this specific segfault:
==9819== Memcheck, a memory error detector
==9819== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==9819== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==9819== Command: ./target/debug/examples/other_app
==9819==
0x49a1c8
Commit
0x49a1c8
==9819==
==9819== Process terminating with default action of signal 11 (SIGSEGV)
==9819== Bad permissions for mapped region at address 0x49A1C8
==9819== at 0x1423E4: <other_app::EmptyApp as abci::Application>::commit (other_app.rs:48)
==9819== by 0x1B2257: abci::tcpserver::respond (tcpserver.rs:95)
==9819== by 0x1B173A: abci::tcpserver::handle_stream (tcpserver.rs:59)
==9819== by 0x144F4F: <abci::tcpserver::TCPServer<A>>::serve::{{closure}} (tcpserver.rs:37)
==9819== by 0x14371A: std::sys_common::backtrace::__rust_begin_short_backtrace (backtrace.rs:133)
==9819== by 0x13EE52: std::thread::Builder::spawn::{{closure}}::{{closure}} (mod.rs:406)
==9819== by 0x1427CA: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once (panic.rs:293)
==9819== by 0x1407C7: std::panicking::try::do_call (panicking.rs:479)
==9819== by 0x45A0EE: __rust_maybe_catch_panic (lib.rs:102)
==9819== by 0x14070E: std::panicking::try (panicking.rs:458)
==9819== by 0x142801: std::panic::catch_unwind (panic.rs:358)
==9819== by 0x13ECC3: std::thread::Builder::spawn::{{closure}} (mod.rs:405)
==9819==
==9819== HEAP SUMMARY:
==9819== in use at exit: 1,088 bytes in 10 blocks
==9819== total heap usage: 18 allocs, 8 frees, 3,152 bytes allocated
==9819==
==9819== LEAK SUMMARY:
==9819== definitely lost: 0 bytes in 0 blocks
==9819== indirectly lost: 0 bytes in 0 blocks
==9819== possibly lost: 864 bytes in 3 blocks
==9819== still reachable: 224 bytes in 7 blocks
==9819== suppressed: 0 bytes in 0 blocks
==9819== Rerun with --leak-check=full to see details of leaked memory
==9819==
==9819== For counts of detected and suppressed errors, rerun with: -v
==9819== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
[1] 9819 segmentation fault (core dumped) valgrind ./target/debug/examples/other_app
Looking into this, I quickly figured out the problem: Rust had marked the APP
static variable as non-writable, because I had not put mut
there. So reading from self
inside the Application
functions worked fine, but trying to write any values caused a self-fault because writing was not allowed in that region. The fix was easy: use static mut APP
instead of static APP
, and everything worked fine.
Internally, this library uses hacky pointer magic (which I wrote) to convert an immutable reference to a mutable one internally. This is a really bad idea, because we have to trust that the immutable reference can be modified. This works fine with lazy_static
and static mut
. static
? Not so much. Anyone using this library can get seg-faulted unexpectedly, and since they're trusting that this libraries' code only takes an immutable reference (because of our interface), this causes a ton of opaque errors. We should change server::new
to take a mutable reference, so our unsafe code can be much more safe.
Cargo.toml
and what shouldn'tA declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.