crypto-com/tmkms-light

TEE-based Key Management System for Tendermint validators.

tmkms-light's Issues

Problem: custom IMDS client implementation

The Nitro helper uses a custom client for obtaining AWS credentials for KMS: https://github.com/crypto-com/tmkms-light/blob/main/providers/nitro/nitro-helper/src/key_utils.rs#L10
The mentioned issue is resolved and IMDS is supported in the official SDK: https://docs.rs/aws-config/0.9.0/aws_config/imds/index.html so it should be possible to use it.

Problem: upgrade to sys info 0.19 fails

   Compiling tmkms-nitro-helper v0.2.0 (/home/runner/work/tmkms-light/tmkms-light/providers/nitro/nitro-helper)
398
error[E0599]: no method named `get_processes` found for struct `sysinfo::System` in the current scope
399
   --> providers/nitro/nitro-helper/src/command.rs:110:12
400
    |
401
110 |     system.get_processes().iter().any(|(_pid, p)| {
402
    |            ^^^^^^^^^^^^^ method not found in `sysinfo::System`
403

404
error: unused import: `ProcessExt`
405
 --> providers/nitro/nitro-helper/src/command.rs:6:15
406
  |
407
6 | use sysinfo::{ProcessExt, SystemExt};
408
  |               ^^^^^^^^^^
409
  |
410
  = note: `-D unused-imports` implied by `-D warnings`
411

412
error: aborting due to 2 previous errors

Problem: upgrade to vsock 0.2.3 fails

   Compiling tmkms-nitro-helper v0.1.0 (/home/runner/work/tmkms-light/tmkms-light/providers/nitro/nitro-helper)
431
error[E0308]: mismatched types
432
   --> providers/nitro/nitro-helper/src/command.rs:141:54
433
    |
434
141 |         let mut socket = vsock::VsockStream::connect(&addr).map_err(|e| {
435
    |                                                      ^^^^^ expected enum `vsock::SockAddr`, found enum `nix::sys::socket::SockAddr`
436
    |
437
    = note: expected reference `&vsock::SockAddr`
438
               found reference `&nix::sys::socket::SockAddr`
439
    = note: perhaps two different versions of crate `nix` are being used?
440

441
error[E0308]: mismatched types
442
  --> providers/nitro/nitro-helper/src/proxy.rs:34:44
443
   |
444
34 |         let listener = VsockListener::bind(&sockaddr)
445
   |                                            ^^^^^^^^^ expected enum `vsock::SockAddr`, found enum `nix::sys::socket::SockAddr`
446
   |
447
   = note: expected reference `&vsock::SockAddr`
448
              found reference `&nix::sys::socket::SockAddr`
449
   = note: perhaps two different versions of crate `nix` are being used?
450

451
error[E0308]: mismatched types
452
  --> providers/nitro/nitro-helper/src/state.rs:57:50
453
   |
454
57 |         let vsock_listener = VsockListener::bind(&sockaddr).map_err(|e| {
455
   |                                                  ^^^^^^^^^ expected enum `vsock::SockAddr`, found enum `nix::sys::socket::SockAddr`
456
   |
457
   = note: expected reference `&vsock::SockAddr`
458
              found reference `&nix::sys::socket::SockAddr`
459
   = note: perhaps two different versions of crate `nix` are being used?
460

https://github.com/crypto-com/tmkms-light/pull/12/checks?check_run_id=2119561886#step:5:431

Problem: fixed CID for Nitro Enclave

currently, CID is configured in tmkms.toml, but every time the enclave is launched, CID changes... it may be better to allow "auto" configuration -- e.g. trying to call similarly to https://github.com/aws/aws-nitro-enclaves-cli/blob/main/src/enclave_proc/commands.rs#L108 and if there's only one enclave, just pick its CID otherwise report an error?

Or perhaps at least allow passing CID as a command line argument (requested by @allthatjazzleo )

Problem: no production mode logging from enclave on NE

On Nitro Enclave, one doesn't have a console output in the production mode.
with tracing crate, one can potentially create a custom subscriber that will redirect the logs to a designated vsock port that the helper on the host will listen on and redirect to a console or a file

Problem: upgrade to prost 0.7 fails

   Compiling tmkms-light v0.1.0 (/home/runner/work/tmkms-light/tmkms-light)
280
error[E0599]: no function or associated item named `decode_length_delimited` found for struct `tendermint_proto::privval::Message` in the current scope
281
  --> src/rpc.rs:41:32
282
   |
283
41 |         let msg = PrivMessage::decode_length_delimited(msg.as_ref())
284
   |                                ^^^^^^^^^^^^^^^^^^^^^^^ function or associated item not found in `tendermint_proto::privval::Message`
285
   |
286
   = help: items from traits can only be used if the trait is in scope
287
   = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
288
           `use prost::message::Message;`
289

290
error[E0599]: no method named `encode_length_delimited` found for struct `tendermint_proto::privval::Message` in the current scope
291
   --> src/rpc.rs:191:14
292
    |
293
191 |             .encode_length_delimited(&mut buf)
294
    |              ^^^^^^^^^^^^^^^^^^^^^^^ method not found in `tendermint_proto::privval::Message`
295
    | 
296
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/prost-0.6.1/src/message.rs:64:8
297
    |
298
64  |     fn encode_length_delimited<B>(&self, buf: &mut B) -> Result<(), EncodeError>
299
    |        ----------------------- the method is available for `Box<tendermint_proto::privval::Message>` here
300
    |
301
    = help: items from traits can only be used if the trait is in scope
302
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
303
            `use prost::message::Message;`
304

305
error: unused import: `prost::Message`
306
 --> src/rpc.rs:7:5
307
  |
308
7 | use prost::Message as _;
309
  |     ^^^^^^^^^^^^^^
310
  |
311
  = note: `-D unused-imports` implied by `-D warnings`
312

https://github.com/crypto-com/tmkms-light/pull/7/checks?check_run_id=2119555563#step:5:279

Problem: upgrade to prost 0.8.0 fails

error[E0599]: no function or associated item named `decode_length_delimited` found for struct `tendermint_proto::privval::Message` in the current scope
523
  --> src/rpc.rs:41:32
524
   |
525
41 |         let msg = PrivMessage::decode_length_delimited(msg.as_ref())
526
   |                                ^^^^^^^^^^^^^^^^^^^^^^^ function or associated item not found in `tendermint_proto::privval::Message`
527
   |
528
   = help: items from traits can only be used if the trait is in scope
529
   = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
530
           `use prost::message::Message;`
531

532
error[E0599]: no method named `encode_length_delimited` found for struct `tendermint_proto::privval::Message` in the current scope
533
   --> src/rpc.rs:191:14
534
    |
535
191 |             .encode_length_delimited(&mut buf)
536
    |              ^^^^^^^^^^^^^^^^^^^^^^^ method not found in `tendermint_proto::privval::Message`
537
    | 
538
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/prost-0.7.0/src/message.rs:65:8
539
    |
540
65  |     fn encode_length_delimited<B>(&self, buf: &mut B) -> Result<(), EncodeError>
541
    |        ----------------------- the method is available for `tendermint_proto::privval::Message` here
542
    |
543
    = help: items from traits can only be used if the trait is in scope
544
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
545
            `use prost::message::Message;`
546

547
error: unused import: `prost::Message`
548
 --> src/rpc.rs:7:5
549
  |
550
7 | use prost::Message as _;
551
  |     ^^^^^^^^^^^^^^
552
  |
553
  = note: `-D unused-imports` implied by `-D warnings`
554

555
error: aborting due to 3 previous errors

Problem: Nitro helper requires a separate enclave launching

it may be more convenient to have a "launch-all" command that would need the EIF file path and do the required steps (run-enclave + if not running, perhaps vsock_proxy)...
This may perhaps be better done when https://github.com/aws/aws-nitro-enclaves-cli is exposed as a library / crate, so the functionality can be directly called (instead of indirectly invoking it via a shell command)

Problem: upgrade to rsa 0.5.0 fails

https://github.com/crypto-com/tmkms-light/pull/140/checks?check_run_id=3224899855#step:4:385

error[E0412]: cannot find type `RSAPublicKey` in crate `rsa`
386
   --> providers/sgx/sgx-runner/src/shared.rs:227:28
387
    |
388
227 |         wrap_pub_key: rsa::RSAPublicKey,
389
    |                            ^^^^^^^^^^^^ help: a struct with a similar name exists: `RsaPublicKey`
390
    | 
391
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/rsa-0.5.0/src/key.rs:48:1
392
    |
393
48  | pub struct RsaPublicKey {
394
    | ----------------------- similarly named struct `RsaPublicKey` defined here
395

396
error[E0412]: cannot find type `RSAPublicKey` in crate `rsa`
397
   --> providers/sgx/sgx-runner/src/shared.rs:243:38
398
    |
399
243 | pub fn get_claim(wrap_pub_key: &rsa::RSAPublicKey) -> String {
400
    |                                      ^^^^^^^^^^^^ help: a struct with a similar name exists: `RsaPublicKey`
401
    | 
402
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/rsa-0.5.0/src/key.rs:48:1
403
    |
404
48  | pub struct RsaPublicKey {
405
    | ----------------------- similarly named struct `RsaPublicKey` defined here
406

407
error: unused import: `rsa::PublicKeyParts`
408
 --> providers/sgx/sgx-runner/src/shared.rs:1:5
409
  |
410
1 | use rsa::PublicKeyParts;
411
  |     ^^^^^^^^^^^^^^^^^^^
412
  |
413
  = note: `-D unused-imports` implied by `-D warnings`
414

Problem: `priv_validator_state.json` may get corrupted

@borischeuk-crypto came across an issue where priv_validator_state.json was empty and the nitro helper would crash with:

failed to get a state syncing helper: Error(Context { kind: SyncError, source: Some(Message("error parsing .../.tmkms/state/priv_validator_state.json: EOF while parsing a value at line 1 column 0")), backtrace: Some(   0: tmkms_nitro_helper::command::start

There are probably a few ways how this can be improved:

1. check the state syncing helper code whether atomic writing via tempfile can be improved (also not sure if the helper @borischeuk-crypto used had https://github.com/crypto-com/tmkms-light/pull/115/files )
2. improve the error message to e.g. give some troubleshooting suggestion (re-initialize the file with the latest block height?)
3. add a different option for storing/retrieving priv_validator_state.json... e.g. with DynamoDB instead of a local file

Problem: option key_backup_data_path means differently for command init and recover in sgx provider

Option key_backup_data_path in command::init refer to directory where consensus-key.backup will be created
while
Option key_backup_data_path in command::recover refer to the file path of consensus-key.backup

It could be confusing.

key_backup_data_path in command::recover should refer to directory as well?

Problem: yanked and unmaintained dependencies

error: 3 denied warnings found!
Crate:         cpuid-bool
Version:       0.1.2
Warning:       unmaintained
Title:         `cpuid-bool` has been renamed to `cpufeatures`
Date:          2021-05-06
ID:            RUSTSEC-2021-0064
URL:           https://rustsec.org/advisories/RUSTSEC-2021-0064
Dependency tree: 
cpuid-bool 0.1.2

Crate:         cpuid-bool
Version:       0.2.0
Warning:       unmaintained
Title:         `cpuid-bool` has been renamed to `cpufeatures`
Date:          2021-05-06
ID:            RUSTSEC-2021-0064
URL:           https://rustsec.org/advisories/RUSTSEC-2021-0064
Dependency tree: 
cpuid-bool 0.2.0

Crate:         aead
Version:       0.4.0
Warning:       yanked
Dependency tree: 
aead 0.4.0

Problem: legacy rusoto crates are used

one can use the official AWS SDK: https://github.com/awslabs/aws-sdk-rust

Problem: a yank crate version

Crate:         crypto-bigint
14
Version:       0.2.8
15
Warning:       yanked
16
Dependency tree: 
17
crypto-bigint 0.2.8
18
└── der 0.4.0
19
    ├── spki 0.4.0
20
    │   └── pkcs8 0.7.5
21
    │       └── rsa 0.5.0
22
    │           ├── tmkms-light-sgx-runner 0.2.0
23
    │           │   └── tmkms-light-sgx-app 0.2.0
24
    │           └── tmkms-light-sgx-app 0.2.0
25
    ├── pkcs8 0.7.5
26
    └── pkcs1 0.2.3
27
        ├── rsa 0.5.0
28

Problem: upgrading sha2 to 0.10.* fails to compile on sgx targets

   Compiling tmkms-light-sgx-app v0.3.0 (/home/runner/work/tmkms-light/tmkms-light/providers/sgx/sgx-app)
388
error[E0277]: the trait bound `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>: digest::Update` is not satisfied
389
   --> providers/sgx/sgx-app/src/sgx_app/cloud.rs:78:13
390
    |
391
78  |             PaddingScheme::new_oaep::<sha2::Sha256>(),
392
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `digest::Update` is not implemented for `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>`
393
    |
394
    = note: required because of the requirements on the impl of `ed25519_dalek::Digest` for `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>`
395
note: required by a bound in `PaddingScheme::new_oaep`
396
   --> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/rsa-0.5.0/src/padding.rs:111:34
397
    |
398
111 |     pub fn new_oaep<T: 'static + Digest + DynDigest>() -> Self {
399
    |                                  ^^^^^^ required by this bound in `PaddingScheme::new_oaep`
400

401
error[E0277]: the trait bound `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>: digest::Reset` is not satisfied
402
   --> providers/sgx/sgx-app/src/sgx_app/cloud.rs:78:13
403
    |
404
78  |             PaddingScheme::new_oaep::<sha2::Sha256>(),
405
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `digest::Reset` is not implemented for `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>`
406
    |
407
    = note: required because of the requirements on the impl of `ed25519_dalek::Digest` for `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>`
408
note: required by a bound in `PaddingScheme::new_oaep`
409
   --> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/rsa-0.5.0/src/padding.rs:111:34
410
    |
411
111 |     pub fn new_oaep<T: 'static + Digest + DynDigest>() -> Self {
412
    |                                  ^^^^^^ required by this bound in `PaddingScheme::new_oaep`
413

414
error[E0277]: the trait bound `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>: digest::fixed::FixedOutputDirty` is not satisfied
415
   --> providers/sgx/sgx-app/src/sgx_app/cloud.rs:78:13
416
    |
417
78  |             PaddingScheme::new_oaep::<sha2::Sha256>(),
418
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `digest::fixed::FixedOutputDirty` is not implemented for `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>`
419
    |
420
    = note: required because of the requirements on the impl of `digest::fixed::FixedOutput` for `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>`
421
    = note: required because of the requirements on the impl of `ed25519_dalek::Digest` for `CoreWrapper<CtVariableCoreWrapper<Sha256VarCore, aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<aes_gcm_siv::aead::generic_array::typenum::UInt<UTerm, B1>, B0>, B0>, B0>, B0>, B0>>>`
422
note: required by a bound in `PaddingScheme::new_oaep`
423
   --> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/rsa-0.5.0/src/padding.rs:111:34
424
    |
425
111 |     pub fn new_oaep<T: 'static + Digest + DynDigest>() -> Self {
426
    |                                  ^^^^^^ required by this bound in `PaddingScheme::new_oaep`

https://github.com/crypto-com/tmkms-light/runs/4512066100?check_suite_focus=true#step:4:387

Problem: upgrade to tendermint 0.18 fails

   Compiling tmkms-light v0.1.0 (/home/runner/work/tmkms-light/tmkms-light)
286
error[E0277]: the trait bound `tendermint::vote::SignVoteRequest: From<tendermint_proto::privval::SignVoteRequest>` is not satisfied
287
  --> src/rpc.rs:47:27
288
   |
289
47 |                 let svr = SignVoteRequest::try_from(req).map_err(|e| {
290
   |                           ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<tendermint_proto::privval::SignVoteRequest>` is not implemented for `tendermint::vote::SignVoteRequest`
291
   |
292
   = note: required because of the requirements on the impl of `Into<tendermint::vote::SignVoteRequest>` for `tendermint_proto::privval::SignVoteRequest`
293
   = note: required because of the requirements on the impl of `TryFrom<tendermint_proto::privval::SignVoteRequest>` for `tendermint::vote::SignVoteRequest`
294
   = note: required by `try_from`
295

296
error[E0277]: the trait bound `tendermint::proposal::SignProposalRequest: From<tendermint_proto::privval::SignProposalRequest>` is not satisfied
297
  --> src/rpc.rs:57:27
298
   |
299
57 |                 let spr = SignProposalRequest::try_from(spr).map_err(|e| {
300
   |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<tendermint_proto::privval::SignProposalRequest>` is not implemented for `tendermint::proposal::SignProposalRequest`
301
   |
302
   = note: required because of the requirements on the impl of `Into<tendermint::proposal::SignProposalRequest>` for `tendermint_proto::privval::SignProposalRequest`
303
   = note: required because of the requirements on the impl of `TryFrom<tendermint_proto::privval::SignProposalRequest>` for `tendermint::proposal::SignProposalRequest`
304
   = note: required by `try_from`
305

306
error[E0277]: the trait bound `tendermint::public_key::PubKeyRequest: From<tendermint_proto::privval::PubKeyRequest>` is not satisfied
307
  --> src/rpc.rs:67:27
308
   |
309
67 |                 let pkr = PubKeyRequest::try_from(pkr).map_err(|e| {
310
   |                           ^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<tendermint_proto::privval::PubKeyRequest>` is not implemented for `tendermint::public_key::PubKeyRequest`
311
   |
312
   = note: required because of the requirements on the impl of `Into<tendermint::public_key::PubKeyRequest>` for `tendermint_proto::privval::PubKeyRequest`
313
   = note: required because of the requirements on the impl of `TryFrom<tendermint_proto::privval::PubKeyRequest>` for `tendermint::public_key::PubKeyRequest`
314
   = note: required by `try_from`
315

316
error[E0277]: the trait bound `tendermint_proto::privval::SignedVoteResponse: From<tendermint::vote::SignedVoteResponse>` is not satisfied
317
   --> src/rpc.rs:162:67
318
    |
319
162 |             Response::SignedVote(resp) => Sum::SignedVoteResponse(resp.into()),
320
    |                                                                   ^^^^^^^^^^^ the trait `From<tendermint::vote::SignedVoteResponse>` is not implemented for `tendermint_proto::privval::SignedVoteResponse`
321
    |
322
    = help: the following implementations were found:
323
              <tendermint_proto::privval::SignedVoteResponse as From<tendermint::vote::sign_vote::SignedVoteResponse>>
324
    = note: required because of the requirements on the impl of `Into<tendermint_proto::privval::SignedVoteResponse>` for `tendermint::vote::SignedVoteResponse`
325

326
error[E0277]: the trait bound `tendermint_proto::privval::SignedProposalResponse: From<tendermint::proposal::SignedProposalResponse>` is not satisfied
327
   --> src/rpc.rs:163:75
328
    |
329
163 |             Response::SignedProposal(resp) => Sum::SignedProposalResponse(resp.into()),
330
    |                                                                           ^^^^^^^^^^^ the trait `From<tendermint::proposal::SignedProposalResponse>` is not implemented for `tendermint_proto::privval::SignedProposalResponse`
331
    |
332
    = help: the following implementations were found:
333
              <tendermint_proto::privval::SignedProposalResponse as From<tendermint::proposal::sign_proposal::SignedProposalResponse>>
334
    = note: required because of the requirements on the impl of `Into<tendermint_proto::privval::SignedProposalResponse>` for `tendermint::proposal::SignedProposalResponse`
335

336
error: aborting due to 5 previous errors
337

338
For more information about this error, try `rustc --explain E0277`.

https://github.com/crypto-com/tmkms-light/pull/14/checks?check_run_id=2119673471#step:5:330

Problem: upgrade to rand_core 0.6.2 fails

   Compiling tmkms-nitro-helper v0.1.0 (/home/runner/work/tmkms-light/tmkms-light/providers/nitro/nitro-helper)
429
error[E0277]: the trait bound `OsRng: rand_core::CryptoRng` is not satisfied
430
   --> providers/nitro/nitro-helper/src/key_utils.rs:19:46
431
    |
432
19  |     let keypair: Keypair = Keypair::generate(&mut csprng);
433
    |                                              ^^^^^^^^^^^ the trait `rand_core::CryptoRng` is not implemented for `OsRng`
434
    | 
435
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/ed25519-dalek-1.0.1/src/keypair.rs:129:12
436
    |
437
129 |         R: CryptoRng + RngCore,
438
    |            --------- required by this bound in `Keypair::generate`
439

440
error[E0277]: the trait bound `OsRng: rand_core::RngCore` is not satisfied
441
   --> providers/nitro/nitro-helper/src/key_utils.rs:19:46
442
    |
443
19  |     let keypair: Keypair = Keypair::generate(&mut csprng);
444
    |                                              ^^^^^^^^^^^ the trait `rand_core::RngCore` is not implemented for `OsRng`
445
    | 
446
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/ed25519-dalek-1.0.1/src/keypair.rs:129:24
447
    |
448
129 |         R: CryptoRng + RngCore,
449
    |                        ------- required by this bound in `Keypair::generate`
450

451
error: aborting due to 2 previous errors

https://github.com/crypto-com/tmkms-light/pull/17/checks?check_run_id=2119555530#step:5:442

Problem: IMDSv1 used to obtain the credentials information

use IMDSv2

Problem: upgrade to tracing-subscribed 0.3 fails

   Compiling tmkms-nitro-helper v0.2.0 (/home/runner/work/tmkms-light/tmkms-light/providers/nitro/nitro-helper)
392
error[E0407]: method `new_span` is not a member of trait `tracing_subscriber::Layer`
393
  --> providers/nitro/nitro-helper/src/tracing_layer.rs:52:5
394
   |
395
52 | /     fn new_span(&self, attrs: &Attributes, id: &Id, ctx: Context<S>) {
396
53 | |         let span = ctx.span(id).expect("unknown span");
397
54 | |         let mut buf = Vec::with_capacity(256);
398
55 | |
399
...  |
400
68 | |         span.extensions_mut().insert(SpanFields(buf));
401
69 | |     }
402
   | |_____^ not a member of trait `tracing_subscriber::Layer`
403

Problem: hazardous cloud backup/recovery option

currently, one needs a symmetric key in plaintext on the machine temporarily

Problem: upgrade to tracing-subscriber 0.2.19 fails

due to deprecated functions https://github.com/crypto-com/tmkms-light/pull/111/checks?check_run_id=2936133752#step:5:393

   Compiling tmkms-nitro-helper v0.2.0 (/home/runner/work/tmkms-light/tmkms-light/providers/nitro/nitro-helper)
394
error: use of deprecated associated function `tracing_subscriber::registry::SpanRef::<'a, R>::parents`: equivalent to `self.parent().into_iter().flat_map(SpanRef::scope)`, but consider whether excluding `self` is actually intended
395
  --> providers/nitro/nitro-helper/src/tracing_layer.rs:56:26
396
   |
397
56 |         let depth = span.parents().count();
398
   |                          ^^^^^^^
399
   |
400
   = note: `-D deprecated` implied by `-D warnings`
401

402
error: use of deprecated associated function `tracing_subscriber::registry::SpanRef::<'a, R>::parents`: equivalent to `self.parent().into_iter().flat_map(SpanRef::scope)`, but consider whether excluding `self` is actually intended
403
  --> providers/nitro/nitro-helper/src/tracing_layer.rs:73:26
404
   |
405
73 |         let depth = span.parents().count();
406
   |                          ^^^^^^^
407

408
error: use of deprecated associated function `tracing_subscriber::layer::Context::<'a, S>::scope`: equivalent to `self.current_span().id().and_then(|id| self.span_scope(id).from_root())` but consider passing an explicit ID instead of relying on the contextual span
409
  --> providers/nitro/nitro-helper/src/tracing_layer.rs:87:25
410
   |
411
87 |         for span in ctx.scope() {
412
   |                         ^^^^^
413

414
error: aborting due to 3 previous errors
415

416
error: could not compile `tmkms-nitro-helper`
417

418
To learn more, run the command again with --verbose.
419
warning: build failed, waiting for other jobs to finish...
420
error: build failed
421
Error: Process completed with exit code 101.

Problem: upgrade to tendermint 0.21 fails

error[E0277]: the trait bound `tendermint_proto::privval::SignedProposalResponse: From<tendermint::proposal::SignedProposalResponse>` is not satisfied
346
   --> src/rpc.rs:163:75
347
    |
348
163 |             Response::SignedProposal(resp) => Sum::SignedProposalResponse(resp.into()),
349
    |                                                                           ^^^^^^^^^^^ the trait `From<tendermint::proposal::SignedProposalResponse>` is not implemented for `tendermint_proto::privval::SignedProposalResponse`
350
    |
351
    = help: the following implementations were found:
352
              <tendermint_proto::privval::SignedProposalResponse as From<tendermint::proposal::sign_proposal::SignedProposalResponse>>
353
    = note: required because of the requirements on the impl of `Into<tendermint_proto::privval::SignedProposalResponse>` for `tendermint::proposal::SignedProposalResponse`

https://github.com/crypto-com/tmkms-light/pull/131/checks?check_run_id=3165853429#step:5:345

Problem: Nitro build fails with upgrade of zeroize 1.4.0

#30 1.787   Downloaded zeroize v1.4.0
1850
#30 1.811 error: failed to parse manifest at `/root/.cargo/registry/src/github.com-1ecc6299db9ec823/zeroize-1.4.0/Cargo.toml`
1851
#30 1.811 
1852
#30 1.811 Caused by:
1853
#30 1.811   feature `resolver` is required
1854
#30 1.811 
1855
#30 1.811   this Cargo does not support nightly features, but if you
1856
#30 1.812   switch to nightly channel you can add
1857
#30 1.812   `cargo-features = ["resolver"]` to enable this feature
1858
#30 ERROR: executor failed running [/bin/sh -c cargo build         --target=x86_64-unknown-linux-musl         --package tmkms-nitro-enclave         --release     && cp target/x86_64-unknown-linux-musl/release/tmkms-nitro-enclave /usr/bin/tmkms-nitro-enclave]: exit code: 101
1859
------
1860
 > [builder 3/8] RUN cargo build         --target=x86_64-unknown-linux-musl         --package tmkms-nitro-enclave         --release     && cp target/x86_64-unknown-linux-musl/release/tmkms-nitro-enclave /usr/bin/tmkms-nitro-enclave:
1861
------
1862
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c cargo build         --target=x86_64-unknown-linux-musl         --package tmkms-nitro-enclave         --release     && cp target/x86_64-unknown-linux-musl/release/tmkms-nitro-enclave /usr/bin/tmkms-nitro-enclave]: exit code: 101
1863
Error: buildx failed with: error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c cargo build         --target=x86_64-unknown-linux-musl         --package tmkms-nitro-enclave         --release     && cp target/x86_64-unknown-linux-musl/release/tmkms-nitro-enclave /usr/bin/tmkms-nitro-enclave]: exit code: 101

https://github.com/crypto-com/tmkms-light/pull/128/checks?check_run_id=3108220791#step:3:1849

Problem: upgrade to rand 0.8.3 fails

  Compiling tmkms-light-sgx-app v0.1.0 (/home/runner/work/tmkms-light/tmkms-light/providers/sgx/sgx-app)
323
error[E0277]: the trait bound `OsRng: rand_core::CryptoRng` is not satisfied
324
   --> providers/sgx/sgx-app/src/sgx_app.rs:100:40
325
    |
326
100 |             let kp = Keypair::generate(&mut csprng);
327
    |                                        ^^^^^^^^^^^ the trait `rand_core::CryptoRng` is not implemented for `OsRng`
328
    | 
329
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/ed25519-dalek-1.0.1/src/keypair.rs:129:12
330
    |
331
129 |         R: CryptoRng + RngCore,
332
    |            --------- required by this bound in `Keypair::generate`
333

334
error[E0277]: the trait bound `OsRng: rand_core::RngCore` is not satisfied
335
   --> providers/sgx/sgx-app/src/sgx_app.rs:100:40
336
    |
337
100 |             let kp = Keypair::generate(&mut csprng);
338
    |                                        ^^^^^^^^^^^ the trait `rand_core::RngCore` is not implemented for `OsRng`
339
    | 
340
   ::: /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/ed25519-dalek-1.0.1/src/keypair.rs:129:24
341
    |
342
129 |         R: CryptoRng + RngCore,
343
    |                        ------- required by this bound in `Keypair::generate`

https://github.com/crypto-com/tmkms-light/pull/4/checks?check_run_id=2119575521#step:4:340

Problem: upgrade aes to 0.8 fails on sgx targets

error[E0310]: the parameter type `BS` may not live long enough
[424](https://github.com/crypto-com/tmkms-light/runs/5190820448?check_suite_focus=true#step:4:424)
   --> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/cipher-0.4.0/src/stream_core.rs:272:5
[425](https://github.com/crypto-com/tmkms-light/runs/5190820448?check_suite_focus=true#step:4:425)
    |
[426](https://github.com/crypto-com/tmkms-light/runs/5190820448?check_suite_focus=true#step:4:426)
272 |     type BlockSize = BS;
[427](https://github.com/crypto-com/tmkms-light/runs/5190820448?check_suite_focus=true#step:4:427)
    |     ^^^^^^^^^^^^^^^^^^^^

Problem: yanked crates

error: 2 denied warnings found!
13
Crate:         base64ct
14
Version:       1.0.0
15
Warning:       yanked
16
Dependency tree: 
17
base64ct 1.0.0
18
└── pem-rfc7468 0.2.0
19
    ├── pkcs8 0.7.5
20
    │   └── rsa 0.5.0
21
    │       ├── tmkms-light-sgx-runner 0.2.0
22
    │       │   └── tmkms-light-sgx-app 0.2.0
23
    │       └── tmkms-light-sgx-app 0.2.0
24
    └── pkcs1 0.2.3
25
        ├── rsa 0.5.0
26
        └── pkcs8 0.7.5
27

28
Crate:         crypto-bigint
29
Version:       0.2.2
30
Warning:       yanked
31
Dependency tree: 
32
crypto-bigint 0.2.2
33
└── der 0.4.0
34
    ├── spki 0.4.0
35
    │   └── pkcs8 0.7.5
36
    │       └── rsa 0.5.0
37
    │           ├── tmkms-light-sgx-runner 0.2.0
38
    │           │   └── tmkms-light-sgx-app 0.2.0
39
    │           └── tmkms-light-sgx-app 0.2.0
40
    ├── pkcs8 0.7.5
41
    └── pkcs1 0.2.3
42
        ├── rsa 0.5.0
43
        └── pkcs8 0.7.5

https://github.com/crypto-com/tmkms-light/runs/3344416765?check_suite_focus=true#step:5:12

Problem: CI actions sometimes not executed on PRs

e.g. #94 didn't execute the tmkms.yml workflow

Problem: Nitro enclave environment arguments are not documented

Right now, the default Dockerfile executes the enclave without any arguments: https://github.com/crypto-com/tmkms-light/blob/main/Dockerfile.nitro#L246

With that Dockerfile, default choices will be used: https://github.com/crypto-com/tmkms-light/blob/main/providers/nitro/nitro-enclave/src/main.rs#L15

1. the enclave code will listen on the vsock port 5050 for the config;
2. the enclave code will listen on the vsock port 6050 for forwarding the logging information;
3. the enclave code will use the INFO log level.

This may be confusing to node operators who may expect the enclave code would follow their configuration provided in the helper code on their host (in the toml configuration or from command line arguments).

Problem: upgrade to tendermint-rs 0.19 fails

error[E0277]: the trait bound `tendermint::vote::SignVoteRequest: From<tendermint_proto::privval::SignVoteRequest>` is not satisfied
297
  --> src/rpc.rs:47:27
298
   |
299
47 |                 let svr = SignVoteRequest::try_from(req).map_err(|e| {
300
   |                           ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<tendermint_proto::privval::SignVoteRequest>` is not implemented for `tendermint::vote::SignVoteRequest`
301
   |
302
   = note: required because of the requirements on the impl of `Into<tendermint::vote::SignVoteRequest>` for `tendermint_proto::privval::SignVoteRequest`
303
   = note: required because of the requirements on the impl of `TryFrom<tendermint_proto::privval::SignVoteRequest>` for `tendermint::vote::SignVoteRequest`
304
   = note: required by `try_from`
305

306
error[E0277]: the trait bound `tendermint::proposal::SignProposalRequest: From<tendermint_proto::privval::SignProposalRequest>` is not satisfied
307
  --> src/rpc.rs:57:27
308
   |
309
57 |                 let spr = SignProposalRequest::try_from(spr).map_err(|e| {
310
   |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<tendermint_proto::privval::SignProposalRequest>` is not implemented for `tendermint::proposal::SignProposalRequest`
311
   |
312
   = note: required because of the requirements on the impl of `Into<tendermint::proposal::SignProposalRequest>` for `tendermint_proto::privval::SignProposalRequest`
313
   = note: required because of the requirements on the impl of `TryFrom<tendermint_proto::privval::SignProposalRequest>` for `tendermint::proposal::SignProposalRequest`
314
   = note: required by `try_from`
315

316
error[E0277]: the trait bound `tendermint::public_key::PubKeyRequest: From<tendermint_proto::privval::PubKeyRequest>` is not satisfied
317
  --> src/rpc.rs:67:27
318
   |
319
67 |                 let pkr = PubKeyRequest::try_from(pkr).map_err(|e| {
320
   |                           ^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<tendermint_proto::privval::PubKeyRequest>` is not implemented for `tendermint::public_key::PubKeyRequest`
321
   |
322
   = note: required because of the requirements on the impl of `Into<tendermint::public_key::PubKeyRequest>` for `tendermint_proto::privval::PubKeyRequest`
323
   = note: required because of the requirements on the impl of `TryFrom<tendermint_proto::privval::PubKeyRequest>` for `tendermint::public_key::PubKeyRequest`
324
   = note: required by `try_from`
325

326
error[E0277]: the trait bound `tendermint_proto::privval::SignedVoteResponse: From<tendermint::vote::SignedVoteResponse>` is not satisfied
327
   --> src/rpc.rs:162:67
328
    |
329
162 |             Response::SignedVote(resp) => Sum::SignedVoteResponse(resp.into()),
330
    |                                                                   ^^^^^^^^^^^ the trait `From<tendermint::vote::SignedVoteResponse>` is not implemented for `tendermint_proto::privval::SignedVoteResponse`
331
    |
332
    = help: the following implementations were found:
333
              <tendermint_proto::privval::SignedVoteResponse as From<tendermint::vote::sign_vote::SignedVoteResponse>>
334
    = note: required because of the requirements on the impl of `Into<tendermint_proto::privval::SignedVoteResponse>` for `tendermint::vote::SignedVoteResponse`
335

336
error[E0277]: the trait bound `tendermint_proto::privval::SignedProposalResponse: From<tendermint::proposal::SignedProposalResponse>` is not satisfied
337
   --> src/rpc.rs:163:75
338
    |
339
163 |             Response::SignedProposal(resp) => Sum::SignedProposalResponse(resp.into()),
340
    |                                                                           ^^^^^^^^^^^ the trait `From<tendermint::proposal::SignedProposalResponse>` is not implemented for `tendermint_proto::privval::SignedProposalResponse`
341
    |
342
    = help: the following implementations were found:
343
              <tendermint_proto::privval::SignedProposalResponse as From<tendermint::proposal::sign_proposal::SignedProposalResponse>>
344
    = note: required because of the requirements on the impl of `Into<tendermint_proto::privval::SignedProposalResponse>` for `tendermint::proposal::SignedProposalResponse`
345

Problem: fixed log level in Nitro provider

e.g. https://github.com/crypto-com/tmkms-light/blob/main/providers/nitro/nitro-enclave/src/main.rs#L29
it may be good in the enclave and the helper to be able to provide a -verbose argument that would switch from info to debug or something like that

Problem: yanked and unsound dependencies

Crate:         nix
118
Version:       0.19.1
119
Warning:       unsound
120
Title:         Out-of-bounds write in nix::unistd::getgrouplist
121
Date:          2021-09-27
122
ID:            RUSTSEC-2021-0119
123
URL:           https://rustsec.org/advisories/RUSTSEC-2021-0119
124
Dependency tree: 
125
nix 0.19.1
126

127
Crate:         nix
128
Version:       0.22.0
129
Warning:       unsound
130
Title:         Out-of-bounds write in nix::unistd::getgrouplist
131
Date:          2021-09-27
132
ID:            RUSTSEC-2021-0119
133
URL:           https://rustsec.org/advisories/RUSTSEC-2021-0119
134
Dependency tree: 
135
nix 0.22.0
136

137
Crate:         zeroize_derive
138
Version:       1.1.0
139
Warning:       yanked

Problem: custom Nitro Enclave launcher and EIF

there is now https://github.com/fortanix/rust-sgx/tree/master/fortanix-vme
so it may be perhaps possibly to unify the SGX and Nitro enclave code better

Problem: fixed log level in SGX provider

https://github.com/crypto-com/tmkms-light/blob/main/providers/sgx/sgx-app/src/main.rs#L7
https://github.com/crypto-com/tmkms-light/blob/main/providers/sgx/sgx-runner/src/main.rs#L74

It may be good to be able to provide a e.g. -verbose argument that would switch from "info" to "debug" in both the enclave and the runner

Problem: log server sometimes does not work

log:

Sep 16 02:22:29 ... tmkms[21550]: Sep 16 02:22:29.810 DEBUG tmkms_light::utils: writing u16-sized payload
Sep 16 02:22:29 ... tmkms[21550]: Sep 16 02:22:29.811 DEBUG tmkms_light::utils: successfully wrote u16-sized payload
Sep 16 02:22:29 ... tmkms[21550]: Sep 16 02:22:29.811 DEBUG tmkms_nitro_helper::command: ...: Creating a proxy unix:///......
Sep 16 02:22:29 ... tmkms[21550]: Sep 16 02:22:29.811  INFO tmkms_nitro_helper::state: listening for enclave persistence
Sep 16 02:22:29 ... tmkms[21550]: Sep 16 02:22:29.811  INFO tmkms_nitro_helper::enclave_log_server: binding enclave log server to vsock port: 5556
Sep 16 02:22:29 ...tmkms[21550]: Sep 16 02:22:29.812  INFO tmkms_nitro_helper::enclave_log_server: Bound enclave log server to Vsock(cid: 3 port: 5556)
Sep 16 02:22:29 ... tmkms[21550]: Sep 16 02:22:29.812  INFO tmkms_nitro_helper::proxy: binding proxy to vsock port: 5000
Sep 16 02:22:29 ... tmkms[21550]: Sep 16 02:22:29.812  INFO tmkms_nitro_helper::proxy: Bound to Vsock(cid: 3 port: 5000

Problem: anomaly is deprecated

anomaly is retired and will receive no further updates.

We recommend transitioning to one or more of the following alternatives:

• anyhow
• eyre
• thiserror

Problem: the process is not shutdown gracefully

signal capture and shutdown gracefully

Problem: upgrade to tendermint-* 0.22.0 fails

https://github.com/crypto-com/tmkms-light/pull/177/checks?check_run_id=3725412334#step:5:572

Problem: upgrade to nix 0.20 fails

error[E0308]: mismatched types
432
   --> providers/nitro/nitro-helper/src/command.rs:141:54
433
    |
434
141 |         let mut socket = vsock::VsockStream::connect(&addr).map_err(|e| {
435
    |                                                      ^^^^^ expected enum `nix::sys::socket::addr::SockAddr`, found enum `SockAddr`
436
    |
437
    = note: expected reference `&nix::sys::socket::addr::SockAddr`
438
               found reference `&SockAddr`
439
    = note: perhaps two different versions of crate `nix` are being used?
440

441
error[E0308]: mismatched types
442
  --> providers/nitro/nitro-helper/src/proxy.rs:34:44
443
   |
444
34 |         let listener = VsockListener::bind(&sockaddr)
445
   |                                            ^^^^^^^^^ expected enum `nix::sys::socket::addr::SockAddr`, found enum `SockAddr`
446
   |
447
   = note: expected reference `&nix::sys::socket::addr::SockAddr`
448
              found reference `&SockAddr`
449
   = note: perhaps two different versions of crate `nix` are being used?
450

451
error[E0308]: mismatched types
452
  --> providers/nitro/nitro-helper/src/state.rs:57:50
453
   |
454
57 |         let vsock_listener = VsockListener::bind(&sockaddr).map_err(|e| {
455
   |                                                  ^^^^^^^^^ expected enum `nix::sys::socket::addr::SockAddr`, found enum `SockAddr`
456
   |
457
   = note: expected reference `&nix::sys::socket::addr::SockAddr`
458
              found reference `&SockAddr`
459
   = note: perhaps two different versions of crate `nix` are being used?
460

461
error: aborting due to 3 previous errors
462

https://github.com/crypto-com/tmkms-light/pull/20/checks?check_run_id=2119554777#step:5:431

Problem: no proof that a keypair was generated inside NE

Failed state connection

Describe the bug
enclave + helper can't communicate on initial state file from host.

To Reproduce
Steps to reproduce the behavior:

1. I mostly used the steps here https://crypto.org/docs/getting-started/advanced-tmkms-integration.html#setting-up-aws-nitro-enclaves-tendermint-kms-for-signing-blocks
2. ./run.sh

Expected behavior
A working state sync + attempts to reach validator address.

Desktop (please complete the following information):

• amazonlinux

Additional context
enclave output

...
[    0.132043] Write protecting the kernel read-only data: 14336k
[    0.133920] Freeing unused kernel memory: 2016K
[    0.135182] Freeing unused kernel memory: 476K
[    0.135827] nsm: loading out-of-tree module taints kernel.
[    0.136271] nsm: module verification failed: signature and/or required key missing - tainting kernel
May 20 15:55:09.589  INFO tmkms_nitro_enclave: waiting for config to be pushed on cid: 4294967295 port: 5050
[    1.218763] random: crng init done
May 20 15:55:10.666  INFO tmkms_nitro_enclave: got connection on Vsock(cid: 4294967295 port: 5050)
Connected.
Disconnected.
Connected.
Disconnected.
May 20 15:55:11.412 ERROR tmkms_nitro_enclave: io error I/O error: failed get state connection

helper output

[ec2-user@ip-10-0-0-172 ~]$ ./run.sh
Start allocating memory...
Started enclave with enclave-cid: 62, memory: 8192 MiB, cpu-ids: [1, 3]
{
  "EnclaveID": "i-0c3cd5d3dbff334de-enc1798a7cccefaf24",
  "ProcessID": 22651,
  "EnclaveCID": 62,
  "NumberOfCPUs": 2,
  "CPUIDs": [
    1,
    3
  ],
  "MemoryMiB": 8192
}
Os { code: 2, kind: NotFound, message: "No such file or directory" }

It fails here

tmkms-light/providers/nitro/nitro-enclave/src/nitro/state.rs

Line 23 in b7b9081

let state_conn = vsock::VsockStream::connect(&addr)?;

. I've tried changing both of these settings enclave_config_cid = 61 and enclave_state_port = 5555 but they all produce this same result.

The fact that it made it to this point means that the config pushed to enclave and kms decryption was successful. I can even change the state_file_path to an incorrect value and produce an error saying it was incorrect, which means that when I have it correct it is finding and pulling the initial state correctly.

Problem: nsm crates taken from Git

There's now an official crate: aws-nitro-enclaves-nsm-api
so perhaps better to switch to that

Problem: Nitro initial key generation done on the host

As mentioned in the comment, it can be done in the enclave: https://github.com/crypto-com/tmkms-light/blob/main/providers/nitro/nitro-helper/src/key_utils.rs#L11

Problem: missing documentation for Nitro Enclave

Problem: clippy and fmt issues

https://github.com/crypto-com/tmkms-light/runs/2855299679

error: this function has too many arguments (8/7)
502
   --> providers/sgx/sgx-runner/src/command.rs:261:1
503
    |
504
261 | / pub fn recover(
505
262 | |     config_path: Option<PathBuf>,
506
263 | |     pubkey_display: Option<PubkeyDisplay>,
507
264 | |     bech32_prefix: Option<String>,
508
...   |
509
269 | |     log_level: String,
510
270 | | ) -> Result<(), String> {
511
    | |_______________________^
512
    |
513
    = note: `-D clippy::too-many-arguments` implied by `-D warnings`
514
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#too_many_arguments
515

516
error: aborting due to previous error

Diff in /home/runner/work/tmkms-light/tmkms-light/providers/sgx/sgx-app/src/main.rs at line 6:
11
     let mut args = std::env::args();
12
     let command = args.next();
13
     let log_level = match args.next() {
14
-        Some(s) if s == "verbose" => {
15
-            tracing::Level::DEBUG
16
-        },
17
+        Some(s) if s == "verbose" => tracing::Level::DEBUG,
18
         _ => tracing::Level::INFO,
19
     };
20
     let subscriber = tracing_subscriber::FmtSubscriber::builder()
21
Error: The process '/usr/share/rust/.cargo/bin/cargo' failed with exit code 1

will be good to resolve #97 first

Problem: tmkms-light-sgx-runner fails to compile

https://github.com/crypto-com/tmkms-light/runs/2854780929#step:5:8

error[E0277]: the trait bound `SubCommand: FromStr` is not satisfied
420
  --> providers/sgx/sgx-runner/src/main.rs:23:5
421
   |
422
23 |     subcommand: SubCommand,
423
   |     ^^^^^^^^^^^^^^^^^^^^^^ the trait `FromStr` is not implemented for `SubCommand`
424
   |
425
   = note: required by `std::str::FromStr::from_str`
426

427
error[E0599]: no associated item named `Debug` found for struct `Level` in the current scope
428
  --> providers/sgx/sgx-runner/src/main.rs:84:21
429
   |
430
84 |         2 => Level::Debug,
431
   |                     ^^^^^ associated item not found in `Level`
432

433
error[E0223]: ambiguous associated type
434
  --> providers/sgx/sgx-runner/src/main.rs:92:9
435
   |
436
92 |         TmkmsLight::CloudWrapKeyGen {
437
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use fully-qualified syntax: `<TmkmsLight as Trait>::CloudWrapKeyGen`
438

439
error[E0223]: ambiguous associated type
440
   --> providers/sgx/sgx-runner/src/main.rs:103:9
441
    |
442
103 |         TmkmsLight::Init {
443
    |         ^^^^^^^^^^^^^^^^ help: use fully-qualified syntax: `<TmkmsLight as Trait>::Init`
444

445
error[E0223]: ambiguous associated type
446
   --> providers/sgx/sgx-runner/src/main.rs:119:9
447
    |
448
119 |         TmkmsLight::Start { config_path } => command::start(config_path, log_level_str),
449
    |         ^^^^^^^^^^^^^^^^^ help: use fully-qualified syntax: `<TmkmsLight as Trait>::Start`
450

451
error[E0223]: ambiguous associated type
452
   --> providers/sgx/sgx-runner/src/main.rs:120:9
453
    |
454
120 |         TmkmsLight::Recover {
455
    |         ^^^^^^^^^^^^^^^^^^^ help: use fully-qualified syntax: `<TmkmsLight as Trait>::Recover`
456

457
error: aborting due to 6 previous errors
458

459
Some errors have detailed explanations: E0223, E0277, E0599.
460
For more information about an error, try `rustc --explain E0223`.
461
error: could not compile `tmkms-light-sgx-runner`
462

463
To learn more, run the command again with --verbose.
464
Error: Process completed with exit code 101.

Problem: SGX compilation instructions may not be using HW cryptographic acceleration

Right now, the enclave app is compiled with plain cargo build.
For example, the aes crate uses the cpufeatures crate to detect whether to use AES-NI -- for SGX, that would only be done if all the target features are enabled: https://github.com/RustCrypto/utils/blob/master/cpufeatures/src/x86.rs#L10
which may not always be the case -- so perhaps instructions should explicitly say to put it in the cargo config or RUSTFLAGS="-C target-feature=+aes " (... and all the other needed target-feature ...)

Problem: missing workflow to build and publish tmkms-nitro-enclave

Problem: extra ? operator is accidentally added

https://github.com/crypto-com/tmkms-light/runs/2396137140?check_suite_focus=true

Problem: incorrect character set for external cloud payload in SGX

Problem: yanked dependency

error: 1 denied warning found!
Crate:         crossbeam-epoch
Version:       0.9.4
Warning:       yanked
Dependency tree: 
crossbeam-epoch 0.9.4