openenclave / openenclave Goto Github PK

SDK for developing enclaves

License: MIT License

C 62.69% Assembly 0.49% C++ 17.84% Shell 2.05% CMake 10.74% Python 1.14% GDB 0.18% Pawn 0.16% PowerShell 1.21% C# 0.63% TypeScript 1.03% JavaScript 0.02% Jinja 0.02% Dockerfile 0.09% Groovy 1.69%

openenclave's Introduction

Open Enclave SDK

Integration Partners

Agnostic Cloud Provider

Azure

Introduction

The Open Enclave SDK is a hardware-agnostic open source library for developing applications that utilize Hardware-based Trusted Execution Environments, also known as Enclaves.

Open Enclave (OE) is an SDK for building enclave applications in C and C++. An enclave application partitions itself into two components:

An untrusted component (called the host) and
A trusted component (called the enclave).

An enclave is a protected memory region that provides confidentiality for data and code execution. It is an instance of a Trusted Execution Environment (TEE) which is usually secured by hardware, for example, Intel Software Guard Extensions (SGX).

This SDK aims to generalize the development of enclave applications across TEEs from different hardware vendors. The current implementation provides support for Intel SGX as well as preview support for OP-TEE OS on ARM TrustZone. As an open source project, this SDK also strives to provide a transparent solution that is agnostic to specific vendors, service providers and choice of operating systems.

Getting Started Using OE SDK

You'll find comprehensive documentation in the Getting Started Guide.

Contributing to OE SDK

The community documentation hosts lots of information on where to go to get engaged with the community, whether you want to contribute code, add test cases, help improve our documentation, or something else. If you're looking for information on how to join meetings or who to contact about what, you will find it there.

You don't necessarily need a hardware enclave to develop OE SDK; some tests and code paths can be executed in simulation mode for the purposes of testing on non-TEE-enabled hardware.

Licensing

This project is released under the MIT License.

Send Feedback

Send general questions, announcements, and discussion to the [email protected] Mailing List.

To report a problem or suggest a new feature, file a GitHub issue.

To report a security issue, please follow the process to report a vulnerability.

openenclave's People

Contributors

Stargazers

Watchers

Forkers

dthaler nikileshsa kernullist parikshit-hooda maneeshs shyamalschandra y1026 kant jumaffre novikov-alexander-zz oprinmarius ahapliuc ionutbalutoiu ptcrews britel mofanv icaro-lima lzha101 crisl-ms vishnumohan1991 rschust davidleeh mjp41 aserdean stefanlupsa jiria l1212s slavymihov gitter-badger dayeol koprowskit gafachilabs sridharan8888 andyleejordan makroeze-msft cupertinodude wintersteiger 9176324 fengjixuchui jordanrh1 hilvert ruide mariom3 pwarias pkulach2 tornadory kkx jackspice emilalexandrustoica rgerganov bfjelds ajaybrahmakshatriya george-caluta letmaik secopsgit kushg02 l1212s-tee desaijay315 eozturk1 bigdata-memory jovanbulck dio qwerwon donaldww qinxgit doneladams dut3062796s hhy5277 stevezhou6 deeglaze jhand2 wangke0809 olgavrou lsds pyliaorachel nburnettt gupta-ak ichetty raccoonrat tehsiuhuang xnetrunnerx yentsanglee ypjiang2020 limorl tcollavo nonpolarity frenchypeanut thomasten nunofernandes-plight ryanhsu19 jiahaoc1993 brmclaren retzger mingweishih yufjiang mirekphd mfriedl podcastinator daperoni hernangatta

openenclave's Issues

Port libc and libcxx test suites to OpenEnclave for enclave tests

This initial work also sets up the basic framework for a vendor to come in and port remaining test suites into OpenEnclave.

Implement GDB debugger plugin for Linux

Provide a debugging support for GDB on Linux for parity with the Intel SDK.

OE-host: Find/confirm the right mechansim to expose OE_EXPORTed types on Windows

Subitem of #35

Document Windows enclave debugging instructions

Follow up to #37, we need to provide clear guidance an instructions for debugging enclaves on Windows.

Core Testing: Expand testing of memory management support in enclaves

We need to flush out the details for behaviors of the enclave APIs an validate them. This applies to the updated memory model within the enclave:

Review test coverage for allocation methods provided by the enclave runtime (PR #654, PR #655)
Validate attempts to alloc/free across the enclave boundary (PR #654)
Verify memory alloc/free is thread-safe (PR #654)
Performance tuning of allocation granularity within enclave (See #25 (comment))
Memory teardown semantics in enclave when host process is abruptly terminated (Moved to #589)

App devs can build and run a host app on Windows that loads ELF enclave binaries

Requires #34.
To enable OpenEnclave to run on Windows, we will need the host library to be built as a Windows library that can be incorporated into the Windows SDK toolchains and system dependencies. This includes converting the C runtime dependencies as well as any enclave call-outs to system to use Windows equivalents.

Support setting of enclave metadata in code via named data segment macro

The current MS SDK provide configuration of some enclave parameters via ENCLAVE_USE_POLICY() in the source code - creating a specific data entry evaluated by the signing tool. Could OE have something similar?

Verify external downloads

Reconsider download of external, unverified sources: We cannot build using external downloads without verifying them for integrity

Example: openenclave/3rdparty/dlmalloc/

Convert OpenEnclave to use CMake for multi-platform build project generation

OpenEnclave currently relies on Makefiles exclusively, which do not work cross-platform for Windows. To allow the host libraries and tools to be built on Windows, we will migrate to CMake for cross-platform build project generation.

Maintainers can rely on a CI system to enforce build and test checks for PRs for Windows

We need to onboard a Windows as a default CI configuration in Jenkins to catch build regressions, and specifically regressions in the ELF enclaves on Windows scenario:

#993 Enable Windows CI Builds
#1011 Test ELF enclaves on Windows in CI
#1002 Add a test-build-config batch file for Windows builds
Support Windows enclaves on DCAP in CI (resolved by PR #1417)

The previous investigation on this has been closed in favor of converging on the same CI Jenkins infrastructure as used for the Linux configurations:

Need to set up a continuous integration pipelines for Windows, which requires extra investigation as the existing solution for Linux (#50) does not converge for Windows.

AppVeyor is one solution that other Azure teams have used and provide CI that is publicly visible to OSS contributors as well.

If we can get a single solution, this can be merged with the Linux one #50.

Publish OSS contribution policy for OpenEnclave project

We should have this published to have common standards for contribution and CLA established early on.

Integrate new Intel AESM/driver dependencies

OpenEnclave needs to switch its existing dependencies on the AESM and isgx driver over to the new set of architectural enclaves and FLC-aware driver.

Define generic OpenEnclave identity structures for use with sealing and attestation APIs

As an extension of the generic nature of the OpenEnclave API surface, we need to provide OE_* abstractions for data structures involved in the identity model used for sealing policies as well as attestation checks.

Double-check/CR proper integer type usage in OE code

GCC & Windows VSC have different interpretations of "long" in 64-bit mode. include/openenclave/defs.h had a static assert of "long" being 64-bit, making it plausible code used plain "long"-derived types (instead of uint64_t/int64_t) assuming 64-bit width.
Do a CR to replace thus usages.

Subitem of #35

Windows: Implement mmap and virtual memory management model in enclaves

For convergence with Windows, we would like to implement the mmap family of memory management functions for the enclave runtime that has a better parity with the Windows VirtualAlloc family of methods. This would also enable us to switch over to the MUSL malloc implementation instead of the separate dlmalloc dependency we currently pull in.

As part of this work, we should also implement the virtual address descriptor model so that we have consistent enclave internal memory management semantics.

The current driver for this change will likely be EDMM support in the OE SDK, so leaving this on the backlog.

Maintainers can rely on a CI system to enforce build and test checks for PRs for Linux

Need to set up a continuous integration pipelines for Linux.

To explicitly scope the goals for this story, most of the CI systems explored only support hosted builds by default or in their basic/free plans, which precludes running on Azure SGX machines. As such, this story primarily covers enforcing basic engineering checks during PR submission:

Enforcing use of validation scripts for license checking and code formatting
Building all flavors (debug, release, relwithdebinfo)
Building against different architectural versions (legacy AESM libraries, NGSA libraries for coffeelake)
Running all tests supported in simulation mode

For enabling CI to run on Azure SGX machines, see #374.
If we can get a single solution, this can be merged with the Windows one #51.

App devs can obtain and use Open Enclave SDK on Ubuntu via apt-get

To ease adoption of the SDK, we should provide a binary and library drop that does not require non-contributors to build the OpenEnclave project. For Ubuntu, this would be a .deb package published by Microsoft repository with an appropriate hash value documented for it.

The current plan is to release RelWithDebInfo builds of the SDK, and the Open Enclave CMake project is already configured to generate .deb packages via:

build$ cmake -DCMAKE_INSTALL_PREFIX:PATH=/opt/openenclave ..
build$ cpack -G DEB

This story depends on:

#301 Update Open Enclave SDK package configuration

This is related to #373 tracking publishing of .rpm packages for RHEL.

App devs on Linux can attach oe-gdb to a running process

OE SGX GDB should be able to attach to a running process, discover existing enclaves in that process, and debug these enclaves.

This story should also have additional tests that should be added along with #95

Refactor public OpenEnclave API surface

In preparation for public preview, we will need to provide a specific API surface for public consumption. The refactor will enforce internal and public separation of use for the existing API surface, as well as break runtime headers into functional groupings for clarity of documentation.

Publish and enforce engineering guidelines for OpenEnclave contribution

This is needed to establish a consistent engineering baseline that the maintainers adhere to upfront to avoid double standards on accepting contributions. We would like this to be as lightweight and automated as possible, including the application of tools such as clang.format to enforce mechanical issues such as C/C++ code style.

Core Testing: Expand testing of enclave management scenarios

We need to flush out the details for behaviors of the enclave APIs an validate them. This includes:

Repeated enclave creation and termination (#359 )
Understandable errors returned on failed enclave creation at boundary conditions (#444 / #360 )
Enclave creation from a call out to host (#431)
Enclave behavior on abort() and unexpected exception (#432 )
~~Teardown semantics in enclave when host process is abruptly terminated~~ (Tracking as additional work item dependent on Intel driver update, see #589)
Teardown semantics when enclave thread is abruptly terminated (#432)

App devs can obtain and use Open Enclave SDK on Windows via packaged install

To ease adoption of the SDK, we should provide a binary and library drop that does not require non-contributors to build the OpenEnclave project.

Define OE SDK package for Windows (nuget?)
Document OE SDK install and usage instructions for Windows (a la install_oe_sdk.md)

This is related to story #371

Add VerifyReport API

Requires #41, #68, #72
For symmetry, since we are not exposing a GetReportingKey API, OpenEnclave needs to offer an abstraction for report verification that support arbitrary local attestation scenarios.

Note that this requires the use of crypto libraries and would require us to design a reverse dependency injection mechanism so that an crypto lib of the developer's choice can be plugged into the enclave runtime (if they choose not to use the default mbedTLS).

Document Windows dependencies and instructions for developing on Windows

Requires #35.
We need to update the getting started docs and readmes we already have for Linux to support Windows developers and minimize their friction in adopting the SDK.

App devs can expect Open Enclave unit tests as ELF on Windows to pass like Linux

We need to ensure that the existing test suites pass for the equivalent implementation on Windows. This includes additional porting efforts as necessary for the tests hosts to run on Windows where appropriate.

#1008 Resize enclave tests to support Windows CI
#1033 Re-enable ELF enclave on Windows tests
Enable crypto tests on Windows (part of #375)
#1034 Support libunwind and libc tests for ELF enclaves on Windows
- Should encompass libcxx, libcxxrt, libc and libunwind test failures.

Remaining tests disabled on Windows:

tests/memory (requires investigation after #632)
tests/create-rapid (requires investigation after #632)
tests/ecall_ocall (build failure on clangw on Windows)
tests/tls_e2e (depends on socket support as part of PR #1960)

Support exception handling in enclaves

We need to design out and implement exception handling support in enclaves for both Linux and Windows. This does not include supporting Structured Exception Handling for Windows.

Define crypto abstraction interface for host and enclave runtime libs

On the host side, this will be necessary for allowing the host code to work across OS and the FIPS certified crypto implementations they ship with (BCrypt on Windows and OpenSSL in Ubuntu). Enclave hashing is already needed, and additional support will be needed for quote verification.

On the enclave side, this will be necessary for allowing a developer to plug in their choice of crypto lib instead of mbedTLS. This will need to support hashing, report/quote verification. In the future, this will also need to support seal/unseal.

SHA hashing methods for enclave hashing
AES-CMAC signature validation methods for report verification
ECDSA signature validation methods for quote verification
X509/ASN certificate chain methods for quote verification

Build-system: Consider out-of-source-tree build

Many OSS projects switched to out-of-source tree builds for the benefits of a cleaner separation.

Consider this for OE as well, perhaps utilizing cmake.

App devs can debug a Windows app hosting an ELF enclave built with Open Enclave

Requires #35.
Once we have a working OpenEnclave stack on Windows, we need to implement a custom debugging solution for it as enclaves will be ELF binaries hosted in a Windows PE process, and the existing Windows debuggers do not handle ELF symbol parsing, nor understand the stack stitching needed for the enclave stack.

This is currently being pursued with the Windbg team to add support in windbg for stack-stitched debugging with OE ELF enclaves.

Windbg supports debugging ELF enclaves on Windows (External)
#38 Document Windows debugging instructions

Core Testing: Expand tests for thread binding behavior for enclaves

We need to flush out the details for behaviors of the enclave APIs and validate them. This includes thread binding behavior and its interactions in multi-threaded scenarios:

~~Thread binding correctly avoids deadlocks from thread exhaustion in the enclave~~
- Thread binding now fails if we exhaust all enclave threads, we should simply verify that host handles this failure gracefully. (Addressed by PR #1023 )
- Validate implementations of all OE synchronization primitives (Covered with #476 )
- Stress and load tests for multi-threaded enclave applications (Missing coverage provided with #612 )

Update dependency on Intel AESM to Intel SGX lib for OpenEnclave

OpenEnclave currently depends on the existing Intel AESM with dependencies on Intel signing for launch and EPID for quoting. We are requesting that Intel supports non-Intel specific versions of these architectural enclaves packaged as a dependency library for the OpenEnclave project so that it can be consumed by the OSS community without business relationships to Intel. This includes:

Using the Intel upstream kernel driver and Launch Enclave implementation that does not require Intel signing
Using the Intel ECDSA Quoting Enclave that relies on standard PKI and not EPID
Abstracting away the differences between the Intel drivers for enclave management functionality
Enabling a cross-platform dependency with Windows

Replace GetAttestationReportForRemoteAttestation with generic GetReport

The current implementation for obtaining a report only supports its use for remote attestation via the Intel AESM quoting enclave. We need to make the method generic for local attestation use by accepting explicit and targetinfo.

Update oesign tool to support digest signing with Windows signtool

Until #44 is complete, production testing of SGX enclaves requires Intel signing. For the Windows developers, supporting digest generation will allow integration with signtool as part of the Windows development tool chain, which supports existing workflows for signing via HSM through BCrypt.

Port mbedTLS test suite to OpenEnclave for enclave tests

Using the work done in #17 as a model, port the mbedTLS test suite to OpenEnclave and ensure all supported tests pass.

Remove static OpenSSL dependencies from host

As a scoping exercise, we will support and test mbedTLS for use in enclaves but not OpenSSL. As such we should remove the existing host dependencies on OpenSSL and the OpenSSL 3rdparty build project entirely.

Scrub all registers on enclave exit

Istvan Haller: Scrub all registers on enclave exit, including extended state.
This must include the SSE/AVX registers. For details, see xsave/xrstore/vzeroall.

Support SGX quote verification within an enclave

This is a Coco requirement.

The existing Intel ECDSA-based quote creation and verification only happens on the host side as part of #44. Porting the extra crypto support and methods to be able to run the verification within an enclave is extra work that has a dependency on #68.

Expand testing of CallHost & CallEnclave scenarios

We need to flush out the details for behaviors of the enclave APIs an validate them. Additional cases include:

Calling in and out of an enclave recursively on the boundary
Attempting to invoke enclave methods not marked for export
Handling reentrancy in host and enclave
Global initializer functions that call out

Port libunwind test suite to OpenEnclave for enclave tests

Using the work done in #17 as a model, port the libunwind test suite to OpenEnclave and ensure all supported tests pass.

Doc: Specify supported/unsupported libc/libc++ features

Documentation: We should specify which libc/libc++ functions/features are supported by the OE runtime, and which are absent from the standard.
We should clarify differences & explicitly mention which of the functions/features may result in OCALLS.

Remove dead code before public preview

Tracking issue to accumulate known cleanup prior to public preview:

Document modifications to libunwind and refactor to facilitate auditing

Follow-up to #4.

This issue tracks all functional edits to libunwind code for Open Enclave which are done directly in the 3rdparty sources (unlike the libcxx/libc where our changes kept in separate source files. We need to:

Refactor the edits out of the cloned source files where possible to make our changes easy to audit and to apply to updated versions of the libunwind library we clone.
- This should include eliminating unchanged sources that are not built in this project (e.g. the renamed aux-aux folder from #34)
Document any remaining changes that could not be refactored out for transparency, for example, including diffs where the changes cannot be separated from existing source files.

One solution to investigate:

In the CMake process, libunwind needs changes against the cloned sources anyway to allow the solution to be compatible with Windows file system (renaming or removing the aux folder)
In the other 3rdparty projects, we already implement an update.make script which automates cloning the target version to the Open Enclave repo. Implementing such a script for libunwind makes sense, as it could automate the aux folder handling as well.
As part of this script, it might also be possible to add logic to patch the sources based on a git patch (see commits below) or other build patch format. This would complete the automation by allowing us to have a script that applies our changes directly on top of newly downloaded sources, and do the appropriate merge conflict resolutions at clone time rather than build time as with the other 3rdparty projects.

The diffs currently tracked for libunwind, excluding changes to the build infrastructure include:

244efe2: Initial commit of libunwind library based on vanilla_pathscale branch in https://github.com/pathscale/libunwind with the following modifications:
- Checked in files generated by autogen.sh script, these should be removed:

3rdparty/libunwind/libunwind/include/libunwind.h
3rdparty/libunwind/libunwind/include/tdep/libunwind_i.h
3rdparty/libunwind/libunwind/src/coredump/libunwind-coredump.pc 
3rdparty/libunwind/libunwind/src/ptrace/libunwind-ptrace.pc
3rdparty/libunwind/libunwind/src/setjmp/libunwind-setjmp.pc
3rdparty/libunwind/libunwind/src/unwind/libunwind.pc

#ifdef/ifndef OPENENCLAVE to selectively compile some parts of these sources:

3rdparty/libunwind/libunwind/src/dwarf/Gfind_proc_info-lsb.c
3rdparty/libunwind/libunwind/src/elfxx.h
3rdparty/libunwind/libunwind/src/os-linux.c
3rdparty/libunwind/libunwind/src/os-linux.h
3rdparty/libunwind/libunwind/src/x86_64/Gglobal.c
3rdparty/libunwind/libunwind/src/x86_64/Ginit.c
3rdparty/libunwind/libunwind/src/x86_64/Gtrace.c
3rdparty/libunwind/libunwind/src/x86_64/setcontext.S

1aefd81: Removed *_u forms for pthread functions in our 3rdparty/libunwind/stubs.h:

#define pthread_mutex_lock pthread_mutex_lock_u
#define pthread_mutex_unlock pthread_mutex_unlock_u

be71a48: Added unmodified libunwind headers missed in 244efe2
3fd3739: Renamed aux folder to aux-aux as part of shift to CMake build so as to support Windows (aux is a reserved name in the Windows file system).
30d89a6: Disabled use of adaptive mutexes in libunwind with change to 3rdparty/libunwind/libunwind/include/libunwind_i.h:

/* Disable use of adaptive mutexes, which are defined by GCC headers but not
 * supported by MUSL pthreads. Note that libunwind is compiled with GCC headers 
 * but linked with MUSL libc. 
 */
#ifdef PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP
#undef PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP
#endif

2c0e713: Changes to libunwind to block stack unwinding across enclave boundaries during C++ exception handling in the following files:

3rdparty/libunwind/libunwind/src/mi/init.c
3rdparty/libunwind/libunwind/src/unwind/RaiseException.c
3rdparty/libunwind/libunwind/src/unwind/unwind-internal.h

Double-check all host-pointers are outside of the enclave

To prevent corruption by a malicious or buggy host, all pointers handed from the host need to be verified to be outside enclave memory and sanitized.
Double-check all pointers handed from the host are checked as of today.
Create design/coding principles to help developers handling safe host data exchange.

App devs get deprecation warnings for unsupported enclave c/c++ library functions

Add a deprecations header for all unsupported enclave library functions (libc, libcxx etc.) Attempts to link will already fail from missing symbols today, but this would allow the unsupported methods to be made explicit at compile time. The proposal is as follows:

Move OE_DEPRECATED(MSG) definition out of defs.h
Put all deprecations in deprecations.h (under openenclave/include/libc/bits)
Append the following line to all headers under openenclave/include/libc: #include <bits/deprecations.h>
Provide a macro called OELIBC_DISABLE_DEPRECATIONS so developers can disable this mechanism.
Provide per-function macros so developer can selectively disable this mechanism by function.

Here’s are sample contents of deprecations.h:

 #ifndef OELIBC_DEPRECATED
 #define OELIBC_DEPRECATED(MSG) __attribute__((deprecated(MSG)))
 #endif

 #ifndef OELIBC_DISABLE_DEPRECATIONS
 #ifndef _OELIBC_DEPRECATED_H
 #define _OELIBC_DEPRECATED_H

 #include <bits/alltypes.h>

 #ifndef OELIBC_ALLOW_PUTCHAR
 OELIBC_DEPRECATED(“some message”) int putchar(int c);
 #endif

 #endif /* _OELIBC_DEPRECATED_H */
 #endif /* OELIBC_DISABLE_DEPRECATIONS */

The user can disable this mechanism with the OELIBC_DISABLE_DEPRECATIONS macro or he can disable one or more functions with the corresponding macro (OELIBC_ALLOW_PUTCHAR).

The proposed mechanism can be ifdef to work with Windows as well via __declspec(deprecated(MSG)) as long as the developer is compiling with /W3 or better.

Part of this story also includes verifying that we also deprecate unsafe CRT methods in favor of their safe equivalents as mentioned in #215

Enclave Initialization: Call global/static C++ constructors upon init

For proper C++ execution, global/static C++ constructors must be called.

Add GetSealingKey API

To provide support for sealing scenarios using SGX, we need to provide an enclave method to access sealing keys.

App devs have a clear understanding of simulator mode support in Open Enclave

Document existing level of support for simulator mode and ensure that unsupported functions have a consistent graceful degradation strategy (fail fast or no-op success). We should consider removing support and the Intel dependency entirely if the value of this outweighs the maintenance cost as features are added.

This story requires extra investigation into effort involved for documenting limits of support and cost to attain Intel parity support for simulator mode.

Supported – tests that work with no or with minor modifications.
Unsupported – tests that should not work because they rely on unsupported enclave features such as:
- Thread creation
- File I/O
- Signal handling (revisit after #21)
- Floating point exceptions (all C math tests, revisit after #21)
Broken – tests that are currently broken and should be:
- Fixed and moved to the supported group
- Obsoleted and moved to unsupported

Currently:

97% of C++ tests pass (4218 supported, 779 unsupported, 143 broken)
80% of C tests pass (50 supported, 295 unsupported, 11 broken)

This issue tracks resolving the broken tests in this set.