gramineproject / gsc Goto Github PK
View Code? Open in Web Editor NEWGramine Shielded Containers (Docker integration)
License: BSD 3-Clause "New" or "Revised" License
Gramine Shielded Containers (Docker integration)
License: BSD 3-Clause "New" or "Revised" License
The first version of Graphene Shielded Containers (PR gramineproject/graphene#1430) is limited in scope and we've frozen the feature list. This issue lists limitations and possible future features, and should serve as a discussion forum for others to highlight which features are a priority.
Security issues:
--insecure-args
during build.Minor features:
build
arguments such as build_args
or network_mode
in the configuration filedocker build parameter
docker_api.build(path='gsc-' + image, tag=gsc_image_name(image), nocache=True)
gsc build
into two commands, gsc build-graphene
and gsc build
.
gsc build-graphene
is the first stage of current gsc build
building a Graphene runtime Docker image. This is independent of the application image and can be generalized and does not require to be rebuild for every image.gsc build
depends on the previous command gsc build-graphene
and copies the runtime over to the application image.gsc build-graphene
on dockerhub for supported Distributions (e.g. graphene:ubuntu18.04-sgx1.9)Major features:
I encountered some errors when building a simple HelloWorld image. It appears that some downloads failed to complete in the build section due to network problems. Should I try to go into the image and change the "source. txt" in it or modify the command in a file called "dockerfile.build" to add a copy file operation?
Is it to support (insecure) arguments?
gsc/templates/apploader.template
Line 14 in 21b30f2
This is an existing on-prem (Intel Nuc) system that has been used for graphene-sgx in the past. Our team is now looking to upgrade to gramine, and for the most part everything appears to be working correcly, however, when we attempt build and test container images, the test fails at runtime.
Any help or pointers are appreciated.
./gsc build -d --insecure-args ubuntu18.04-bash test/ubuntu18.04-bash.manifest
./gsc sign-image ubuntu18.04-bash ~/.config/gramine/enclave-key.pem
docker run --privileged --device=/dev/isgx --device=/dev/gsgx -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-ubuntu18.04-bash -c ls
Files are listed
The following error is printed:
+ find /gramine/meson_build_output/lib -type d -path '*/site-packages'
+ export PYTHONPATH=:/gramine/meson_build_output/lib/python3.6/site-packages
+ PYTHONPATH=:/gramine/meson_build_output/lib/python3.6/site-packages
++ find /gramine/meson_build_output/lib -type d -path '*/pkgconfig'
+ export PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ '[' -z '' ']'
+ gramine-sgx-get-token --sig /entrypoint.sig --output /entrypoint.token
Attributes:
mr_enclave: 208579237095fcc1358557f23e5903c4bca6d9d36b495a319ba787b497b1e0bc
mr_signer: 89139782896bcfc6cdc46924b53fdc83b8886ea681230bc5bb77f584bacaf255
isv_prod_id: 0
isv_svn: 0
attr.flags: 0000000000000004
attr.xfrm: 0000000000000007
mask.flags: ffffffffffffffff
mask.xfrm: fffffffffff9ff1b
misc_select: 00000000
misc_mask: ffffffff
modulus: dd47372296ab46c9351e57564457be48...
exponent: 3
signature: e7327dff8d92bebf22f34a78c864ff4c...
date: 2022-01-06
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-get-token", line 20, in <module>
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/gramine/meson_build_output/bin/gramine-sgx-get-token", line 16, in main
token = get_token(sig, verbose=verbose)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/sgx_get_token.py", line 152, in get_token
token = connect_aesmd(sig['enclave_hash'], sig['modulus'], sig['attribute_flags'], xfrms)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/sgx_get_token.py", line 89, in connect_aesmd
raise Exception(f'Failed. (Error Code = {ret_msg.ret.error})')
Exception: Failed. (Error Code = 6)
SGX is installed, aesmd is running (see output from is-sgx-below) and our old graphene-sgx containers function correctly
is-sgx-available
SGX supported by CPU: true
SGX1 (ECREATE, EENTER, ...): true
SGX2 (EAUG, EACCEPT, EMODPR, ...): false
Flexible Launch Control (IA32_SGXPUBKEYHASH{0..3} MSRs): false
SGX extensions for virtualizers (EINCVIRTCHILD, EDECVIRTCHILD, ESETCONTEXT): false
Extensions for concurrent memory management (ETRACKC, ELDBC, ELDUC, ERDINFO): false
CET enclave attributes support (See Table 37-5 in the SDM): false
Key separation and sharing (KSS) support (CONFIGID, CONFIGSVN, ISVEXTPRODID, ISVFAMILYID report fields): false
Max enclave size (32-bit): 0x80000000
Max enclave size (64-bit): 0x1000000000
EPC size: 0x5d80000
SGX driver loaded: true
AESMD installed: true
SGX PSW/libsgx installed: true
Hi,
I try to run hashgraph in a GSC container. I could run it in successfully in gramine-sgx. Now I would like to run it also in a GSC.
This is my dockerfile:
FROM ubuntu:20.04 AS build-stage
# suppress region selection during dependency installation
ARG DEBIAN_FRONTEND=noninteractive
# install dependencies and build tools
RUN apt-get update && apt-get install --no-install-recommends -qq -y \
cmake \
automake \
bison \
flex \
g++ \
git \
libboost-all-dev \
libevent-dev \
libssl-dev \
libtool \
make \
pkg-config \
openssl \
apt-utils
# copy the source code into the container
COPY examples/hashgraph /hashgraph
# switch to build folder
WORKDIR /hashgraph/build
# build
RUN cmake .. && make
# everything within the export-stage will be exported to the host
FROM scratch AS export-stage
# copy executable from build-state to export-stage
COPY --from=build-stage /hashgraph/build/hashgraph /build/hashgraph
# copy crypto material from build-state to export-stage
COPY --from=build-stage /hashgraph/tests/config /tests/config
# to avoid entrypoint issue
CMD ["bash"]
The docker build finish successfully. As soon as I run the gsc build
it runs into a issue without details. I run the following command:
./gsc build -d --insecure-args hashgraph test/hashgraph.manifest
These are the last outputs from the terminal:
---> 81fe0d2c8ac6
Step 8/22 : RUN apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y binutils libprotobuf-c-dev locales locales-all openssl python3 python3-pip python3-protobuf && python3 -B -m pip install click jinja2 protobuf 'toml>=0.10'
---> Running in 907471e22004
Failed to build unsigned graminized docker image `gsc-hashgraph-unsigned`.
Here is also my manifest content. I took it from the old PyTorch example:
sgx.enclave_size = "16G"
sgx.thread_num = 32
sys.stack.size = "2M"
# Python's memory allocator by default creates a new arena for each thread, and each new arena
# allocates 128MB of memory. By default, PyTorch creates as many threads as there are available CPU
# cores, so this may quickly deplete available enclave memory (e.g. Python eats 2GB on a 16-core
# platform). Let's restrict the number of created threads.
loader.env.OMP_NUM_THREADS = "8"
loader.env.MKL_NUM_THREADS = "8"
Do you have an idea how to get more detailed information to figure out what's going wrong?
Hello,
I am still having a hard time to make the attestation flow working on kubernetes on azure.
Since last time, I am able to deploy my sgx enabled container on azure aks and write to /dev/attestation/user_report_data
unfortunately, I get an error when doing so:
cat: /dev/attestation/quote: Permission denied
error: Cannot connect to aesm_service (tried sgx_aesm_socket_base and /var/run/aesmd/aesm.socket UNIX sockets).
Please check its status! (`service aesmd status` on Ubuntu)
error: Failed to get quote
I check out what is my system status with "is-sgx-available"
and here is what I get:
SGX supported by CPU: true
SGX1 (ECREATE, EENTER, ...): true
SGX2 (EAUG, EACCEPT, EMODPR, ...): true
Flexible Launch Control (IA32_SGXPUBKEYHASH{0..3} MSRs): true
SGX extensions for virtualizers (EINCVIRTCHILD, EDECVIRTCHILD, ESETCONTEXT): false
Extensions for concurrent memory management (ETRACKC, ELDBC, ELDUC, ERDINFO): false
CET enclave attributes support (See Table 37-5 in the SDM): false
Key separation and sharing (KSS) support (CONFIGID, CONFIGSVN, ISVEXTPRODID, ISVFAMILYID report fields): true
Max enclave size (32-bit): 0x80000000
Max enclave size (64-bit): 0x100000000000000
EPC size: 0x200000000
SGX driver loaded: false
AESMD installed: false
SGX PSW/libsgx installed: false
For deploying my container to kubernetes I followed the recommendation provided by azure doc:
https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-nodes-aks-addon
including the mounting of the /var/run/aesmd/ volume
my configuration is here:
apiVersion: apps/v1
kind: Deployment
metadata:
name: dv-algo
namespace: app-db9fbac56e14
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: dv-algo
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
algo-sandboxed: "false"
app: dv-algo
spec:
containers:
- env:
- name: SGX_AESM_ADDR
value: "1"
image: dvaks.azurecr.io/gsc-dv-algo:db9fbac56e14-221005122423
imagePullPolicy: Always
name: main
resources:
limits:
cpu: 500m
memory: 2Gi
sgx.intel.com/epc: 2Gi
requests:
cpu: 50m
memory: 1Gi
securityContext:
privileged: true
runAsUser: 0
volumeMounts:
- mountPath: /cache
name: cache-volume
- mountPath: /var/run/aesmd
name: var-run-aesmd
volumes:
- emptyDir: {}
name: cache-volume
- hostPath:
path: /var/run/aesmd
type: ""
name: var-run-aesmd
At this point any help would be very useful.
Thanks in advance
Loic
Support for running the application as non-root user inside GSC docker container is needed.
Should be able to run the application as non-root user inside docker container
Running application with non-root user fails with below error.
Observe failure in thread creation
Using gramine release tag 1.0
We are running the OpenVINO Model Server within GSC signed docker image. In our case, OVMS is running along with NGINX configured in reverse proxy mode.
OVMS should be running and should be able to load the model successfully and inference should be successful
pthread creation is failing while loading the model and OVMS is exiting.
Below is the snippet of log:
2021-11-08 14:09:27.718214: F external/org_tensorflow/tensorflow/core/platform/default/env.cc:73] Check failed: ret == 0 (11 vs. 0)Thread creation via pthread_create() failed.
xargs: /ovms/bin/./ovms: terminated by signal 6
Attaching the logs & manifest file use
gsc_manifest.txt
ovms_rel1.0_6.log
d
Hi
I got the following error, if I try to run the gsc container:
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:
- loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------
[P1:T1:] error: Error opening executable /hashgraph: -13
[P1:T1:] error: Error during shim_init() in init_important_handles (-13)
This is my dockerfile:
From ubuntu:20.04
RUN apt-get update \
&& apt-get install -y libssl-dev \
openssl
COPY ./hashgraph /hashgraph
WORKDIR /hashgraph
ENTRYPOINT ["./hashgraph"]
The user permission of the executable:
-rwxrwxr-x 1 developer developer 8364240 Nov 25 17:19 hashgraph*
What I'm doing wrong?
CentOS8 supported by GSC has already reached its EOL that was planned at Dec. 31st 2021, and it is the only officially supported CentOS distro by GSC as of now.
CentOS7.9, on the other hand, is still widely used in production deployment. Its official EOL is June 30, 2024. As confidential computing gains industrial awareness, Gramine and GSC are their top choice to enter CC field and protect their business workload inside a TEE.
We hope the GSC community consider supporting CentOS7.9 in GSC and Gramine, so as to enable the TEE protections for existing industrial partner to protect their workload. Thank you very much for your considerations!
Creating a graminized image of Ubuntu 20.04 with bash fails with /bin/bash: Permission denied.
test.sh
#!/bin/sh
echo "Hello Bash Shell Script"
/bin/bash sample.sh
sample.sh
#!/bin/sh
echo "Hello Sample Shell Script"
Dockerfile:
From ubuntu:20.04
RUN apt-get update
COPY test.sh .
COPY sample.sh .
CMD ["/bin/bash", "test.sh"]
When graminized application is running, it should return "Hello Bash Shell Script"
docker run --device=/dev/sgx/enclave -it gsc-bash-test
Gramine is starting. Parsing TOML manifest file, this may take some time...
/usr/local/bin/entry_script_bash.sh: line 4: /bin/bash: Permission denied
In all ubuntu:20.04 based docker images, /bin is symlink to /usr/bin. Because of this GSC doesn't add any /bin file to trusted files. Any bash script executed at run time having shebang #!/bin/bash is not able to access /bin/bash as it's not part of trusted files list. We get error: /bin/bash: bad interpreter: Permission denied".
Hello,
I am continuing my exploration of GSC+SGX generated enclave, and I am hurting a new issue and I have a more general question.
Let starts by the issue:
I have a mounted volume in my docker image (and container) that is located at "/resources",
Note that this volume is initially empty.
I'd like to mount this directory and access it from within my GSCfied running container.
If in my gsc manifest I specify
sgx.allowed_files = [ "file:/resources/" ]
It all works fine.
On the other hand, if I use instead
sgx.trusted_files = [ "file:/resources/" ],
then I have a permission issues when trying to write in this directory.
I would have prefer to use the trusted_files paradigm to delegate the file on disk encryption to gramine and make my life easier.
Any idea of what could cause the permission denied error ?
I've also tried, without success, with:
fs.mounts = [
{ type = "encrypted", path = "/resources", uri = "/resources", key_name = "file_key" },
]
Now, the general question.
Can I retrieve somehow (from within the enclave) a private/public key pair that is only known by the enclave ?
I was hoping to find that in the /attestation/keys directory, but this one is initially empty and acquire a /attestation/key/default file after I generate an attestation for the enclave, but that default key is actually empty.
I suppose it's related to
https://gramine.readthedocs.io/en/stable/attestation.html#high-level-secret-provisioning-interface,
but I am not sure. Also when I setup an env variable SECRET_PROVISION_CONSTRUCTOR=1, nothing happens.
I was hoping to see something either in /attestation/keys/ or an env variable SECRET_PROVISION_SECRET_STRING to be created. But none of this happened.
Thanks in advance for your advices,
Loic
Dear all, i'm trying to deploy a docker image of mongodb (in particular mongo:4.4) with gsc on ubuntu 20.04, I've successfully accomplished the test with ubuntu-bash image; but the deployment of mongo is still giving me the following issue:
(''' error: Error during libos_init() in init_exec_handle (-13) ''')
I'm using this code, with a default ManifestFile:
./gsc build --insecure-args mongo:4.4 ManifestFile.manifest
&& openssl genrsa -3 -out key.pem 3072
&& ./gsc sign-image gsc-mongo:4.4-unsigned key.pem
&& docker run --device=/dev/sgx_enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-mongo:4.4 -c ls
these are the log after build and sign:
(''' Successfully built an unsigned graminized Docker image gsc-library/mongo:4.4-unsigned
from original application image library/mongo:4.4
''')
(''' Successfully built a signed Docker image gsc-library/mongo:4.4
from gsc-library/mongo:4.4-unsigned
. ''')
After that give me the error.
Unable to convert Flask server docker container using GSC
Hi,
I am trying to do a simple conversion of docker images running Flask server with REST API using GSC but fail, reproduced steps as below
app.py
from flask import Flask, jsonify, request
app=Flask(__name__)
#we are importing our function from the colors.py file
@app.route("/",methods=['GET','POST'])
def index():
if request.method=='GET':
#getting the url argument
url = request.args.get('url')
result={'Result': "GET result"}
return jsonify(result)
else:
return jsonify({'Error':"This is a GET API method"})
if __name__ == '__main__':
app.run(debug=True,host='0.0.0.0', port=5000)
ubuntu18.04-flask.dockerfile
# Sets the base image for subsequent instructions
FROM ubuntu:18.04
# FROM python:3.7-slim-stretch
# Sets the working directory in the container
WORKDIR /app
RUN apt-get update -y
RUN apt-get upgrade -y
RUN apt-get install -y python3
RUN apt-get install -y python3-distutils python3-pip python3-apt
# Copies the dependency files to the working directory
COPY requirements.txt /app/requirements.txt
# Install dependencies
RUN pip3 install -r requirements.txt
# Copies everything to the working directory
COPY . /app
RUN chmod 777 ./app.py
# Command to run on container start
CMD [ "python3" , "./app.py" ]
requirements.txt
click==7.1.2
Flask==1.1.2
itsdangerous==1.1.0
Jinja2==2.11.3
MarkupSafe==1.1.1
Werkzeug==1.0.1
Build the docker image
docker build -t ubuntu18.04-flask-demo -f ./ubuntu18.04-flask.dockerfile .
Convert the build (using generic manifest)
./gsc build --insecure-args ubuntu18.04-flask-demo test/generic.manifest
./gsc sign-image ubuntu18.04-flask-demo enclave-key.pem
Run the converted docker
docker run -d --device=/dev/sgx_enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket -p 5000:5000 gsc-ubuntu18.04-flask-demo
Calling at localhost should return response
curl localhost:5000
{
"Result": "GET result"
}
The docker container hanging with these logs only
Gramine is starting. Parsing TOML manifest file, this may take some time...
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:
- loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------
Emulating a raw syscall instruction. This degrades performance, consider patching your application to use Gramine syscall API.
* Serving Flask app "app" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
* Restarting with stat
Emulating a raw syscall instruction. This degrades performance, consider patching your application to use Gramine syscall API.
* Debugger is active!
* Debugger PIN: 138-003-840
python:3.7-slim-stretch
) or it has to be ubuntu:18.04
and then install python3 in the docker image?docker run
command ? something like this:docker run -p 5000:5000 -it --device=/dev/sgx_enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-<some-converted-image> '$ARG'
Thanks a lot
I noticed that the gethostname function inside container does not report the same result when running as plain docker container (where it produces the container id in docker) compared to running a gramenized gsc-image (where it appears to provide the underlying node OS's hostname)
Simple python web server providing back own hostname:
#!/usr/bin/env python3
import socket
import sys
from http.server import BaseHTTPRequestHandler, HTTPServer
class DummyRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
my_id = socket.gethostname()
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
print("container id:")
print(my_id)
self.wfile.write(my_id.encode())
def main(argv):
if len(argv) != 2:
print(f'Usage: {argv[0]} <PORT>', file=sys.stderr)
return 1
port = int(argv[1])
srv = HTTPServer(('0.0.0.0', port), DummyRequestHandler)
srv.serve_forever()
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))
Running
python3 servehostname.py 12345 &
curl localhost:12345
gives back machine's hostname --> expected.
Doing a docker build image and docker run image
Running same curl localhost:12345
will yield containerID as hostname. --> also expected.
You would expect the same containerID when run as gsc image.
However, when running the gsc-built and signed image, the result will nwo again be the machine's hostname, resulting in multiple gsc containers on the same host having an identical hostname.
Hi,
I'm run Mosquitto in gsc.
It creates the log file when the application is started and I get the following message after it is up and running
Error: Unable to open logfile /var/log/mosquitto/mosquitto.log for writing.
If I want to have a log file, I have to add the /var/log/mosquitto/mosquitto.log
to the allowed files.
But then I get the following message from Gramine:
Gramine detected the following insecure configurations:
- loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
- sgx.allowed_files = [ ... ] (some files are passed through from untrusted host without verification)
Gramine will continue application execution, but this configuration must not be used in production!
If I add it to the trusted_files
I get the following error message.
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 70, in <module>
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 30, in main
expanded = manifest.expand_all_trusted_files()
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/manifest.py", line 193, in expand_all_trusted_files
append_trusted_dir_or_file(trusted_files, tf, expanded)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/manifest.py", line 57, in append_trusted_dir_or_file
raise ManifestError(f'Cannot resolve {path}')
graminelibos.manifest.ManifestError: Cannot resolve /var/log/mosquitto/mosquitto.log
If I create the file in the dockerfile and add it to the trusted_files
I get the messages:
Error: Unable to open logfile `/var/log/mosquitto/mosquitto.log` for writing.
error: Disallowing create/write/append to a trusted file '/var/log/mosquitto/mosquitto.log'
Is the allowed_files
the only way to make this file available for the application?
Hi,
I try to run the GSC Bash example on an Azure machine. I followed the instructions in the README.
I could build the container, but the last command docker run ...
command runs into an error.
My first try was to run the command from the README file
$ sudo docker run --device=/dev/sgx_enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-ubuntu18.04-bash -c ls
docker: Error response from daemon: error gathering device information while adding custom device "/dev/sgx_enclave": no such file or directory.
On my Azure machine is no /dev/sgx_enclave
directory. Should it be there?
There is an /dev/sgx
directory, which contains two character devices (enclave
and provision
).
If I pass this directory --device
argument I get the same message as above at the end, but with some more steps before:
docker run --device=/dev/sgx -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-ubuntu18.04-bash -c ls
++ find /gramine/meson_build_output/lib -type d -path '*/site-packages'
+ export PYTHONPATH=:/gramine/meson_build_output/lib/python3.6/site-packages
+ PYTHONPATH=:/gramine/meson_build_output/lib/python3.6/site-packages
++ find /gramine/meson_build_output/lib -type d -path '*/pkgconfig'
+ export PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ '[' -z '' ']'
+ gramine-sgx-get-token --sig /entrypoint.sig --output /entrypoint.token
Attributes:
...
date: 2021-11-02
+ gramine-sgx /entrypoint -c ls
error: Cannot open device /dev/sgx_enclave. Please make sure the Intel SGX kernel module is loaded.
error: load_enclave() failed with error -2
I can run the bash example in the CI-Examples folder without any issues.
I saw in the README the line change --device=/dev/sgx_enclave to your version of the Intel SGX driver if needed
.
How I can figure this out?
Converting the official mongo docker image to gsc fails because of the symlink /lib -> usr/lib, the finalize_manifest.py doesn't include the /lib directory in the manifest, due to os.walk not traversing symlinks, but later on the application tries to access the /lib directory.
Gramine dependency and tensorflow-avx512 dependency causing conflict in baremetal, unable to run tensorflow-avx512 with gramine in baremetal
When we install intel-tensorflow-avx512, MarkupSafe library is upgraded to 2.1.2 latest version,
If you run gramine-sgx-gen-private-key, it fails with "cannot import name 'soft_unicode' from 'markupsafe' ". The solution is to downgrade the MarkUpSafe to 2.0.1, when we downgrade, gramine-sgx-gen-private-key works, but tensorflow-avx512 get broken
When we run tensorflow-avx512, it returns with numpy error
gramine-sgx-gen-private-key -f
Traceback (most recent call last):
ย File "/home/gramineperf2/gramerf_framework/gramine_install/usr/bin/gramine-sgx-gen-private-key", line 11, in <module>
ย ย ย from graminelibos import sgx_sign
ย File "/home/gramineperf2/gramerf_framework/gramine_install/usr/lib/python3.8/site-packages/graminelibos/__init__.py", line 18, in <module>
ย ย ย from .gen_jinja_env import make_env
ย File "/home/gramineperf2/gramerf_framework/gramine_install/usr/lib/python3.8/site-packages/graminelibos/gen_jinja_env.py", line 7, in <module>
ย ย ย import jinja2
ย File "/usr/lib/python3/dist-packages/jinja2/__init__.py", line 33, in <module>
ย ย ย from jinja2.environment import Environment, Template
ย File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 15, in <module>
ย ย ย from jinja2 import nodes
ย File "/usr/lib/python3/dist-packages/jinja2/nodes.py", line 23, in <module>
ย ย ย from jinja2.utils import Markup
ย File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 656, in <module>
ย ย ย from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/home/gramineperf2/.local/lib/python3.8/site-packages/markupsafe/__init__.py)
tensorflow should be able to build and run inside Gramine
After running tensorflow-avx512
ย ย ย import pandas as pd
../.local/lib/python3.8/site-packages/pandas/__init__.py:22: in <module>
ย ย ย from pandas.compat import is_numpy_dev as _is_numpy_devย # pyright: ignore # noqa:F401
../.local/lib/python3.8/site-packages/pandas/compat/__init__.py:18: in <module>
ย ย ย from pandas.compat.numpy import (
../.local/lib/python3.8/site-packages/pandas/compat/numpy/__init__.py:23: in <module>
ย ย ย raise ImportError(
Eย ย ImportError: this version of pandas is incompatible with numpy < 1.20.3
Eย ย your numpy version is 1.19.5.
Eย ย Please upgrade numpy to >= 1.20.3 to use this pandas version
Dear Gramine (GSC) support,
I have a hard time understanding how to generate a SGX report under a GSC build docker container. The report generation process is pretty well explained in the documentation:
https://gramine.readthedocs.io/en/stable/attestation.html
But my problem is that in my graminized (GSC) container, I was expecting to find a
/dev/attestation/user_report_data virtual file I could write on.
But unfortunately, I don't see any such file on my container system.
So I can't use it for writing. (I tried to create one, but it didn't work)
I do see a /gramine/ repository in my container which indicates that my container is a valid GSC container.
How am I practically supposed to generate an attestation if /dev/attestation is not there ?
Thanks for any help
Loic
Hi,I want to run /gsc/test this examples ,but when I run ./gsc build command.It reports errors as this:
`Step 6/30 : RUN git clone https://github.com/gramineproject/gramine.git /gramine
---> Running in c0d7cf1bead6
Cloning into '/gramine'...
fatal: unable to access 'https://github.com/gramineproject/gramine.git/': Could not resolve host: github.com
Failed to build unsigned graminized Docker image gsc-ubuntu18.04-bash-unsigned
.`
My steps:
`cd gsc-1.3.1
sudo apt-get install docker.io python3 python3-pip
pip3 install docker jinja2 tomli tomli-w pyyaml
pip3 install toml # for compatibility with Gramine v1.3 or lower
cp config.yaml.template config.yaml
docker build --tag ubuntu18.04-bash --file test/ubuntu18.04-bash.dockerfile .
sudo ./gsc build --insecure-args ubuntu18.04-bash test/ubuntu18.04-bash.manifest
These are the last outputs from the terminal:
Step 4/30 : RUN echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' > /etc/apt/sources.list.d/intel-sgx.list && apt-key add intel-sgx-deb.key
---> Using cache
---> f6b6034db113
Step 5/30 : RUN env DEBIAN_FRONTEND=noninteractive apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y libsgx-dcap-quote-verify-dev
---> Using cache
---> 59662e39e4c1
`
Do you have an idea how to get more detailed information to figure out what's going wrong?
What fails?
meson.build:165:8: ERROR: Problem encountered: Invalid SGX driver configuration (-Dsgx_driver and/or -Dsgx_driver_include_path); expected "sgx_user.h" to exist under "/gramine/driver/driver/linux/include"
Suggested fix by Dmitrii: change templates/Dockerfile.common.compile.template to have "-Dsgx_driver=dcap1.6"
2. gsc sign-image fails with:
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 74, in <module>
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 48, in main
sigstruct.sign(sign_with_local_key, key)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/sigstruct.py", line 167, in sign
exponent_int, modulus_int, signature_int = do_sign_callback(data, *args, **kwargs)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/sgx_sign.py", line 584, in sign_with_local_key
modulus = bytes.fromhex(modulus_out[8:8+offs.SE_KEY_SIZE*2].decode())
ValueError: non-hexadecimal number found in fromhex() arg at position 512
I guessed it required newer version of python. And changed templates/ubuntu/Dockerfile.build.template to be:
{% extends "Dockerfile.common.build.template" %}
{% block install %}
RUN apt-get update \
&& apt-get install -y wget libcurl4
RUN wget https://packages.microsoft.com/ubuntu/18.04/prod/pool/main/a/az-dcap-client/az-dcap-client_1.10_amd64.deb \
&& dpkg -i az-dcap-client_1.10_amd64.deb
RUN apt-get update \
&& env DEBIAN_FRONTEND=noninteractive apt-get install -y \
binutils \
libcurl4-openssl-dev \
libffi-dev \
libprotobuf-c-dev \
locales \
locales-all \
openssl \
python3.8 \
python3.8-dev \
python3-cryptography \
python3-pip \
python3-protobuf \
python3-pyelftools
# Default python 3.6 fails to parse key from hex inside
# Gramine sign_key.py. Create link to python 3.8.
RUN rm /usr/bin/python3
RUN ln -s /usr/bin/python3.8 /usr/bin/python3
RUN ls -l /usr/bin/python*
# Older version of markupsafe is required for subsequent install.
RUN pip3 install markupsafe==1.0.0
RUN pip3 install click jinja2 protobuf 'toml>=0.10'
RUN pip3 install -U cffi
{% if debug %}
RUN env DEBIAN_FRONTEND=noninteractive apt-get install -y \
gdb \
less \
libunwind8 \
python3-pytest \
strace \
vim
{% endif %}
RUN locale-gen en_US.UTF-8
ENV LC_ALL en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8
{% endblock %}
----------------------- files ---------------------
Script to run build and sign. Please replace todo.
#!/bin/bash
set -e
SCRIPT_DIR="$(realpath "$(dirname -- "${BASH_SOURCE[0]}")")"
echo $SCRIPT_DIR
# Input/output docker images' tags.
NATIVE_VERSION=0.0.1
ENCRYPTED_VERSION=$NATIVE_VERSION
NATIVE_IMAGE=todo-repo/todo-image-${NATIVE_VERSION}
GSC_IMAGE=gsc-$NATIVE_IMAGE
GSC_UNSIGNED_IMAGE=$GSC_IMAGE-unsigned
ENCRYPTED_IMAGE=todo-repo/todo-image-${ENCRYPTED_VERSION}
# Remove gsc images as the tool checks for their existence as
# a way to check for errors during build.
docker rmi -f $GSC_IMAGE $GSC_UNSIGNED_IMAGE
echo "Native image='$NATIVE_IMAGE'"
echo "Encrypted image='$ENCRYPTED_IMAGE'"
# Get the gsc tool - gsc in the root of the repo is the python script to run.
BASE_DIR=$HOME/tmp
GSC_DIR=$BASE_DIR/gsc
GSC=$GSC_DIR/gsc
mkdir -p $BASE_DIR
if [ ! -f "$GSC" ]; then
git clone --depth 1 https://github.com/gramineproject/gsc.git $GSC_DIR
chmod +x $GSC
fi
# Create venv for bringing python dependencies required by gsc.
VENV_DIR=$BASE_DIR/venv
mkdir -p $VENV_DIR
if [ ! -d "$VENV_DIR/bin" ]; then
python3 -m venv $VENV_DIR
fi
source $VENV_DIR/bin/activate
# Bring dependencies needed by gsc.
pip3 install docker jinja2 toml pyyaml
# Graminize the image.
CONFIG_FILE=$SCRIPT_DIR/gramine_gsc_config.yaml
MANIFEST_FILE=$SCRIPT_DIR/gramine.manifest
cd $GSC_DIR
$GSC build -c $CONFIG_FILE $NATIVE_IMAGE $MANIFEST_FILE || cd -
# Run docker inspect to fail the script if the image was not generated.
docker image inspect $GSC_UNSIGNED_IMAGE
# Generate signing key.
echo "Generating key file"
KEY_FILE=$BASE_DIR/image_key.pem
openssl genrsa -out $KEY_FILE 2048
# Generate signed image <--- THIS STEP FAILS
cd $GSC_DIR
$GSC sign-image -c $CONFIG_FILE $NATIVE_IMAGE $KEY_FILE || cd -
Contents of gramine_gsc_config.yaml:
Distro: "ubuntu:18.04"
Registry: ""
Gramine:
Repository: "https://github.com/gramineproject/gramine.git"
Branch: "v1.2"
SGXDriver:
# Intel recommended using LD_1.33 - special version for Azure and DCAP_1.6
# in gsc/templates/Dockerfile.common.compile.template
Repository: "https://github.com/intel/SGXDataCenterAttestationPrimitives.git"
Branch: "DCAP_1.6 && cp -r driver/linux/* ."
Contents of gramine.manifest:
sgx.remote_attestation = false
sgx.enclave_size = "2G"
Contents of dockerfile for native image, hello.py contains print("testing"):
FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y \
python3.8
WORKDIR /app
COPY hello.py .
ENTRYPOINT ["python3", "-m", "hello"]
See gramineproject/gramine#411 for context.
Since Gramine doesn't support Alpine Linux (gramineproject/gramine#993), same thing for GSC.
Note that this issue depends on the feature of supporting Gramine in Alpine.
Currently GSC supports Ubuntu 18.04 as base OS. Can you please add support for Ubuntu 20.04 also?
After step setup build/ --buildtype=release -Ddirect=enabled -Dsgx=enabled \
when I build gramine using ninja command the following error comes up:
~/Desktop/gramine$ ninja -C build/ ninja: Entering directory `build/' [0/19] Generating subprojects/glibc-2.34-1/glibc with a custom command glibc: see /home/augusto/Desktop/gramine/build/subprojects/glibc-2.34-1/glibc-build.log for full build log glibc: preparing sources... glibc: running configure... [16/18] Linking target Pal/src/host/Li...s-report/gramine-sgx-ias-verify-report FAILED: subprojects/glibc-2.34-1/crt1.o subprojects/glibc-2.34-1/crti.o subprojects/glibc-2.34-1/crtn.o subprojects/glibc-2.34-1/libdl.so subprojects/glibc-2.34-1/libdl.so.2 subprojects/glibc-2.34-1/libc.so subprojects/glibc-2.34-1/libc.so.6 subprojects/glibc-2.34-1/libutil.so subprojects/glibc-2.34-1/libutil.so.1 subprojects/glibc-2.34-1/libm.so subprojects/glibc-2.34-1/libm.so.6 subprojects/glibc-2.34-1/libnsl.so subprojects/glibc-2.34-1/libnsl.so.1 subprojects/glibc-2.34-1/libpthread.so subprojects/glibc-2.34-1/libpthread.so.0 subprojects/glibc-2.34-1/libthread_db.so subprojects/glibc-2.34-1/libthread_db.so.1 subprojects/glibc-2.34-1/libnss_compat.so subprojects/glibc-2.34-1/libnss_compat.so.2 subprojects/glibc-2.34-1/libnss_db.so subprojects/glibc-2.34-1/libnss_db.so.2 subprojects/glibc-2.34-1/libnss_files.so subprojects/glibc-2.34-1/libnss_files.so.2 subprojects/glibc-2.34-1/libanl.so subprojects/glibc-2.34-1/libanl.so.1 subprojects/glibc-2.34-1/libnss_dns.so subprojects/glibc-2.34-1/libnss_dns.so.2 subprojects/glibc-2.34-1/libresolv.so subprojects/glibc-2.34-1/libresolv.so.2 subprojects/glibc-2.34-1/librt.so subprojects/glibc-2.34-1/librt.so.1 subprojects/glibc-2.34-1/ld-linux-x86-64.so.2 subprojects/glibc-2.34-1/ld.so subprojects/glibc-2.34-1/libmvec.so subprojects/glibc-2.34-1/libmvec.so.1 /bin/sh /home/augusto/Desktop/gramine/subprojects/glibc-2.34-1/compile.sh x86_64 ../subprojects/glibc-2.34-1 ../subprojects/glibc-2.34-1/glibc-2.34 /home/augusto/Desktop/gramine/build/subprojects/glibc-2.34-1 subprojects/glibc-2.34-1/crt1.o.p /usr/local lib/x86_64-linux-gnu '' csu/crt1.o csu/crti.o csu/crtn.o dlfcn/libdl.so dlfcn/libdl.so.2 libc.so libc.so.6 login/libutil.so login/libutil.so.1 math/libm.so math/libm.so.6 nis/libnsl.so nis/libnsl.so.1 nptl/libpthread.so nptl/libpthread.so.0 nptl_db/libthread_db.so nptl_db/libthread_db.so.1 nss/libnss_compat.so nss/libnss_compat.so.2 nss/libnss_db.so nss/libnss_db.so.2 nss/libnss_files.so nss/libnss_files.so.2 resolv/libanl.so resolv/libanl.so.1 resolv/libnss_dns.so resolv/libnss_dns.so.2 resolv/libresolv.so resolv/libresolv.so.2 rt/librt.so rt/librt.so.1 elf/ld-linux-x86-64.so.2 elf/ld.so mathvec/libmvec.so mathvec/libmvec.so.1 ninja: build stopped: subcommand failed.
There are 2 dangling docker images are present after every workload graminization
Graminize any workload
Once it is done, you will see a final gsc image and 2 untagged images
No untagged images, Cleanup should happen correctly
| gsc-redis | 7.0.0x | 08a251a6a362 | 14 minutes ago | 750MB |
| < none > | < none > | 331be109bac2 | 14 minutes ago | 750MB |
| < none > | < none > | d0e4340ffa5f | 15 minutes ago | 1.58GB |
| ubuntu | 18.04 | 8d5df41c547b | 8 days ago | 63.1MB |
| redis | 7.0.0 | 53aa81e8adfa | 2 months ago | 117MB |
Hello,
I am trying to graminize an image on a debian distro,
but I am getting the following error:
...
Executing subproject mbedtls-mbedtls-3.2.1
mbedtls-mbedtls-3.2.1| Project name: mbedtls
mbedtls-mbedtls-3.2.1| Project version: undefined
mbedtls-mbedtls-3.2.1| C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
mbedtls-mbedtls-3.2.1| C linker for the host machine: cc ld.bfd 2.34
mbedtls-mbedtls-3.2.1| Program compile-gramine.sh found: YES (/bin/sh /gramine/subprojects/mbedtls-mbedtls-3.2.1/compile-gramine.sh)
mbedtls-mbedtls-3.2.1| Program compile-pal.sh found: YES (/gramine/subprojects/mbedtls-mbedtls-3.2.1/compile-pal.sh)
mbedtls-mbedtls-3.2.1| Build targets in project: 5
mbedtls-mbedtls-3.2.1| Subproject mbedtls-mbedtls-3.2.1 finished.
Program protoc-c found: YES (/usr/bin/protoc-c)
Program protoc found: NO
meson.build:262:4: ERROR: Program 'protoc' not found or not executable
A full log can be found at /gramine/build/meson-logs/meson-log.txt
Failed to build unsigned graminized docker image `gsc-dv-algo-unsigned`.
What is puzzling me is that protobuf-compiler is already installed.
> apt-get install -y libprotobuf-dev protobuf-compiler
Reading package lists... Done
Building dependency tree
Reading state information... Done
libprotobuf-dev is already the newest version (3.6.1.3-2ubuntu5).
protobuf-compiler is already the newest version (3.6.1.3-2ubuntu5).
0 upgraded, 0 newly installed, 0 to remove and 59 not upgraded.
Any help or insight would be appreciated.
Thanks in advance
I am trying to follow the python example (https://gramine.readthedocs.io/projects/gsc/en/latest/#example) and build the gsc-python image successfully. But I found that in the container of gsc-python, I could not use the command "gramine-sgx /entrypoint xxx.py" to run the script even using the root user. I have confirmed that "chmod 777 xxx.py". Only the commands like "gramine-sgx /entrypoint -c print("1")" can work.
And in my customized image, there is the same phenomenon. In my customized image, the entrypoint is "/bin/bash". And the command "gramine-sgx /entrypoint -c ls" could work, but the command "gramine-sgx /entrypoint xxx.sh" failed and the error is "Permission denied" as well.
print("1")
The script "test.py" runs successfully.
Gramine is starting. Parsing TOML manifest file, this may take some time...
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:
loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------
python3: can't open file '/test.py': [Errno 13] Permission denied
If I use
gramine-sgx /entrypoint -c "print("1")"
the result is
ramine is starting. Parsing TOML manifest file, this may take some time...
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:
- loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------
1
What's the cause of the problem and how can I use "gramine-sgx /entrypoint test.py" to run the script?
Some images can have a registry and repository component to their names. It would be good if GSC supported them when building images.
As an example, the Red Hat UBI image for Python 3.8 is at registry.access.redhat.com/ubi8/python-38
.
Add something like this to config.yaml
Distro: "registry.access.redhat.com/ubi8/python-38
An additional variable could also be used to store the new path component (see Additional information). Note that this example uses a non-supported image (that will be a separate issue/PR), but the idea is the same for e.g. my.private.registry.cloud/dev/centos
, which would use the centos
templates but include the full path in the FROM
.
The build process should understand that structure and use in e.g. the FROM
fields of the Dockerfile
, and the build process should continue normally.
There is currently no support for this so it fails
$ ./gsc build registry.access.redhat.com/ubi8/python-38 test/generic.manifest
Building unsigned graminized Docker image `gsc-registry.access.redhat.com/ubi8/python-38-unsigned` from original application image `registry.access.redhat.com/ubi8/python-38`...
Traceback (most recent call last):
File "./gsc", line 12, in <module>
sys.exit(main(sys.argv))
File "/home/frmuno/orig/og/gsc/./gsc.py", line 476, in main
return args.command(args)
File "/home/frmuno/orig/og/gsc/./gsc.py", line 179, in gsc_build
distro, _ = distro.split(':')
ValueError: not enough values to unpack (expected 2, got 1)
Hi.
I'me having an issue while signing an custom, non-root image in debug mode:
./gsc sign-image 4651f63e3c27 enclave-key.pem ๎ฒ ๎ master โ ๎ฒ 07:09 ๎ฒ 07:43:52
Signing graminized Docker image `unsigned_image_name` -> `gsc-4651f63e3c27`...
Step 1/6 : FROM gsc-4651f63e3c27-unsigned as unsigned_image
---> a54c6566fe5c
Step 2/6 : COPY gsc-signer-key.pem /gramine/app_files/gsc-signer-key.pem
---> 493eb743c8d6
Step 3/6 : RUN export PYTHONPATH="${PYTHONPATH}:$(find /gramine/meson_build_output/lib -type d -path '*/site-packages')" && gramine-sgx-sign --key /gramine/app_files/gsc-signer-key.pem --manifest /gramine/app_files/entrypoint.manifest --output /gramine/app_files/entrypoint.manifest.sgx
---> Running in 217a21c6ac85
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 74, in <module>
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 34, in main
expanded = manifest.expand_all_trusted_files()
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/manifest.py", line 175, in expand_all_trusted_files
append_trusted_dir_or_file(trusted_files, tf, expanded)
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/manifest.py", line 70, in append_trusted_dir_or_file
append_tf(trusted_files, uri, hash_file_contents(path))
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/manifest.py", line 29, in hash_file_contents
with open(path, 'rb') as f:
PermissionError: [Errno 13] Permission denied: '/gramine/subprojects/packagecache/mbedtls-2.26.0.tar.gz'
Removing intermediate container 217a21c6ac85
Failed to build a signed graminized Docker image `gsc-4651f63e3c27`.
Indeed, the whole /gramine/subprojects/packagecache/
folder's content has 600
permission with 0:0
UID/GID.
This appears to be a failure due to update in major release version of protobuf from 3.20.1 to 4.21.0 - https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Below are the logs when I attempt to run gsc sign-image IMAGE_NAME SIGNING_KEY_FILE
...
Step 2/6 : COPY gsc-signer-key.pem /gsc-signer-key.pem
---> aa1594fff42b
Step 3/6 : RUN export PYTHONPATH="${PYTHONPATH}:$(find /gramine/meson_build_output/lib -type d -path '*/site-packages')" && gramine-sgx-sign --key /gsc-signer-key.pem --manifest /entrypoint.manifest --output /entrypoint.manifest.sgx
---> Running in a3c8fd7333a5
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 11, in <module>
from graminelibos import (
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/__init__.py", line 24, in <module>
from .sgx_get_token import get_token
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/sgx_get_token.py", line 9, in <module>
from . import _aesm_pb2 as aesm_pb2
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/_aesm_pb2.py", line 38, in <module>
_descriptor.FieldDescriptor(
File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py", line 560, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Removing intermediate container a3c8fd7333a5
Exception: Cannot resolve "file:/usr/local/lib/python3.8/dist-packages/pandas/tests/io/data/legacy_pickle/1.2.4/empty_frame_v1_2_4-GH or the file does not exist.
when running
sudo ./gsc sign-image <docker_name> enclave-key.pem
The issue won't reproduce on the current master branch. I am following up on this issue.
git clone http://github.com/oscarlab/graphene
cd graphene/Tools/gsc
cp config.yaml.template config.yaml
sed -i "s/2e737e69f076c60918f87d6829bb769925e75fec/a743b6a83b4f3587e1d72b53509239c21d3c1231/g" config.yaml
sudo apt install python3 python3-pip
sudo pip3 install docker jinja2 toml pyyaml
sudo ./gsc build --insecure-args <docker_name> ./test/ubuntu18.04-pytorch.manifest -nc --rm
openssl genrsa -3 -out enclave-key.pem 3072
sudo ./gsc sign-image <docker_name> enclave-key.pem
A signed copy of the container being created.
Fails with the following error:
Step 7/11 : RUN /graphene/python/graphene-sgx-sign -libpal /graphene/Runtime/libpal-Linux-SGX.so -key /gsc-signer-key.pem -manifest /entrypoint.manifest -output /entrypoint.manifest.sgx
---> Running in 3102aa0715d3
Attributes:
size: 0x100000000
thread_num: 32
isv_prod_id: 0
isv_svn: 0
attr.flags: 0600000000000000
attr.xfrm: 0300000000000000
misc_select: 00000000
date: 2021-08-10
Trusted files:
Traceback (most recent call last):
File "/graphene/python/graphene-sgx-sign", line 5, in <module>
sys.exit(main())
File "/graphene/python/graphenelibos/sgx_sign.py", line 851, in main
return main_sign(args)
File "/graphene/python/graphenelibos/sgx_sign.py", line 774, in main_sign
for key, val in get_trusted_files(manifest).items():
File "/graphene/python/graphenelibos/sgx_sign.py", line 242, in get_trusted_files
targets[key] = (val, resolve_uri(val, check_exist))
File "/graphene/python/graphenelibos/sgx_sign.py", line 211, in resolve_uri
raise Exception(
Exception: Cannot resolve "file:/usr/local/lib/python3.8/dist-packages/pandas/tests/io/data/legacy_pickle/1.2.4/empty_frame_v1_2_4-GH or the file does not exist.
Removing intermediate container 3102aa0715d3
Failed to build a signed graphenized Docker image `<docker_name>`.
Hello @dimakuv
I am experiencing problems if I use master branch to deploy on azure AKS.
At runtime, my container fails with: "error: Cannot open device /dev/sgx_enclave. Please make sure the Intel SGX kernel module is loaded."
If I revert gsc to commit 7d5ca56, then It woks perfectly.
In both case, my config.yaml contains:
SGXDriver:
Repository: "https://github.com/intel/SGXDataCenterAttestationPrimitives.git"
Branch: "DCAP_1.11 && cp -r driver/linux/* ."
I also tried to keep the default in recent version, but it doesn't work better.
So I suspect compatibility with AKS was broken recently
Hey there, just a wondering dev interested in unikernels but I was reading on your project's efforts to make unikernel containers and was wondering why not make an OCI-compliant base image from graphene(gramine?) that way it can take advantage of the existing ecosystem of available tools. Alternatively, if graphene dynamically adds only the necessary OS libraries required by a specific application to run, why not make a build system plugin for existing CI/CD pipelines that wraps the deployed app in a OCI-compliant, graphene unikernel that would then get run?
This is all just a suggestion, I don't have the proper expertise on LibOSs nor do I know your internal priorities (it seems SGX is high on that list) to know why you're making the design decisions you are.
PR gramineproject/gramine#1093 in core Gramine removes the need for explicit gramine-sgx-get-token
action. (And the follow-up PR gramineproject/gramine#1094 removes such possibility for DCAP/upstream completely.)
When those PRs are merged plus the new version of Gramine is released, we should apply the same change here.
While trying to build base gramine image, I am encountering build failures due to lack of key availability.
Command: ./gsc build-gramine gramine-base
Output: Step 3/9 : COPY intel-sgx-deb.key /
Failed to build a base-Gramine image gsc-gramine-base
.
Config.yaml file contents are as described below.
Distro: "ubuntu:20.04"
Registry: ""
Gramine:
Repository: "https://github.com/gramineproject/gramine.git"
Branch: "v1.2"
After having look into gsc.py, the below line was missing in "def gsc_build_gramine(args)" function.
shutil.copyfile('keys/intel-sgx-deb.key', tmp_build_path / 'intel-sgx-deb.key')
Raised PR #92 to address the issue.
Hi,
I got the following error, if I try to run the gsc container:
[P1:T1:mosquitto] trace: ---- shim_geteuid() = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(AT_FDCWD, "/etc/nsswitch.conf", 0x25a027630, 0) = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(AT_FDCWD, "/", 0x25a027750, 0) = 0x0
[P1:T1:mosquitto] trace: ---- shim_openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|0x80000, 0000) = 0x4
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a027550, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x1ee
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a027630, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_close(4) = 0x0
[P1:T1:mosquitto] trace: ---- shim_openat(AT_FDCWD, "/etc/passwd", O_RDONLY|0x80000, 0000) = 0x4
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a0276b0, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_lseek(4, 0x0, SEEK_SET) = 0x0
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x3d8
[P1:T1:mosquitto] trace: ---- shim_close(4) = 0x0
[P1:T1:mosquitto] trace: ---- shim_openat(AT_FDCWD, "/proc/sys/kernel/ngroups_max", O_RDONLY|0x80000, 0000) = -2
[P1:T1:mosquitto] trace: ---- shim_newfstatat(AT_FDCWD, "/etc/nsswitch.conf", 0x25a027690, 0) = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(AT_FDCWD, "/", 0x25a0277b0, 0) = 0x0
[P1:T1:mosquitto] trace: ---- shim_openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|0x80000, 0000) = 0x4
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a0275b0, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x1ee
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a027690, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_close(4) = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(AT_FDCWD, "/etc/nsswitch.conf", 0x25a027690, 0) = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(AT_FDCWD, "/", 0x25a0277b0, 0) = 0x0
[P1:T1:mosquitto] trace: ---- shim_openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|0x80000, 0000) = 0x4
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a0275b0, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x1ee
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x0
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a027690, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_close(4) = 0x0
[P1:T1:mosquitto] trace: ---- shim_openat(AT_FDCWD, "/etc/group", O_RDONLY|0x80000, 0000) = 0x4
[P1:T1:mosquitto] trace: ---- shim_newfstatat(4, "", 0x25a027240, 4096) = 0x0
[P1:T1:mosquitto] trace: ---- shim_lseek(4, 0x0, SEEK_SET) = 0x0
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x1cf
[P1:T1:mosquitto] trace: ---- shim_read(4, 0x258342d70, 0x2000) ...
[P1:T1:mosquitto] trace: ---- return from shim_read(...) = 0x0
[P1:T1:mosquitto] trace: ---- shim_close(4) = 0x0
[P1:T1:mosquitto] trace: ---- shim_setgroups(1, 0x25a8ffce0) = 0x0
[P1:T1:mosquitto] warning: Unsupported system call setresgid
[P1:T1:mosquitto] trace: ---- shim_time(0) = 0x61e60300
[P1:T1:mosquitto] debug: ---- shim_exit_group (returning 1)
[P1:T1:mosquitto] debug: clearing POSIX locks for pid 1
[P1:T1:mosquitto] debug: sync client shutdown: closing handles
[P1:T1:mosquitto] debug: sync client shutdown: waiting for confirmation
[P1:T1:mosquitto] debug: sync client shutdown: finished
[P1:shim] debug: IPC worker: exiting worker thread
[P1:T1:mosquitto] debug: process 1 exited with status 1
debug: DkProcessExit: Returning exit code 1
This is my dockerfile:
From ubuntu:20.04
ENV LD_LIBRARY_PATH = "${LD_LIBRARY_PATH}:/usr/lib/x86_64-linux-gnu/"
RUN apt-get update -y && apt-get install -y \
openssl \
mosquitto \
apt-utils
EXPOSE 1883
EXPOSE 8883
COPY examples/mosquitto/ /etc/mosquitto/
ENTRYPOINT ["mosquitto"]
CMD ["-c", "/etc/mosquitto/mosquitto.conf", "-v"]
What is confusing me about this issue is that it is working fine if I run it in gramine-sgx
. I don't run into the missing setresgid
warning. What's the difference between them?
The example works if I don't use the -c /etc/mosquitto/mosquitto.conf
argument. For some reason, this is causing the issue. mosquitto
takes a default configuration, if the argument is missing.
I need the mosquitto.conf
, because I want to add TLS to mosquitto
and here is the place where to put the paths for the certificates.
Thanks in advance.
I've been waiting for #60 to be merged, and it seems that it introduces an issue dur to the fact that entrypoint.manifest
was moved from /
to /gramine/app_files
.
Further details:
After wiping my local Docker images repo to start from scratch, I build a custom Ubutu-based image of NodeRed, then Gramininzed it:
./gsc build -c config.yaml --insecure-args b604f652bdea nodered.manifest
Using Dive I inspected it and it looks like the file is actually there.
Signing, however, cannot find it:
ubuntu@sgx ~/gsc ./gsc sign-image b604f652bdea enclave-key.pem
Signing graminized Docker image `unsigned_image_name` -> `gsc-b604f652bdea`...
Step 1/6 : FROM gsc-b604f652bdea-unsigned as unsigned_image
---> 47dc02f98a1a
Step 2/6 : COPY gsc-signer-key.pem /gramine/app_files/gsc-signer-key.pem
---> 549ee88b74ce
Step 3/6 : RUN export PYTHONPATH="${PYTHONPATH}:$(find /gramine/meson_build_output/lib -type d -path '*/site-packages')" && gramine-sgx-sign --key /gramine/app_files/gsc-signer-key.pem --manifest /gramine/app_files/entrypoint.manifest --output /gramine/app_files/entrypoint.manifest.sgx
---> Running in 7dd953b572a7
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 74, in <module>
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 34, in main
expanded = manifest.expand_all_trusted_files()
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/manifest.py", line 175, in expand_all_trusted_files
append_trusted_dir_or_file(trusted_files, tf, expanded)
File "/gramine/meson_build_output/lib/python3.8/site-packages/graminelibos/manifest.py", line 56, in append_trusted_dir_or_file
raise ManifestError(f'Cannot resolve {path}')
graminelibos.manifest.ManifestError: Cannot resolve /entrypoint.manifest
Removing intermediate container 7dd953b572a7
Failed to build a signed graminized Docker image `gsc-b604f652bdea`.
From here, it looks like gramine-sgx-sign cannot handle this change. Sorry I don't have enough time to dive in deeper.
gsc sign-image command fails while doing gramin-sgx-sign
Some images work with gsc (example python), however some images have issues. Looked at this one which has similar error:
#44. But couldn't tell what the solution was and that issue is still open.
Git commit: b7f0576
./gsc build --insecure-args gcr.io/k8s-minikube/kicbase:v0.0.35 test/generic.manifest
...
Successfully built 26719820b50d
Successfully tagged gsc-gcr.io/k8s-minikube/kicbase:v0.0.35-unsigned
Successfully built an unsigned graminized Docker image `gsc-gcr.io/k8s-minikube/kicbase:v0.0.35-unsigned` from original application image `gcr.io/k8s-minikube/kicbase:v0.0.35`.
./gsc sign-image gcr.io/k8s-minikube/kicbase:v0.0.35 ~/.config/gramine/enclave-key.pem
./gsc sign-image gcr.io/k8s-minikube/kicbase:v0.0.35 ~/.config/gramine/enclave-key.pem
Signing graminized Docker image `gsc-gcr.io/k8s-minikube/kicbase:v0.0.35-unsigned` -> `gsc-gcr.io/k8s-minikube/kicbase:v0.0.35`...
Step 1/9 : FROM gsc-gcr.io/k8s-minikube/kicbase:v0.0.35-unsigned as unsigned_image
---> 26719820b50d
Step 2/9 : COPY gsc-signer-key.pem /gramine/app_files/gsc-signer-key.pem
---> 9f2ee808f0b5
Step 3/9 : ARG passphrase
---> Running in 34cf15b6f0fe
Removing intermediate container 34cf15b6f0fe
---> ed5c751e5e60
Step 4/9 : COPY sign.sh /gramine/app_files/sign.sh
---> 92692aa0cb11
Step 5/9 : RUN chmod +x /gramine/app_files/sign.sh
---> Running in 821233a47801
Removing intermediate container 821233a47801
---> 3052b70804c1
Step 6/9 : RUN export PYTHONPATH="${PYTHONPATH}:$(find /gramine/meson_build_output/lib -type d -path '*/site-packages')" && /gramine/app_files/sign.sh /gramine/app_files/gsc-signer-key.pem /gramine/app_files/entrypoint.manifest /gramine/app_files/entrypoint.manifest.sgx $passphrase
---> Running in 9db7da5563ad
spawn gramine-sgx-sign --key /gramine/app_files/gsc-signer-key.pem --manifest /gramine/app_files/entrypoint.manifest --output /gramine/app_files/entrypoint.manifest.sgx
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/toml/decoder.py", line 511, in loads
ret = decoder.load_line(line, currentlevel, multikey,
File "/usr/local/lib/python3.8/dist-packages/toml/decoder.py", line 778, in load_line
value, vtype = self.load_value(pair[1], strictly_valid)
File "/usr/local/lib/python3.8/dist-packages/toml/decoder.py", line 880, in load_value
return (self.load_array(v), "array")
File "/usr/local/lib/python3.8/dist-packages/toml/decoder.py", line 1026, in load_array
nval, ntype = self.load_value(a[i])
File "/usr/local/lib/python3.8/dist-packages/toml/decoder.py", line 866, in load_value
raise ValueError("Reserved escape sequence used")
ValueError: Reserved escape sequence used
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 74, in <module>
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 32, in main
manifest = Manifest.load(manifest_file)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/manifest.py", line 150, in load
return cls.loads(f.read())
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/manifest.py", line 146, in loads
return cls(s)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/manifest.py", line 84, in __init__
manifest = toml.loads(manifest_str)
File "/usr/local/lib/python3.8/dist-packages/toml/decoder.py", line 514, in loads
raise TomlDecodeError(str(err), original, pos)
toml.decoder.TomlDecodeError: Reserved escape sequence used (line 11 column 1 char 274)
Hi,
I want to create a GSC container that runs the hashgraph executable. If I use vanilla docker it works. Unfortunately, I get the following error if I want to run the docker in GSC mode:
+ gramine-sgx-get-token --sig /entrypoint.sig --output /entrypoint.token
Attributes:
mr_enclave: f221ea933e3a0c8b5863b3d1dc7c30a1dff49bb7bb08e3a10fe786e9655af0fd
mr_signer: 95374182109bb1a4127542af2c4af54fe844becb4d08eb815743c93acb521c4c
isv_prod_id: 0
isv_svn: 0
attr.flags: 0000000000000004
attr.xfrm: 0000000000000007
mask.flags: ffffffffffffffff
mask.xfrm: fffffffffff9ff1b
misc_select: 00000000
misc_mask: ffffffff
modulus: 63a8561e12f90901c8cb3272dacdcb00...
exponent: 3
signature: db532a08aa38d66301c7ea3960ff8543...
date: 2021-12-02
+ gramine-sgx /entrypoint node0/settings.yaml
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:
- loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------
hashgraph: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
This is my Dockerfile:
From ubuntu:20.04
RUN apt-get update -y && apt-get install -y \
openssl \
build-essential \
libssl-dev
RUN mkdir hashgraph-dir
COPY hashgraph /hashgraph-dir
WORKDIR /hashgraph-dir
RUN cp hashgraph /usr/local/bin/
ENTRYPOINT ["hashgraph"]
CMD ["node0/settings.yaml"]
And this is my manifest:
sgx.enclave_size = "16G"
sgx.thread_num = 32
sys.stack.size = "2M"
loader.pal_internal_mem_size = "128M"
Do I have to add some manifest option to the get it working?
I am trying to run a modified PostgreSQL (with customized executors and so on) inside an enclave. Hence, I pulled the ubuntu18.04 image from dockerhub, built the modified PostgreSQL inside the ubuntu docker, and then docker commit
the modified image.
After I have signed the modified image, it failed to load and get error code 6 when I hit docker run.
My system configuration:
Ubuntu 18.04 with kernel 5.9.0
Docker version 20.10.16, build aa7e414
gramine built on branch v1.1
SGXDriver:
Repository: "https://github.com/intel/linux-sgx-driver.git"
Branch: "sgx_driver_2.11"
Then I createad the gsc-signed image using:
./gsc build --insecure-args myImage test/generic.manifest
./gsc sign-image myImage enclave-key.pem
./gsc info-image gsc-myImage
and I got output from gsc info-image as
mr_enclave = "2454c58cafad79b1ded05a276bef96ccff8b77dbca61071928da014e6183d4e9"
mr_signer = "5416a28ebb3a9ebd0bef05431b2c4ea9eccaec008d7691ef772fa12c2d045bec"
isv_prod_id = 0
isv_svn = 0
date = "2022-05-20"
flags = "0400000000000000"
xfrms = "0300000000000000"
misc_select = "00000000"
debug = false
docker run --device=/dev/isgx \
-v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
-it gsc-myImage
enter the container successfully with everything protected by SGX
++ find /gramine/meson_build_output/lib -type d -path '*/site-packages'
+ export PYTHONPATH=:/gramine/meson_build_output/lib/python3.6/site-packages
+ PYTHONPATH=:/gramine/meson_build_output/lib/python3.6/site-packages
++ find /gramine/meson_build_output/lib -type d -path '*/pkgconfig'
+ export PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ '[' -z '' ']'
+ gramine-sgx-get-token --sig /entrypoint.sig --output /entrypoint.token
Attributes:
mr_enclave: 2454c58cafad79b1ded05a276bef96ccff8b77dbca61071928da014e6183d4e9
mr_signer: 5416a28ebb3a9ebd0bef05431b2c4ea9eccaec008d7691ef772fa12c2d045bec
isv_prod_id: 0
isv_svn: 0
attr.flags: 0000000000000004
attr.xfrm: 0000000000000007
mask.flags: ffffffffffffffff
mask.xfrm: fffffffffff9ff1b
misc_select: 00000000
misc_mask: ffffffff
modulus: 63b8dd6ab325beb315c5828b811f983e...
exponent: 3
signature: f4459c545e01a11c46f7f4a7c50b8dd0...
date: 2022-05-20
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-get-token", line 20, in <module>
main() # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/gramine/meson_build_output/bin/gramine-sgx-get-token", line 16, in main
token = get_token(sig, verbose=verbose)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/sgx_get_token.py", line 151, in get_token
token = connect_aesmd(sig['enclave_hash'], sig['modulus'], sig['attribute_flags'], xfrms)
File "/gramine/meson_build_output/lib/python3.6/site-packages/graminelibos/sgx_get_token.py", line 88, in connect_aesmd
raise Exception(f'Failed. (Error Code = {ret_msg.ret.error})')
Exception: Failed. (Error Code = 6)
It's a common security measure to run containers with their rootfs mounted as read-only. In kubernetes this is enforced in the apps securityContext
and some clusters may even enforce security policies to forbid running apps without read-only=true.
With docker
this is run as:
$ docker run -it --read-only --device /dev/sgx_enclave -v /dev/sgx:/dev/sgx -v /var/run/aesmd:/var/run/aesmd <GSC'd image>
The image runs OK.
graphene-sgx-get-token: error: argument --output/-output: can't open '/entrypoint.token': [Errno 30] Read-only file system: '/entrypoint.token'
This is related to #37 which also talks about making it "pythonless". Could the token be generated by the loader(?) so that it's does not need to be stored on a rootfs?
Support for non-root user with GSC docker build is needed. Currently support is available only for ROOT user.
Build is successful.
Below is the error seen with GSC build.
E: Could not open lock file /var/lib/apt/lists/lock - open (13: Permission denied)
E: Unable to lock directory /var/lib/apt/lists/
Failed to build unsigned graminized docker image `gsc-ubuntu18.04-bash-nonroot-unsigned`.
I tried to run flink SQL through gsc
, but encountered an error at the docker run gsc-XXX
step.
step 1: Prepare the container with flink-1.14.4 installed
step 2: Modify the flink-generic.manifest file
# Some workloads like Python may generate huge manifest files
loader.pal_internal_mem_size = "128M"
sgx.enclave_size = "64G"
sgx.thread_num = 128
sgx.trusted_files = [
"file:/gramine/app_files/entrypoint.manifest", # unused entry, only to test merging of manifests
]
Here is my config.yaml
content
# Specify the OS distro. Currently tested distros are
# - ubuntu:18.04, ubuntu:20.04, ubuntu:21.04
# - debian:10, debian:11
# - centos:8
Distro: "ubuntu:20.04"
# If the image has a specific registry, define it here.
# Empty by default; example value: "registry.access.redhat.com/ubi8".
Registry: ""
# If you're using your own fork and branch of Gramine, specify the GitHub link and the branch name
# below; typically, you want to keep the default values though
Gramine:
Repository: "https://github.com/gramineproject/gramine.git"
Branch: "v1.3.1"
# Specify the Intel SGX driver installed on your machine (more specifically, on the machine where
# the graminized Docker container will run); there are several variants of the SGX driver:
#
# - legacy out-of-tree driver: use something like the below values, but adjust the branch name
# Repository: "https://github.com/01org/linux-sgx-driver.git"
# Branch: "sgx_driver_1.9"
#
# - DCAP out-of-tree driver: use something like the below values
# Repository: "https://github.com/intel/SGXDataCenterAttestationPrimitives.git"
# Branch: "DCAP_1.11 && cp -r driver/linux/* ."
#
# - DCAP in-kernel driver: use empty values like below
# Repository: ""
# Branch: ""
#
SGXDriver:
Repository: ""
Branch: ""
step 3:According to https://gramine.readthedocs.io/projects/gsc/en/latest/#example, get gsc-a1db02aeb076
image
step 4: docker run --device=/dev/sgx_enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-a1db02aeb076
Starting cluster.
Starting standalonesession daemon on host adminroot.
Starting taskexecutor daemon on host adminroot.
[P1:T1:bash] error: process creation failed
docker-entrypoint.sh
content is:
#!/usr/bin/env bash
./flink-1.14.4/bin/start-cluster.sh
part of the full error log is(I omitted the similar log in the middle):
root@adminroot:/home/adminroot/gramine/gramine-1.3.1/tools/gsc# docker run --device=/dev/sgx_enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-a1db02aeb076
+ set -e
++ find /gramine/meson_build_output/lib -type d -path '*/site-packages'
+ export PYTHONPATH=:/gramine/meson_build_output/lib/python3.8/site-packages
+ PYTHONPATH=:/gramine/meson_build_output/lib/python3.8/site-packages
++ find /gramine/meson_build_output/lib -type d -path '*/pkgconfig'
+ export PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ PKG_CONFIG_PATH=:/gramine/meson_build_output/lib/x86_64-linux-gnu/pkgconfig
+ '[' -z '' ']'
+ gramine-sgx-get-token --quiet --sig /gramine/app_files/entrypoint.sig --output /gramine/app_files/entrypoint.token
+ gramine-sgx /gramine/app_files/entrypoint ''
Gramine is starting. Parsing TOML manifest file, this may take some time...
debug: Gramine parsed TOML manifest file successfully
debug: Token file: /gramine/app_files/entrypoint.token
debug: Read dummy DCAP token
debug: Enclave created:
debug: base: 0x0000000000000000
debug: size: 0x0000001000000000
debug: misc_select: 0x00000000
debug: attr.flags: 0x0000000000000007
debug: attr.xfrm: 0x00000000000000e7
debug: ssa_frame_size: 4
debug: isv_prod_id: 0x00000000
debug: isv_svn: 0x00000000
debug: Adding pages to SGX enclave, this may take some time...
debug: Adding pages to enclave: 0xffee6d000-0x1000000000 [REG:R--] (manifest) measured
debug: Adding pages to enclave: 0xffea6d000-0xffee6d000 [REG:RW-] (ssa) measured
debug: Adding pages to enclave: 0xffe9ed000-0xffea6d000 [TCS:---] (tcs) measured
...
...
...
debug: Adding pages to enclave: 0xffbd41000-0xffbd94000 [REG:R-X] (code) measured
debug: Adding pages to enclave: 0xffbd95000-0xffbd9a000 [REG:RW-] (data) measured
debug: Adding pages to enclave: 0xffbd9a000-0xffc16d000 [REG:RW-] (bss) measured
debug: Adding pages to enclave: 0x10000-0xffbd41000 [REG:RWX] (free)
debug: Added all pages to SGX enclave
debug: Enclave initializing:
debug: enclave id: 0x0000000ffffff000
debug: mr_enclave: a8b3585a0216edc376f18a5c90deef78d39cd59b467c13836d5e78e8b33cea48
Detected a huge manifest, preallocating 128MB of internal memory.
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:
- sgx.debug = true (this is a debug enclave)
- loader.log_level = warning|debug|trace|all (verbose log level, may leak information)
- loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------
warning: PalVirtualMemoryProtect is unimplemented in Linux-SGX PAL
debug: Gramine was built from commit: e18bc05b17fd704b259cb0401f928dc4ec5199a6
debug: Host: Linux-SGX
debug: LibOS xsave_enabled 1, xsave_size 0xa80(2688), xsave_features 0xe7
debug: Initial VMA region 0xffa748000-0xffa824000 (LibOS) bookkeeped
debug: Initial VMA region 0xffee6d000-0x1000000000 (manifest) bookkeeped
debug: ASLR top address adjusted to 0x5a9b4b000
debug: host is Linux-SGX but remote attestation type is 'none', adding only /dev/attestation/attestation_type file and skipping others (report, etc.)
debug: LibOS loaded at 0xffa748000, ready to initialize
debug: mounting "file:/" (chroot) under /
debug: mounting "proc" (pseudo) under /proc
debug: mounting "dev" (pseudo) under /dev
debug: mounting "dev:tty" (chroot) under /dev/tty
debug: mounting "sys" (pseudo) under /sys
debug: Creating pipe: pipe.srv:bca73b70daa9f328a9165993312477b23c86694fac1290e1894c96e19227d39b
[P1:T1:] debug: File /gramine/app_files/docker-entrypoint.sh not recognized as ELF, looking for shebang
[P1:T1:] debug: Assembling 4 execve arguments (total size is 59 bytes)
[P1:T1:] debug: Interpreter to be used for execve: /usr/bin/env
[P1:T1:env] debug: Creating pipe: pipe.srv:e4d7fef97b6120e004348672cd30c4e248f8347898177a44c25aadf8211b53ba
[P1:T1:env] debug: Allocated stack at 0x5a9b0b000 (size = 0x40000)
[P1:T1:env] debug: loading "file://usr/bin/env"
[P1:T1:env] debug: append_r_debug: adding file://usr/bin/env at 0x5a9afd000
[P1:T1:env] debug: find_interp: searching for interpreter: /gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/ld-linux-x86-64.so.2
[P1:T1:env] debug: loading "file://gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/ld-linux-x86-64.so.2"
[P1:T1:env] debug: append_r_debug: adding file://gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/ld-linux-x86-64.so.2 at 0x5a9ac4000
[P1:T1:env] debug: Creating pipe: pipe.srv:1
[P1:T1:env] debug: Creating pipe: pipe.srv:98e8434545a9894509d434573edf65fe4fe4c7314101cc68621305a325f446fc
[P1:T1:env] debug: LibOS initialized
[P1:libos] debug: IPC worker started
[P1:T1:env] debug: append_r_debug: adding file:[vdso_libos] at 0x5a9ac3000
[P1:T1:env] trace: ---- brk(0) = 0x5aa746000
[P1:T1:env] warning: Not supported flag (0x3001) passed to arch_prctl
[P1:T1:env] trace: ---- arch_prctl(12289, 0x5a9b4ab20) = -38
[P1:T1:env] trace: ---- mmap(0, 0x2000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0x0) ...
[P1:T1:env] trace: ---- return from mmap(...) = 0x5a9ac1000
[P1:T1:env] trace: ---- access("/etc/ld.so.preload", F_OK|X_OK) = -2
[P1:T1:env] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/glibc-hwcaps/x86-64-v4/libc.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:env] trace: ---- newfstatat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/glibc-hwcaps/x86-64-v4", 0x5a9b49d40, 0) = -2
[P1:T1:env] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/glibc-hwcaps/x86-64-v3/libc.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:env] trace: ---- newfstatat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/glibc-hwcaps/x86-64-v3", 0x5a9b49d40, 0) = -2
[P1:T1:env] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:env] trace: ---- newfstatat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/glibc-hwcaps/x86-64-v2", 0x5a9b49d40, 0) = -2
[P1:T1:env] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/tls/haswell/avx512_1/x86_64/libc.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:env] trace: ---- newfstatat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/tls/haswell/avx512_1/x86_64", 0x5a9b49d40, 0) = -2
[P1:T1:env] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/tls/haswell/avx512_1/libc.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:env] trace: ---- newfstatat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/tls/haswell/avx512_1", 0x5a9b49d40, 0) = -2
[P1:T1:env] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/tls/haswell/x86_64/libc.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:env] trace: ---- newfstatat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/tls/haswell/x86_64", 0x5a9b49d40, 0) = -2
...
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/tls/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/tls", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/haswell", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot", 0x5a98c2d50, 0) = 0x0
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/glibc-hwcaps/x86-64-v4/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/glibc-hwcaps/x86-64-v4", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/glibc-hwcaps/x86-64-v3/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/glibc-hwcaps/x86-64-v3", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/glibc-hwcaps/x86-64-v2/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/glibc-hwcaps/x86-64-v2", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/haswell/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls/haswell/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/haswell/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls/haswell/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/haswell/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls/haswell/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/haswell/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls/haswell", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/tls/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/tls", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/haswell/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/haswell/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/haswell/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/haswell/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/haswell/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/haswell/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/haswell/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/haswell", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/usr/local/lib/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/usr/local/lib", 0x5a98c2d50, 0) = 0x0
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v4/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v4", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v3/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v3", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v2/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/glibc-hwcaps/x86-64-v2", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/avx512_1/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/avx512_1/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/avx512_1/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/avx512_1", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/x86_64/libtinfo.so.6", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/lib/x86_64-linux-gnu/x86_64", 0x5a98c2d50, 0) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libtinfo.so.6", O_RDONLY|0x80000, 0000) = 0x3
[P1:T1:bash] trace: ---- read(3, 0x5a98c2eb8, 0x340) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x340
[P1:T1:bash] trace: ---- newfstatat(3, "", 0x5a98c2d50, 4096) = 0x0
[P1:T1:bash] trace: ---- mmap(0, 0x2e9a0, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0x0) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99a7000
[P1:T1:bash] trace: ---- mmap(0x5a99b5000, 0xe000, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99b5000
[P1:T1:bash] trace: ---- mmap(0x5a99c3000, 0xe000, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99c3000
[P1:T1:bash] trace: ---- mmap(0x5a99d1000, 0x5000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x29000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99d1000
[P1:T1:bash] trace: ---- close(3) = 0x0
[P1:T1:bash] debug: glibc register library /lib/x86_64-linux-gnu/libtinfo.so.6 loaded at 0x5a99a7000
[P1:T1:bash] debug: append_r_debug: adding file://lib/x86_64-linux-gnu/libtinfo.so.6 at 0x5a99a7000
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/libdl.so.2", O_RDONLY|0x80000, 0000) = 0x3
[P1:T1:bash] trace: ---- read(3, 0x5a98c2e98, 0x340) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x340
[P1:T1:bash] trace: ---- newfstatat(3, "", 0x5a98c2d30, 4096) = 0x0
[P1:T1:bash] trace: ---- mmap(0, 0x4028, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0x0) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99a2000
[P1:T1:bash] trace: ---- mmap(0x5a99a3000, 0x1000, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99a3000
[P1:T1:bash] trace: ---- mmap(0x5a99a4000, 0x1000, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99a4000
[P1:T1:bash] trace: ---- mmap(0x5a99a5000, 0x2000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99a5000
[P1:T1:bash] trace: ---- close(3) = 0x0
[P1:T1:bash] debug: glibc register library /gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/libdl.so.2 loaded at 0x5a99a2000
[P1:T1:bash] debug: append_r_debug: adding file://gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/libdl.so.2 at 0x5a99a2000
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/libc.so.6", O_RDONLY|0x80000, 0000) = 0x3
[P1:T1:bash] trace: ---- read(3, 0x5a98c2e78, 0x340) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x340
[P1:T1:bash] trace: ---- pread64(3, 0x5a98c2890, 0x310, 0x40) ...
[P1:T1:bash] trace: ---- return from pread64(...) = 0x310
[P1:T1:bash] trace: ---- pread64(3, 0x5a98c2850, 0x30, 0x350) ...
[P1:T1:bash] trace: ---- return from pread64(...) = 0x30
[P1:T1:bash] trace: ---- pread64(3, 0x5a98c2800, 0x44, 0x380) ...
[P1:T1:bash] trace: ---- return from pread64(...) = 0x44
[P1:T1:bash] trace: ---- newfstatat(3, "", 0x5a98c2d10, 4096) = 0x0
[P1:T1:bash] trace: ---- pread64(3, 0x5a98c2960, 0x310, 0x40) ...
[P1:T1:bash] trace: ---- return from pread64(...) = 0x310
[P1:T1:bash] trace: ---- mmap(0, 0x1f9a50, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0x0) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a9689000
[P1:T1:bash] trace: ---- mprotect(0x5a96b5000, 0x1c0000, PROT_NONE) ...
[P1:T1:bash] trace: ---- return from mprotect(...) = 0x0
[P1:T1:bash] trace: ---- mmap(0x5a96b5000, 0x16a000, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2c000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a96b5000
[P1:T1:bash] trace: ---- mmap(0x5a981f000, 0x55000, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x196000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a981f000
[P1:T1:bash] trace: ---- mmap(0x5a9875000, 0x6000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1eb000) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a9875000
[P1:T1:bash] trace: ---- mmap(0x5a987b000, 0x7a50, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED, -1, 0x0) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a987b000
[P1:T1:bash] trace: ---- close(3) = 0x0
[P1:T1:bash] debug: glibc register library /gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/libc.so.6 loaded at 0x5a9689000
[P1:T1:bash] debug: append_r_debug: adding file://gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/libc.so.6 at 0x5a9689000
[P1:T1:bash] trace: ---- mmap(0, 0x2000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0x0) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a99a0000
[P1:T1:bash] trace: ---- arch_prctl(4098, 0x5a99a1400) = 0x0
[P1:T1:bash] trace: ---- set_tid_address(0x5a99a16d0) = 0x1
[P1:T1:bash] trace: ---- set_robust_list(0x5a99a16e0, 0x18) = 0x0
[P1:T1:bash] warning: Unsupported system call rseq
[P1:T1:bash] trace: ---- mprotect(0x5a9875000, 0x3000, PROT_READ) ...
[P1:T1:bash] trace: ---- return from mprotect(...) = 0x0
[P1:T1:bash] trace: ---- mprotect(0x5a99a5000, 0x1000, PROT_READ) ...
[P1:T1:bash] trace: ---- return from mprotect(...) = 0x0
[P1:T1:bash] trace: ---- mprotect(0x5a99d1000, 0x4000, PROT_READ) ...
[P1:T1:bash] trace: ---- return from mprotect(...) = 0x0
[P1:T1:bash] trace: ---- mprotect(0x5a9b34000, 0x3000, PROT_READ) ...
[P1:T1:bash] trace: ---- return from mprotect(...) = 0x0
[P1:T1:bash] trace: ---- mmap(0, 0x2000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0x0) ...
[P1:T1:bash] trace: ---- return from mmap(...) = 0x5a999e000
[P1:T1:bash] trace: ---- mprotect(0x5a9a0e000, 0x2000, PROT_READ) ...
[P1:T1:bash] trace: ---- return from mprotect(...) = 0x0
[P1:T1:bash] trace: ---- prlimit64(0, 3, 0, 0x5a98c38b0) = 0x0
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/dev/tty", O_RDWR|0x800, 0000) = -22
[P1:T1:bash] trace: ---- ioctl(0, TCGETS, 0x5a98c39c0) ...
[P1:T1:bash] trace: ---- return from ioctl(...) = -38
[P1:T1:bash] trace: ---- getrandom(0x5a9880198, 0x8, GRND_NONBLOCK) = 0x8
[P1:T1:bash] trace: ---- brk(0) = 0x5a9fea000
[P1:T1:bash] trace: ---- brk(0x5aa00b000) = 0x5aa00b000
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/locale/locale-archive", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/share/locale/locale.alias", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/locale/C.UTF-8/LC_IDENTIFICATION", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/locale/C.utf8/LC_IDENTIFICATION", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc/locale/C/LC_IDENTIFICATION", O_RDONLY|0x80000, 0000) = -2
[P1:T1:bash] trace: ---- getuid() = 0x0
[P1:T1:bash] trace: ---- getgid() = 0x0
[P1:T1:bash] trace: ---- geteuid() = 0x0
[P1:T1:bash] trace: ---- getegid() = 0x0
[P1:T1:bash] trace: ---- rt_sigprocmask(BLOCK, NULL, [], 0x8) = 0x0
[P1:T1:bash] trace: ---- gettimeofday(0x5a9b3ff10, 0) = 0x0
[P1:T1:bash] trace: ---- ioctl(-1, TIOCGPGRP, 0x5a98c3a3c) ...
[P1:T1:bash] trace: ---- return from ioctl(...) = -9
[P1:T1:bash] trace: ---- sysinfo(0x5a98c37b0) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGCHLD], 0x5a98c3670, 0x5a98c3710, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGCHLD], 0x5a98c3670, 0x5a98c3710, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGINT], 0x5a98c3670, 0x5a98c3710, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGINT], 0x5a98c3670, 0x5a98c3710, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGQUIT], 0x5a98c3670, 0x5a98c3710, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGQUIT], 0x5a98c3670, 0x5a98c3710, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGTSTP], 0x5a98c3640, 0x5a98c36e0, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGTSTP], 0x5a98c3640, 0x5a98c36e0, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGTTIN], 0x5a98c3640, 0x5a98c36e0, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGTTIN], 0x5a98c3640, 0x5a98c36e0, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGTTOU], 0x5a98c3640, 0x5a98c36e0, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGTTOU], 0x5a98c3640, 0x5a98c36e0, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigprocmask(BLOCK, NULL, [], 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGQUIT], 0x5a98c3670, 0x5a98c3710, 0x8) = 0x0
[P1:T1:bash] trace: ---- uname(0x5a98c3770) = 0x0
[P1:T1:bash] trace: ---- getcwd(0x5a9fef1a0, 0x1000) = 0x2
[P1:T1:bash] trace: ---- getpid() = 0x1
[P1:T1:bash] trace: ---- getppid() = 0x0
[P1:T1:bash] trace: ---- stat(".", 0x5a98c37a0) = 0x0
[P1:T1:bash] trace: ---- stat("/gramine/meson_build_output/bin/bash", 0x5a98c3680) = -2
[P1:T1:bash] trace: ---- stat("/usr/local/openjdk-11/bin/bash", 0x5a98c3680) = -2
[P1:T1:bash] trace: ---- stat("/usr/local/sbin/bash", 0x5a98c3680) = -2
[P1:T1:bash] trace: ---- stat("/usr/local/bin/bash", 0x5a98c3680) = -2
[P1:T1:bash] trace: ---- stat("/usr/sbin/bash", 0x5a98c3680) = -2
[P1:T1:bash] trace: ---- stat("/usr/bin/bash", 0x5a98c3680) = -2
[P1:T1:bash] trace: ---- stat("/sbin/bash", 0x5a98c3680) = -2
[P1:T1:bash] trace: ---- stat("/bin/bash", 0x5a98c3680) = 0x0
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/bin/bash", 0x5a98c35b0, 0) = 0x0
[P1:T1:bash] trace: ---- geteuid() = 0x0
[P1:T1:bash] trace: ---- getegid() = 0x0
[P1:T1:bash] trace: ---- getuid() = 0x0
[P1:T1:bash] trace: ---- getgid() = 0x0
[P1:T1:bash] trace: ---- access("/bin/bash", F_OK|R_OK) = 0x0
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/bin/bash", 0x5a98c35b0, 0) = 0x0
[P1:T1:bash] trace: ---- geteuid() = 0x0
[P1:T1:bash] trace: ---- getegid() = 0x0
[P1:T1:bash] trace: ---- getuid() = 0x0
[P1:T1:bash] trace: ---- getgid() = 0x0
[P1:T1:bash] trace: ---- access("/bin/bash", F_OK|X_OK) = 0x0
[P1:T1:bash] trace: ---- stat("/bin/bash", 0x5a98c36c0) = 0x0
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/bin/bash", 0x5a98c35f0, 0) = 0x0
[P1:T1:bash] trace: ---- geteuid() = 0x0
[P1:T1:bash] trace: ---- getegid() = 0x0
[P1:T1:bash] trace: ---- getuid() = 0x0
[P1:T1:bash] trace: ---- getgid() = 0x0
[P1:T1:bash] trace: ---- access("/bin/bash", F_OK|R_OK) = 0x0
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/bin/bash", 0x5a98c35f0, 0) = 0x0
[P1:T1:bash] trace: ---- geteuid() = 0x0
[P1:T1:bash] trace: ---- getegid() = 0x0
[P1:T1:bash] trace: ---- getuid() = 0x0
[P1:T1:bash] trace: ---- getgid() = 0x0
[P1:T1:bash] trace: ---- access("/bin/bash", F_OK|X_OK) = 0x0
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/etc/nsswitch.conf", 0x5a98c34d0, 0) = 0x0
[P1:T1:bash] trace: ---- newfstatat(AT_FDCWD, "/", 0x5a98c35e0, 0) = 0x0
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|0x80000, 0000) = 0x3
[P1:T1:bash] trace: ---- newfstatat(3, "", 0x5a98c33f0, 4096) = 0x0
[P1:T1:bash] trace: ---- read(3, 0x5a9ff0390, 0x1000) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x1ee
[P1:T1:bash] trace: ---- read(3, 0x5a9ff0390, 0x1000) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x0
[P1:T1:bash] trace: ---- newfstatat(3, "", 0x5a98c34d0, 4096) = 0x0
[P1:T1:bash] trace: ---- close(3) = 0x0
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/etc/passwd", O_RDONLY|0x80000, 0000) = 0x3
[P1:T1:bash] trace: ---- newfstatat(3, "", 0x5a98c3550, 4096) = 0x0
[P1:T1:bash] trace: ---- lseek(3, 0x0, SEEK_SET) = 0x0
[P1:T1:bash] trace: ---- read(3, 0x5a9ff0390, 0x1000) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x39a
[P1:T1:bash] trace: ---- close(3) = 0x0
[P1:T1:bash] trace: ---- gettimeofday(0x5a98c3840, 0) = 0x0
[P1:T1:bash] trace: ---- getpid() = 0x1
[P1:T1:bash] trace: ---- getppid() = 0x0
[P1:T1:bash] trace: ---- gettimeofday(0x5a98c3840, 0) = 0x0
[P1:T1:bash] trace: ---- getpid() = 0x1
[P1:T1:bash] trace: ---- getppid() = 0x0
[P1:T1:bash] trace: ---- getpgrp() = 0x1
[P1:T1:bash] trace: ---- ioctl(2, TIOCGPGRP, 0x5a98c38fc) ...
[P1:T1:bash] trace: ---- return from ioctl(...) = 0x0
[P1:T1:bash] trace: ---- rt_sigaction([SIGCHLD], 0x5a98c3660, 0x5a98c3700, 0x8) = 0x0
[P1:T1:bash] trace: ---- prlimit64(0, 6, 0, 0x5a98c3860) = 0x0
[P1:T1:bash] trace: ---- rt_sigprocmask(BLOCK, NULL, [], 0x8) = 0x0
[P1:T1:bash] trace: ---- openat(AT_FDCWD, "/gramine/app_files/docker-entrypoint.sh", O_RDONLY, 0000) = 0x3
[P1:T1:bash] trace: ---- stat("/gramine/app_files/docker-entrypoint.sh", 0x5a98c39b0) = 0x0
[P1:T1:bash] trace: ---- ioctl(3, TCGETS, 0x5a98c39d0) ...
[P1:T1:bash] trace: ---- return from ioctl(...) = -38
[P1:T1:bash] trace: ---- lseek(3, 0x0, SEEK_CUR) = 0x0
[P1:T1:bash] trace: ---- read(3, 0x5a98c3b30, 0x50) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x3a
[P1:T1:bash] trace: ---- lseek(3, 0x0, SEEK_SET) = 0x0
[P1:T1:bash] trace: ---- prlimit64(0, 7, 0, 0x5a98c3a00) = 0x0
[P1:T1:bash] trace: ---- fcntl(255, F_GETFD, 0x5a98c3a24) = -9
[P1:T1:bash] trace: ---- dup2(3, 255) = 0xff
[P1:T1:bash] trace: ---- close(3) = 0x0
[P1:T1:bash] trace: ---- fcntl(255, F_SETFD, 0x1) = 0x0
[P1:T1:bash] trace: ---- fcntl(255, F_GETFL, 0) = 0x0
[P1:T1:bash] trace: ---- fstat(255, 0x5a98c3970) = 0x0
[P1:T1:bash] trace: ---- lseek(255, 0x0, SEEK_CUR) = 0x0
[P1:T1:bash] trace: ---- read(255, 0x5a9ff6de0, 0x3a) ...
[P1:T1:bash] trace: ---- return from read(...) = 0x3a
[P1:T1:bash] trace: ---- rt_sigprocmask(BLOCK, NULL, [], 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigprocmask(BLOCK, [SIGINT,SIGTERM,SIGCHLD,], [], 0x8) = 0x0
[P1:T1:bash] trace: ---- lseek(255, -1, SEEK_CUR) = 0x39
[P1:T1:bash] trace: ---- clone(CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|[SIGCHLD], 0, 0, 0x5a99a16d0, 0) ...
[P1:T1:bash] debug: Creating pipe: pipe.srv:3e0a1b2f86c4f333d6cd9df61f4d1f06aeabdcf976543924ad81b271903ed85a
debug: Gramine parsed TOML manifest file successfully
debug: Token file: /gramine/app_files/entrypoint.token
debug: Read dummy DCAP token
debug: Enclave created:
debug: base: 0x0000000000000000
debug: size: 0x0000001000000000
debug: misc_select: 0x00000000
debug: attr.flags: 0x0000000000000007
debug: attr.xfrm: 0x00000000000000e7
debug: ssa_frame_size: 4
debug: isv_prod_id: 0x00000000
debug: isv_svn: 0x00000000
debug: Adding pages to SGX enclave, this may take some time...
debug: Adding pages to enclave: 0xffee6d000-0x1000000000 [REG:R--] (manifest) measured
debug: Adding pages to enclave: 0xffea6d000-0xffee6d000 [REG:RW-] (ssa) measured
debug: Adding pages to enclave: 0xffe9ed000-0xffea6d000 [TCS:---] (tcs) measured
debug: Adding pages to enclave: 0xffe96d000-0xffe9ed000 [REG:RW-] (tls) measured
debug: Adding pages to enclave: 0xffe92d000-0xffe96d000 [REG:RW-] (stack) measured
...
...
...
debug: Adding pages to enclave: 0xffbd95000-0xffbd9a000 [REG:RW-] (data) measured
debug: Adding pages to enclave: 0xffbd9a000-0xffc16d000 [REG:RW-] (bss) measured
debug: Adding pages to enclave: 0x10000-0xffbd41000 [REG:RWX] (free)
[P1:T1:bash] error: process creation failed
[P1:T1:bash] trace: ---- return from clone(...) = -13
[P1:T1:bash] trace: ---- newfstatat(2, "", 0x5a98c2e60, 4096) = 0x0
[P1:T1:bash] trace: ---- ioctl(2, TCGETS, 0x5a98c2dd0) ...
[P1:T1:bash] trace: ---- return from ioctl(...) = -38
[P1:T1:bash] trace: ---- write(2, 0x5a9ff8e70, 0x41) ...
/gramine/app_files/docker-entrypoint.sh: fork: Permission denied
[P1:T1:bash] trace: ---- return from write(...) = 0x41
[P1:T1:bash] trace: ---- rt_sigprocmask(SETMASK, [], NULL, 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigprocmask(BLOCK, [SIGCHLD,], [], 0x8) = 0x0
[P1:T1:bash] trace: ---- rt_sigprocmask(SETMASK, [], NULL, 0x8) = 0x0
[P1:T1:bash] debug: ---- exit_group (returning 254)
[P1:T1:bash] debug: clearing POSIX locks for pid 1
[P1:T1:bash] debug: sync client shutdown: closing handles
[P1:T1:bash] debug: sync client shutdown: waiting for confirmation
[P1:T1:bash] debug: sync client shutdown: finished
[P1:libos] debug: IPC worker: exiting worker thread
[P1:T1:bash] debug: process 1 exited with status 254
debug: PalProcessExit: Returning exit code 254
Here is the info about is_sgx_available
root@adminroot:/home/adminroot/gramine/gramine-1.3.1/tools/gsc# ./../sgx/is-sgx-available/is_sgx_available
SGX supported by CPU: true
SGX1 (ECREATE, EENTER, ...): true
SGX2 (EAUG, EACCEPT, EMODPR, ...): true
Flexible Launch Control (IA32_SGXPUBKEYHASH{0..3} MSRs): true
SGX extensions for virtualizers (EINCVIRTCHILD, EDECVIRTCHILD, ESETCONTEXT): false
Extensions for concurrent memory management (ETRACKC, ELDBC, ELDUC, ERDINFO): false
CET enclave attributes support (See Table 37-5 in the SDM): false
Key separation and sharing (KSS) support (CONFIGID, CONFIGSVN, ISVEXTPRODID, ISVFAMILYID report fields): true
Max enclave size (32-bit): 0x80000000
Max enclave size (64-bit): 0x100000000000000
EPC size: 0xfe000000
SGX driver loaded: true
AESMD installed: true
SGX PSW/libsgx installed: true
I don't know why the process fails to be created. Is it related to the size of the enclave?
The distro on the Redis docker image is debain 11 and providing Debain11
to config.yaml.template doesn't result in any issues. So we should add Debain 11 as one of the supported distros.
When gsc master is built for gramine v1.3.1 on dcap setup, it fails with error error: Cannot open device /dev/sgx_enclave. Please make sure the Intel SGX kernel module is loaded.
This failure is caused by the changes in gsc repo for SGX driver and with the the config yaml template which was modified to build with in-kernel driver
For another description of the same problem, see #114.
Since, we are building on dcap setup, the sgx mountings are /dev/sgx/enclave and /dev/sgx/provision., so the device path is updated in docker run command
docker run --device=/dev/sgx/enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-ubuntu18.04-bash -c ls
Gramine is starting. Parsing TOML manifest file, this may take some time...
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:
- loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------
bin
boot
dev
etc
gramine
home
lib
lib64
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var
docker run --device=/dev/sgx/enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-ubuntu18.04-bash -c ls
Gramine is starting. Parsing TOML manifest file, this may take some time...
error: Cannot open device /dev/sgx_enclave. Please make sure the Intel SGX kernel module is loaded.
error: load_enclave() failed with error -2
Hi!
I followed the instruction of https://gramine.readthedocs.io/projects/gsc/en/latest/#example
but in the step5, ./gsc sign-image python enclave-key.pem
, I got an error :
gdpmobile11@gdpmobile11-NUC7PJYH:~/gsc$ ./gsc sign-image python enclave-key.pem
Signing graminized Docker image `unsigned_image_name` -> `gsc-python`...
Step 1/6 : FROM gsc-python-unsigned as unsigned_image
---> 5041b986b058
Step 2/6 : COPY gsc-signer-key.pem /gsc-signer-key.pem
---> c35adcb6309b
Step 3/6 : RUN export PYTHONPATH="${PYTHONPATH}:$(find /gramine/meson_build_output/lib -type d -path '*/site-packages')" && gramine-sgx-sign --key /gsc-signer-key.pem --manifest /entrypoint.manifest --output /entrypoint.manifest.sgx
---> Running in cba6f460254a
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 9, in <module>
import click
ModuleNotFoundError: No module named 'click'
Removing intermediate container cba6f460254a
Failed to build a signed graminized Docker image `gsc-python`.
I have modify the image python and make sure it has click. Now I don't know how to solve the problem.
PR gramineproject/gramine#801 removes loader.argv0_override
.
So when this PR is merged and the next Gramine release comes along, we should remove it from GSC:
And test extensively whether it still works fine.
We are unable to install python3-pyelftools
on CentOS and getting below error message.
No match for argument: python3-pyelftools
Error: Unable to find a match: python3-pyelftools
When tried installing on bare-metal, we found that this package is provided by epel.repo
by the following command
sudo yum list installed | grep pyelf
python3-pyelftools.noarch 0.27-5.el8 @epel
Step 3/6 : RUN export PYTHONPATH="${PYTHONPATH}:$(find /gramine/meson_build_output/lib64 -type d -path '*/site-packages')" && gramine-sgx-sign --key /gramine/app_files/gsc-signer-key.pem --manifest /gramine/app_files/entrypoint.manifest --output /gramine/app_files/entrypoint.manifest.sgx
---> Running in 819acb7b78ae
Traceback (most recent call last):
File "/gramine/meson_build_output/bin/gramine-sgx-sign", line 11, in <module>
from graminelibos import (
File "/gramine/meson_build_output/lib64/python3.6/site-packages/graminelibos/__init__.py", line 25, in <module>
from .sgx_sign import get_tbssigstruct, sign_with_local_key, SGX_LIBPAL, SGX_RSA_KEY_PATH
File "/gramine/meson_build_output/lib64/python3.6/site-packages/graminelibos/sgx_sign.py", line 19, in <module>
import elftools.elf.elffile
ModuleNotFoundError: No module named 'elftools'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.