metalbear-co/mirrord

Connect your local process and your cloud environment, and run local code in cloud conditions.

mirrord's Issues

Update README for v2.0

Specify custom names for agent image in E2E tests

The agent image once built is tagged as test:latest, this results in an Elapsed error if the image is not loaded to minikube by any chance, so we should not be hardcoding this. It would be nice to be able to specify the image as an environment variable and read from it, and do the same in the CI.

After about 10 minutes of idle time, mirrord-layer starts reading empty messages

Bug Description

This isn't the result of the agent terminating, as logs indicate it's still alive when that happens.

Steps to Reproduce

Start a process wrapped by mirrord-layer and wait for about 10 minutes.

Backtrace

No response

Your operating system and version

MacOS

Your Node version (node --version)

17.6.0

Additional Info

No response

Remove sleeps from E2E tests

The node API creates a file in cwd on PUT and deletes the file on DELETE request. But to assert the file created exists, we are using a sleep which is not desirable. We need to add a watch which keeps track of create/remove/write filesystem events.
https://docs.rs/notify/latest/notify/ should be helpful.

v2.0 Test Plan

1. Core (Layer + Agent)

1. Happy Flow (Run for Node.js, FastAPI)
   1. Run a process that prints data received with the mirrord env variables
   2. Send three different requests to the impersonated pod
   3. For each request, assert that the correct data was printed
   4. Stop the process
   5. After 30 seconds, assert that the job and agent were both deleted from the cluster
2. Wrong Pod
   1. 1.1.a
   2. Send a request to a different pod in the cluster
   3. Assert that the request wasn’t printed
3. Agent Namespace
   1. Happy Flow
      1. Create a new k8s namespace (call it ‘test-namespace’)
      2. Run a process that prints data received with the mirrord env variables and MIRRORD_AGENT_NAMESPACE=test-namespace
      3. Assert that the job and the agent were created in the cluster under ‘test-namespace’
      4. 1.b-1.c
   2. Nonexistent namespace
      1. Run a process that prints data received with the mirrod env variables and MIRRORD_AGENT_NAMESPACE=
      2. Assert that the process crashes with an indicative error
      3. Assert that a new job and agent weren’t created
4. Pod Namespace
   1. Happy Flow
      1. Create a new k8s namespace (call it ‘test-namespace’)
      2. Create a new nginx pod under test-namespace
      3. Run a process that prints data received with the mirrord env variables, IMPERSONATED_POD_NAME set to the name of the nginx pod, and IMPERSONATED_POD_NAMESPACE set to the name of the nginx namespace
      4. 1.1.b-1.1.c
   2. Nonexistent namespace
      1. Run a process that prints data received with the mirrord env variables and IMPERSONATED_POD_NAMESPACE set to a nonexistent namespace
      2. Assert that the process crashed with an indicative error
      3. Assert that a new job and agent weren’t created

2. CLI

1. Happy Flow (run for Node.js, FastAPI)
   1. Using the CLI, run a process that prints data received
   2. 1.1.b-1.1.e

3. VSCode Extension

1. Happy Flow (Run for Node.js, FastAPI. Test at least one on WSL)
   1. Click ‘enable mirrord’
   2. Set a breakpoint in the debugged process
   3. Start debugging the process
   4. Send a request to the impersonated pod
   5. Assert the breakpoint is hit with the correct data
2. Repeat use
   1. 3.1.a-3.1.e
   2. Stop debugging the process
   3. Click ‘disable mirrord’
   4. 3.1.c-3.1.d
   5. Assert the breakpoint wasn’t hit
   6. 3.1.a-3.1.e

mirrord-cli : reqwest dependency missing "blocking" feature

Bug Description

mirrord-cli uses the "blocking" feature of reqwest:

mirrord/mirrord-cli/src/main.rs

Line 193 in 255fbe5

if let Ok(client) = reqwest::blocking::Client::builder().build() {

but does not enable that feature in its toml file

mirrord/mirrord-cli/Cargo.toml

Line 26 in 255fbe5

reqwest = "0.11.10"

should look something like this (docs):

reqwest = { version = "0.11.10", features = ["blocking"] }

Steps to Reproduce

1. Clone main branch (as of this commit)
2. Open project in dev container (I'm using VSCode remote containers extension)
3. run cargo check or cd mirrord-cli && cargo check
4. 💥 (see backtrace)

Backtrace

if let Ok(client) = reqwest::blocking::Client::builder().build() {
   |                         ^^^^^^^^ could not find `blocking` in `reqwest`

Relevant Logs

No response

Your operating system and version

local: macOS Monterey version 12.4, container: Debian OS variant buster

Local process

N/A

Local process version

No response

Additional Info

If you have already been working with the source code it may be necessary to rebuild your container to simulate this issue.

I'm not sure if I would call it "cached", but after I solved this issue once "in passing" i was unable to make it happen again regardless of cargo clean ext.

Traffic steal

Allow user to "steal" incoming traffic from the remote pod.
First version of this feature will steal all incoming traffic.

Change e2e to use a "real app"

Right now the E2E uses something we found online (http echo) but that means that the TCP/HTTP session behavior can be different leading to odd reactions.
I think building our own docker image for the test would be better. It should be something that accepts the same APIs that our tests use, have files that we expect to have, etc.

Long story:
Right now we work on TCP level, and therefore if the application running locally and the application running remotely are different, one can close the TCP connection for each HTTP request and the other can reuse it.
By using the same app we eliminate randomness.

Add ability to specify container name to impersonate

Today, we pick up the first container in the pod we impersonate. This might not be the case for service mesh setups or more elaborate setups, so we need to let the user specify container name and maybe add a warning when we just assume and use the first one.
Also, we should consider ignoring linkerd/envoy containers by default (unless specified?)
Edit:

Status:

• @camerondurham added basic functionality controlled by env var.
• Need to expose that functionality from the cli
• ^^ VS Code Extension

sockets fd isn't removed on close

Bug Description

We currently don't hook close, meaning that if a program runs close on a socket we manage we don't delete it from our managed fds.

Steps to Reproduce

N/A

Backtrace

No response

Your operating system and version

N/A

Your Node version (node --version)

N/A

Additional Info

No response

Enabling toggling different features via configuration

We should let users turn different features on and off - i.e. if they only want incoming traffic but not file reads, etc.

Code sign our binaries

Code sign our binaries for Apple (maybe for Linux as well?)

Support emulated processes on M1 in the VS Code extension

Emulated processes are currently only supported in the CLI (see PR #122). We need to add support to the VS Code extension as well.

VSCode extension doesn't list pods outside the default namespace

Description:
I went to test the extension with an app written in Golang which runs outside of my cluster's default namespace. After clicking on Start mirrord in the status bar, only pods placed in the default namespace are listed. Because of this, I wasn't able to proceed with the tests.

Notes:
I'm not sure what useful information I could add here since the debugging mode never got to receive forwarded connections. But feel free to ask anything for I'm willing to help =)

Incompatible architecture (have 'arm64', need 'x86_64)

Bug Description

Mirrord v2.1.0 fails to mirror traffic on Mac M1 due to an incompatible mach-o file.

Steps to Reproduce

1. curl -fsSL https://raw.githubusercontent.com/metalbear-co/mirrord/main/scripts/install.sh | bash
2. mirrord exec node app.js -n my-app -p my-app-8vqpp

Backtrace

dyld[39680]: terminating because inserted dylib '/var/folders/my/c7grkbzs6v54vhcxhszg15cw0000gn/T/libmirrord_layer.dylib' could not be loaded: tried: '/var/folders/my/c7grkbzs6v54vhcxhszg15cw0000gn/T/libmirrord_layer.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/libmirrord_layer.dylib' (no such file), '/usr/lib/libmirrord_layer.dylib' (no such file), '/private/var/folders/my/c7grkbzs6v54vhcxhszg15cw0000gn/T/libmirrord_layer.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/libmirrord_layer.dylib' (no such file), '/usr/lib/libmirrord_layer.dylib' (no such file)
dyld[39680]: tried: '/var/folders/my/c7grkbzs6v54vhcxhszg15cw0000gn/T/libmirrord_layer.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/libmirrord_layer.dylib' (no such file), '/usr/lib/libmirrord_layer.dylib' (no such file), '/private/var/folders/my/c7grkbzs6v54vhcxhszg15cw0000gn/T/libmirrord_layer.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/libmirrord_layer.dylib' (no such file), '/usr/lib/libmirrord_layer.dylib' (no such file)
[1]    39680 abort      mirrord exec node app.js -n my-app -p my-app-8vqpp

Your operating system and version

OSX 12.3.1

Your Node version (node --version)

v12.22.12

Additional Info

No response

Support MIRRORD_AGENT_TTL as an argument in the CLI

It's currently only usable as an environment variable.

Use remote service environment variables

When a process is ran using mirrord, we want to be able to set it's environment variables to mirror the remote service one's.

Proposed UX

1. Configuration called REMOTE_ENV - boolean, default true.
2. OVERRIDE_ENV - list of key-(optional)value of environment variables. if value is not provided, use the "original" environment variable.

Implementation

There are various ways we can implement it - as part of taking care of this issue, those ways should be considered (feel free to suggest more)

K8S API

Obtain environment variables via K8S API (as part of creating the agent).

Pros:

1. Doesn't need any change in the agent + agent protocol.
   Cons:
2. if environment is changed post-execution or not via k8s-aware ways, we won't be able to track it.

Agent API

Obtain the environment variables via adding such api in the agent protocol.

Pros:

1. Environment variables will reflect better - this is exactly how the env is being set in the remote process
2. if env is changed during runtime, we can get updates (needs finer implementation, but this creates a path)

Cons:
?

I think Agent API will be better, but when we start working on this we should get to a decision.

Hooking getpeername and getsockname doesn't work in certain setups

Bug Description

When trying to hook getpeername and getsockname, we get the following error:
thread '<unnamed>' panicked at 'called Result::unwrap() on an Err value: InterceptorBadSignature', mirrord-layer/src/sockets.rs:547:5

I've changed it to use try_hook instead of hook to bypass the error. There's no effect on core functionality when this happens, but we don't correctly display the source address of incoming requests.

@aviramha suggested that it might be related to this issue:
frida/frida-gum#273

Steps to Reproduce

Run mirrord on the attached devcontainer. Note that you need to install kubectl on it for it to work
devcontainer.json.zip
.

Backtrace

No response

Your operating system and version

N/A

Your Node version (node --version)

N/A

Additional Info

No response

Handle Job/Pod creation errors in mirrord-layer

The Job/Pod creation in mirrord-layer while creating the agent could result in errors mentioned here https://docs.rs/kube/latest/kube/enum.Error.html and we should handle them instead of a panic.

Add support for docker runtime

We currently only support Kubernetes clusters using the containerd runtime. We need to add support for docker runtime as well. The containerd-specific code is in the agent where we get the linux namespace from the pod name, and in the pod creation in the layer.

CLI Error

After installing the mirrord CLI, It didn't turn available through my terminal.

# Installed the CLI
$ npm install -g mirrord
...

# Checked where it have been installed
$ which mirrord
/home/tom/.nvm/versions/node/v17.6.0/bin/mirrord

# Attempted to use it
$ mirrord -v
/usr/bin/env: ‘npx ts-node’: No such file or directory
/usr/bin/env: use -[v]S to pass options in shebang lines

# Checked for ts-node
$ which ts-node
/home/tom/.nvm/versions/node/v17.6.0/bin/ts-node

# Confirmed that is available
$ npx ts-node
>

If further information is required for your investigation, don't hesitate to ask me for it.

Support ls

Our files feature currently support most of the relevant syscalls, but we still need to add support for ls.

After #904, this is what is left to be done (prioritized)

• Go Linux x86-64 (same flow/integration test should work~)
• Python list directory
• Node list directory
• ls binary - macOS
• ls binary - Linux
• All of the libc functions:
  • opendir
  • fdopendir
  • readdir
  • readdir_r
  • telldir
  • seekdir
  • rewinddir
  • closedir
  • dirfd

I think the best way would be to drop each as separate PRs, first few tasks will resolve most of the libc functions :)

Remove unwrap and handle errors if the impersonated pod doesn't exist

If the pod being impersonated does not exist, mirrord panics like so:

mehula@mehul-machine:~/mirrord$ MIRRORD_AGENT_RUST_LOG=trace RUST_LOG=debug target/debug/mirrord exec -c  --pod-name http-echo-deployment-77fddcdc49-6z22r  node tests/node-e2e/app.js 
2022-06-29T05:04:30.329750Z DEBUG reqwest::connect: starting new connection: https://version.mirrord.dev/    
2022-06-29T05:04:30.330708Z DEBUG hyper::client::connect::dns: resolving host="version.mirrord.dev"
2022-06-29T05:04:30.387242Z DEBUG hyper::client::connect::http: connecting to 34.111.73.222:443
2022-06-29T05:04:30.388883Z DEBUG hyper::client::connect::http: connected to 34.111.73.222:443
2022-06-29T05:04:30.394047Z DEBUG hyper::proto::h1::io: flushed 106 bytes
2022-06-29T05:04:30.924305Z DEBUG hyper::proto::h1::io: parsed 9 headers
2022-06-29T05:04:30.924371Z DEBUG hyper::proto::h1::conn: incoming body is content-length (5 bytes)
2022-06-29T05:04:30.924462Z DEBUG hyper::proto::h1::conn: incoming body completed
2022-06-29T05:04:30.924731Z DEBUG hyper::client::pool: pooling idle connection for ("https", version.mirrord.dev)
2022-06-29T05:04:30.924845Z DEBUG reqwest::async_impl::client: response '200 OK' for https://version.mirrord.dev/get-latest-version?source=2&currentVersion=2.3.0    
2022-06-29T05:04:30.930301Z  INFO mirrord: Launching "node" with arguments ["tests/node-e2e/app.js"]
2022-06-29T05:04:31.844534Z DEBUG mirrord: Extracted library file to "/tmp/libmirrord_layer.so"
2022-06-29T05:04:33.562920Z  INFO mirrord_layer: Initializing mirrord-layer!
2022-06-29T05:04:33.571842Z  WARN mirrord_layer::pod_api: Accepting invalid certificates
2022-06-29T05:04:33.574340Z DEBUG tower::buffer::worker: service.ready=true processing request
2022-06-29T05:04:33.574613Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: kube_client::client::builder: requesting
2022-06-29T05:04:33.575082Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: hyper::client::connect::http: connecting to 192.168.49.2:8443
2022-06-29T05:04:33.575689Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: hyper::client::connect::http: connected to 192.168.49.2:8443
2022-06-29T05:04:33.575911Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::hs: No cached session for IpAddress(192.168.49.2)    
2022-06-29T05:04:33.576378Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::hs: Not resuming any session    
2022-06-29T05:04:33.579170Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::hs: Using ciphersuite TLS13_AES_128_GCM_SHA256    
2022-06-29T05:04:33.579261Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::tls13: Not resuming    
2022-06-29T05:04:33.579974Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::tls13: TLS1.3 encrypted extensions: []    
2022-06-29T05:04:33.580056Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::hs: ALPN protocol is None    
2022-06-29T05:04:33.580187Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::tls13: Got CertificateRequest CertificateRequestPayloadTLS13 { context: , extensions: [Unknown(UnknownExtension { typ: StatusRequest, payload:  }), Unknown(UnknownExtension { typ: SCT, payload:  }), SignatureAlgorithms([RSA_PSS_SHA256, ECDSA_NISTP256_SHA256, ED25519, RSA_PSS_SHA384, RSA_PSS_SHA512, RSA_PKCS1_SHA256, RSA_PKCS1_SHA384, RSA_PKCS1_SHA512, ECDSA_NISTP384_SHA384, ECDSA_NISTP521_SHA512, RSA_PKCS1_SHA1, ECDSA_SHA1_Legacy]), AuthorityNames([3015311330110603550403130a6d696e696b7562654341, 3019311730150603550403130e66726f6e742d70726f78792d6361])] }    
2022-06-29T05:04:33.580331Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: rustls::client::common: Attempting client auth    
2022-06-29T05:04:33.582780Z DEBUG hyper::proto::h1::io: flushed 111 bytes
2022-06-29T05:04:33.583048Z DEBUG rustls::client::tls13: Ticket saved    
2022-06-29T05:04:33.586225Z DEBUG hyper::proto::h1::io: parsed 7 headers
2022-06-29T05:04:33.586286Z DEBUG hyper::proto::h1::conn: incoming body is content-length (242 bytes)
2022-06-29T05:04:33.586400Z DEBUG hyper::proto::h1::conn: incoming body completed
2022-06-29T05:04:33.586633Z DEBUG HTTP{http.method=GET http.url=https://192.168.49.2:8443/api/v1/namespaces/default/pods/http-echo-deployment-77fddcdc49-6z22r otel.name="get" otel.kind="client"}: hyper::client::pool: pooling idle connection for ("https", 192.168.49.2:8443)
2022-06-29T05:04:33.587133Z DEBUG kube_client::client: Unsuccessful: ErrorResponse { status: "Failure", message: "pods \"http-echo-deployment-77fddcdc49-6z22r\" not found", reason: "NotFound", code: 404 }
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Api(ErrorResponse { status: "Failure", message: "pods \"http-echo-deployment-77fddcdc49-6z22r\" not found", reason: "NotFound", code: 404 })', mirrord-layer/src/pod_api.rs:27:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
Aborted (core dumped)

https://github.com/metalbear-co/mirrord/blob/main/mirrord-layer/src/pod_api.rs#L27
perhaps we should convert the return value to a Result type and use ? with necessary changes on all ends.

Windows Support

Currently, we support only macOS and Linux.
Adding Windows support means researching and implementing the hook layer & abstractions for Windows API.

Please upvote so we know if this feature is needed.

Reduce compile time

Right now compile time is quite long, and we don't have that much code.
Use this issue to track possible performance gains.
Concrete issues that can be worked on:

• #223
• #222
• #221
• #220

Add flag for file-worker in agent and re-enable the blackbox test

The test https://github.com/metalbear-co/mirrord/blob/main/mirrord-agent/tests/blackbox.rs#L23 is currently being ignored, a fix would be to enable/disable file operations through a flag

IntelliJ Extension

We want to have similar functionality and user experience in the IntelliJ extension as we have for VSCode.
We haven't researched how the API looks for IntelliJ but in general it should let the developer run the program with mirrord, and let it select pods in a nice way.

Remote network support

Add support for accessing network resources from the remote service context.
This includes dns resolving.

This will add an option that once enabled, all outgoing traffic will be sent from the remote agent context instead of the local context.

Some example use cases:

• User wants their local process to make an API call to another service in the k8s cluster. When the local process makes a request to another service, mirrord intercepts it and executes it through the agent instead. In the future we'd probably want very granular configuration here - so e.g. be able to toggle the feature only for certain destination hostnames, or even certain HTTP methods/parameters. This isn't part of the current requirements but might be worth thinking about when designing the feature.
• User wants their local process to read data from a database on the k8s cluster. When the local process makes a request to the database, mirrord should intercept it and execute it through the agent instead. Note that this would often rely on environment variables being overridden with their remote values, so that the local process would have the correct connection string.
• Users wants their local process to make an API call to an external API accessible from the k8s cluster (e.g. a hosted database).

Sometimes mirrord agent pods aren't deleted

In some cases, the pod that mirrord creates isn't deleted and lives forever in the cluster (until deleted manually).

Can't execute

Bug Description

mirrord-cli in release (downloaded from the script/compiled binary) gets killed when executing the process on macOS

Steps to Reproduce

1. download release build
2. mirrord exec --pod-name test-pod python
3. kill

Backtrace

no backtrace :(

Relevant Logs

No response

Your operating system and version

21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000 arm64

Local process

python

Local process version

No response

Additional Info

No response

Support ip certificates

After we transitioned into using RustTLS in #33 we found out that RustTLS doesn't work with IP hosts, hence causing mirrord to not work in cases like minikube, which is required for #35.
Solution seems to be using https://github.com/rustls/rustls-native-certs + building kube rs client and adding those certificates.

Adapt VSCode extension to new mirrord

Change the VSCode extension to work with the new #22

When running with VSCode's Node debugger, mirrord hooks and breaks an internal socket

Bug Description

VSCode's Node debugger uses a socket for some internal mechanism. Mirrord hooks it, breaking the flow. We should be able to overlook these sockets.

Steps to Reproduce

Debug a node process in VSCode with mirrord-layer in LD_PRELOAD.

Backtrace

No response

Your operating system and version

MacOS

Your Node version (node --version)

v17.6.0

Additional Info

No response

Add detours for dup/dup2/fcntl system calls

When mirrord exec is called with a simple flask API (https://gist.github.com/infiniteregrets/ba753ffb53ff8b98b2dd8f5051323919), where port 80/any other port is explicitly specified, the application seems to ignore that and instead assign a different port. This could perhaps be due to a call to dup or fcntl with F_DUPFD/F_DUPFD_CLOEXEC flags where mirrord is not aware of the new sockfd and probably getsockname returns a different port to the flask app. So we might need to add detours for the mentioned system calls to bring new sockfds to mirrord book-keeping.
More context:
Strace of flask API: https://gist.github.com/infiniteregrets/09773cc3f1aee7983f95fc800ff8d969
Agent logs when enforcing port 3689 in the Flask Application: https://gist.github.com/infiniteregrets/7c42aeae359e3a55c756f2ba9c7e1213

Add auto-update

When starting the CLI or the extension, they should check whether a new version of mirrord is available, and if it is, prompt the user to update.

Package the VSCode extension per platform in the CI

Since the VSCode extension now includes the mirrord-layer binary, it should be packaged separately (using vsce's --target argument) for MacOS and Linux.

Add macOS to e2e matrix

Refactor to support next features

Right now, mirrord can only duplicate traffic from the remote cluster.
We want mirrord to be able to run a process in the context of the remote environment - granting the process access to remote resources (files, environment variables and networking).
In order to accomplish that, we're re-writing mirrord with a new design.
The new design is implemented using hooking syscalls via loading our "so" or "dylib" to the debugged process.
The first version will include same functionality as old mirrord (duplicating traffic) but using the new method.

mirrord extract panics on non-existent directories

mehula@mehul-machine:~/mirrord$ target/debug/mirrord extract blah 
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', mirrord-cli/src/main.rs:70:52
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

It would be nicer to handle the Result type gracefully and return an error or maybe prompt the user whether they want to create the directory (only useful for CLI users)

Tracking issue to make functionality common between file and network ops

Functions like dup/dup2, fcntl, read, write etc. should be common for file and network operations and this tracking issue will be used to keep track of the progress till functionality between hooks is merged

Add support for using ephemeral containers

https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/

We should use these for the mirrord-agent instead of jobs with TTL=0. Since they're currently in beta, they should be implemented with a feature flag.

Revise issue templates

Right now the Bugfix template doesn't match the new design of mirrord (asks for node version) so we need to think what needs to be collected in case of a bug.
I'd assume:

1. distro/os
2. binary being tried to run
3. version of binary
4. hash of it?

instructions to run with debug and print trace?
Feel free to add more stuff a bug report (or other issue templates) should cover.

Add miri to CI

From miri description:

An experimental interpreter for Rust's mid-level intermediate representation (MIR). It can run binaries and test suites of cargo projects and detect certain classes of undefined behavior,

I think this is a good tool for us to have on CI.

mirrord-layer deadlocks if accept is called when there's no data

Bug Description

When accept is called before a connection is read from the agent, it locks the connections struct and there's a deadlock.

Steps to Reproduce

Simulate a delay so that accept is called before a connection is read from the agent.

Backtrace

No response

Your operating system and version

MacOS

Your Node version (node --version)

N/A

Additional Info

No response

Remote file support

Add support for accessing remote files.

Add a new setting that enables/disables (enabled by default) remote files. When enabled, each file opened is bridged to a remote file on the agent.

Add E2E tests

Add a suite of E2E tests to test the integration between the layer and the agent.

mirrord-layer panics without any relevant info if the agent exits

If mirrord-agent exits i.e. if layer reaches its timeout before the agent even started, mirrord-layer panics with an Elapsed error without giving any context what went wrong. It would be nice to remove the unwrap here: https://github.com/metalbear-co/mirrord/blob/main/mirrord-layer/src/pod_api.rs#L163 and handle the error gracefully providing good context and logs.

Add vscode ignore to VSCode extension

As our CI complains:

 This extension consists of 5252 files, out of which 2627 are JavaScript files. For performance reasons, you should bundle your extension: https://aka.ms/vscode-bundle-extension . You should also exclude unnecessary files by adding them to your .vscodeignore: https://aka.ms/vscode-vscodeignore

Contributions welcome!

Cannot test file-ops with mirrord VSCode extension

Bug Description

mirrord's VSCode extension fails to debug the local process.

Steps to Reproduce

• Add this configuration to your launch.json:

 {
      "type": "lldb",
      "request": "launch",
      "name": "ops.py: Debug file ops",
      "program": "python3",
      "args": ["${workspaceFolder}/tests/python-e2e/ops.py"],
      "cwd": "${workspaceFolder}",
      "sourceMap": {},      
      "env": {
        "DYLD_INSERT_LIBRARIES": "${workspaceFolder}/target/debug/libmirrord_layer.dylib",
        "LD_PRELOAD": "${workspaceFolder}/target/debug/libmirrord_layer.so",
        "RUST_LOG": "warn,mirrord=debug",
        "MIRRORD_AGENT_IMPERSONATED_POD_NAME": "http-echo-deployment-64887f7cdb-4lwrs",
        "MIRRORD_AGENT_RUST_LOG": "warn,mirrord=debug",
        "MIRRORD_FILE_OPS": "true",
        "RUST_BACKTRACE": "short", 
        "MIRRORD_ACCEPT_INVALID_CERTIFICATES": "true",
        "MIRRORD_AGENT_IMAGE": "runtime"   
      }
    }

• Start debugging and choose the impersonated pod

Backtrace

2022-06-20T17:26:58.258050Z  INFO mirrord_layer: Initializing mirrord-layer!
2022-06-20T17:26:58.259302Z  WARN mirrord_layer::pod_api: Accepting invalid certificates
2022-06-20T17:26:59.597035Z DEBUG mirrord_layer::sockets: getpeername hooked
2022-06-20T17:26:59.597069Z DEBUG mirrord_layer::sockets: getsockname hooked
2022-06-20T17:26:59.597090Z DEBUG mirrord_layer::sockets: accept4 hooked
2022-06-20T17:26:59.597106Z DEBUG mirrord_layer::sockets: dup3 hooked
2022-06-20T17:26:59.597122Z DEBUG mirrord_layer::sockets: accept hooked
FFF
======================================================================
FAIL: test_lseek (__main__.FileOpsTest)
Seeks character by character in a file with Lorem Ipsum text in "/tmp" and verifies the concatenation of the text.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mehula/mirrord/tests/python-e2e/ops.py", line 27, in test_lseek
    self.assertFalse(self._check_path_exists_on_host(file_path))
AssertionError: True is not false

======================================================================
FAIL: test_openat (__main__.FileOpsTest)
Opens a directory i.e. "/tmp", then opens a file in temp using openat give the directory file desciptor for "/tmp".
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mehula/mirrord/tests/python-e2e/ops.py", line 43, in test_openat
    self.assertFalse(self._check_path_exists_on_host(file_path))
AssertionError: True is not false

======================================================================
FAIL: test_read_write_family (__main__.FileOpsTest)
Reads data from a file in "/tmp" and verifies the text expected is the same as the text written.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mehula/mirrord/tests/python-e2e/ops.py", line 14, in test_read_write_family
    self.assertFalse(self._check_path_exists_on_host(file_path))
AssertionError: True is not false

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=3)
mehula@mehul-machine:~/mirrord$ kubectl get pods
NAME                                    READY   STATUS    RESTARTS     AGE
http-echo-deployment-64887f7cdb-4lwrs   1/1     Running   7 (6d ago)   6d4h
mirrord-agent-kpjldmsafe-2zrr4          1/1     Running   0            3s
mirrord-agent-ykgeipboyw-2g4tj          1/1     Running   0            4s
mehula@mehul-machine:~/mirrord$ kubectl logs mirrord-agent-kpjldmsafe-2zrr4
2022-06-20T17:27:00.470051Z DEBUG mirrord_agent: main -> Initializing mirrord-agent.
2022-06-20T17:27:00.470269Z DEBUG mirrord_agent::sniffer: packet_worker -> setting namespace
2022-06-20T17:27:00.470333Z DEBUG mirrord_agent::file: file_worker -> Setting namespace
2022-06-20T17:27:00.471149Z DEBUG mirrord_agent::runtime: set_namespace -> fd 12
2022-06-20T17:27:00.471166Z DEBUG mirrord_agent::sniffer: preparing sniffer
2022-06-20T17:27:00.471169Z DEBUG mirrord_agent::sniffer: prepare_sniffer -> Preparing interface.
2022-06-20T17:27:00.594518Z DEBUG mirrord_agent: start -> Connection accepted from 127.0.0.1:51646
2022-06-20T17:27:01.597675Z DEBUG mirrord_agent: peer_handler -> daemon_stream.next received a message None
2022-06-20T17:27:01.597730Z DEBUG mirrord_agent: start_agent -> Peer 0 closed
2022-06-20T17:27:01.597750Z DEBUG mirrord_agent: ClientMessage::Close -> peer id 0 sent close
2022-06-20T17:27:01.597779Z DEBUG mirrord_agent::sniffer: packet_worker -> setting ports []
2022-06-20T17:27:01.597783Z DEBUG mirrord_agent::sniffer: packet_worker -> empty ports, setting dummy bpf
mehula@mehul-machine:~/mirrord$ kubectl logs mirrord-agent-ykgeipboyw-2g4tj
2022-06-20T17:26:59.111089Z DEBUG mirrord_agent: main -> Initializing mirrord-agent.
2022-06-20T17:26:59.111252Z DEBUG mirrord_agent::sniffer: packet_worker -> setting namespace
2022-06-20T17:26:59.111253Z DEBUG mirrord_agent::file: file_worker -> Setting namespace
2022-06-20T17:26:59.112627Z DEBUG mirrord_agent::runtime: set_namespace -> fd 12
2022-06-20T17:26:59.112638Z DEBUG mirrord_agent::sniffer: preparing sniffer
2022-06-20T17:26:59.112641Z DEBUG mirrord_agent::sniffer: prepare_sniffer -> Preparing interface.
2022-06-20T17:26:59.593772Z DEBUG mirrord_agent: start -> Connection accepted from 127.0.0.1:51642
2022-06-20T17:27:01.634394Z DEBUG mirrord_agent: peer_handler -> daemon_stream.next received a message None
2022-06-20T17:27:01.634527Z DEBUG mirrord_agent: ClientMessage::Close -> peer id 0 sent close
2022-06-20T17:27:01.634581Z DEBUG mirrord_agent::sniffer: packet_worker -> setting ports []
2022-06-20T17:27:01.634591Z DEBUG mirrord_agent::sniffer: packet_worker -> empty ports, setting dummy bpf
2022-06-20T17:27:01.634536Z DEBUG mirrord_agent: start_agent -> Peer 0 closed

### Relevant Logs

_No response_

### Your operating system and version

mehula@mehul-machine:~/mirrord$ uname -a Linux mehul-machine 5.13.0-1029-azure #34~20.04.1-Ubuntu SMP Thu Jun 9 12:37:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

### Local process

python3 tests/python-e2e/ops.py

### Local process version

mehula@mehul-machine:~/mirrord$ python3 -V Python 3.8.10

### Additional Info

_No response_

Add E2E tests for file operations

With 2.2.0 release, we need e2e tests for the following functions:

• open
• fopen
• fdopen
• openat
• read
• fread
• fileno
• lseek
• write