Giter Site home page Giter Site logo

proxy-wasm / proxy-wasm-cpp-sdk Goto Github PK

View Code? Open in Web Editor NEW
137.0 17.0 63.0 12.25 MB

WebAssembly for Proxies (C++ SDK)

License: Apache License 2.0

Starlark 1.01% Makefile 0.11% Shell 0.23% C++ 97.33% C 1.12% JavaScript 0.20%
proxy wasm webassembly envoy envoy-proxy

proxy-wasm-cpp-sdk's Introduction

WebAssembly for Proxies (C++ SDK)

Build Status Apache 2.0 License

The SDK has dependencies on specific versions of the C++ WebAssembly toolchain Emscripten (https://emscripten.org) and the protobuf library, therefor use of a Docker image is recommended.

Docker

A Dockerfile for the C++ SDK is provided in Dockerfile-sdk.

It can built in this directory by:

docker build -t wasmsdk:v2 -f Dockerfile-sdk .

The docker image can be used for compiling wasm files.

Creating a project for use with the Docker build image

Create a directory with your source files and a Makefile:

PROXY_WASM_CPP_SDK=/sdk

all: myproject.wasm

include ${PROXY_WASM_CPP_SDK}/Makefile.base_lite

Source file (myproject.cc):

#include <string>
#include <unordered_map>

#include "proxy_wasm_intrinsics.h"

class ExampleContext : public Context {
public:
  explicit ExampleContext(uint32_t id, RootContext* root) : Context(id, root) {}

  FilterHeadersStatus onRequestHeaders(uint32_t headers, bool end_of_stream) override;
  void onDone() override;
};
static RegisterContextFactory register_ExampleContext(CONTEXT_FACTORY(ExampleContext));

FilterHeadersStatus ExampleContext::onRequestHeaders(uint32_t headers, bool end_of_stream) {
  logInfo(std::string("onRequestHeaders ") + std::to_string(id()));
  auto path = getRequestHeader(":path");
  logInfo(std::string("header path ") + std::string(path->view()));
  return FilterHeadersStatus::Continue;
}

void ExampleContext::onDone() { logInfo("onDone " + std::to_string(id())); }

Compiling with the Docker build image

Run docker:

docker run -v $PWD:/work -w /work  wasmsdk:v2 /build_wasm.sh

Caching the standard libraries

The first time that emscripten runs it will generate the standard libraries. To cache these in the docker image, after the first successful compilation (e.g myproject.cc above), commit the image with the standard libraries:

docker commit `docker ps -l | grep wasmsdk:v2 | awk '{print $1}'` wasmsdk:v2

This will save time on subsequent compiles.

Using the SDK from a newer/specific version of Envoy

To use a newer/specific version of the SDK (e.g. from the version of Enovy you are going to deploy the WebAssembly module to) bind that volume and use it in the Makefile.

Here is an example Makefile referencing the SDK at ../envoy/api/wasm/cpp and mounted as 'sdk' in the /work directory:

PROXY_WASM_CPP_SDK=/work/sdk

all: myproject.wasm

include ${PROXY_WASM_CPP_SDK}/Makefile.base_lite

Run docker pointing to Envoy sources in a directory parallel (at the same level) as your project directory:

docker run -v $PWD:/work -v $PWD/../envoy/api/wasm/cpp:/work/sdk -w /work  wasmsdk:v2 bash /build_wasm.sh

Using abseil form the image

Abseil (optionally) is built in /root/abseil and can be used. Note that the abseil containers (e.g. absl::flat_hash_set) exercise many syscalls which are not supported. Consequantally individual files should be pulled in which are relatively self contained (e.g. strings). Example customized Makefile:

PROXY_WASM_CPP_SDK=/sdk
CPP_API:=${PROXY_WASM_CPP_SDK}
CPP_CONTEXT_LIB = ${CPP_API}/proxy_wasm_intrinsics.cc
ABSL = /root/abseil-cpp
ABSL_CPP = ${ABSL}/absl/strings/str_cat.cc ${ABSL}/absl/strings/str_split.cc ${ABSL}/absl/strings/numbers.cc ${ABSL}/absl/strings/ascii.cc

all: plugin.wasm

%.wasm %.wat: %.cc ${CPP_API}/proxy_wasm_intrinsics.h ${CPP_API}/proxy_wasm_enums.h ${CPP_API}/proxy_wasm_externs.h ${CPP_API}/proxy_wasm_api.h ${CPP_API}/proxy_wasm_intrinsics.js ${CPP_CONTEXT_LIB}
        ls /root
                em++ --no-entry -s EXPORTED_FUNCTIONS=['_malloc'] --std=c++17 -O3 -flto -I${CPP_API} -I${CPP_API}/google/protobuf -I/usr/local/include -I${ABSL} --js-library ${CPP_API}/proxy_wasm_intrinsics.js ${ABSL_CPP} $*.cc ${CPP_API}/proxy_wasm_intrinsics.pb.cc ${CPP_CONTEXT_LIB} ${CPP_API}/libprotobuf.a -o $*.wasm

Precompiled abseil libraries are also available, so the above can also be done as:

PROXY_WASM_CPP_SDK=/sdk
CPP_API:=${PROXY_WASM_CPP_SDK}
CPP_CONTEXT_LIB = ${CPP_API}/proxy_wasm_intrinsics.cc
ABSL = /root/abseil-cpp
ABSL_LIBS = ${ABSL}/absl/strings/libabsl_strings.a ${ABSL}/absl/strings/libabsl_strings_internal.a  ${ABSL}/absl/strings/libabsl_str_format_internal.a

all: plugin.wasm

%.wasm %.wat: %.cc ${CPP_API}/proxy_wasm_intrinsics.h ${CPP_API}/proxy_wasm_enums.h ${CPP_API}/proxy_wasm_externs.h ${CPP_API}/proxy_wasm_api.h ${CPP_API}/proxy_wasm_intrinsics.js ${CPP_CONTEXT_LIB}
        ls /root
                em++ --no-entry -s EXPORTED_FUNCTIONS=['_malloc'] --std=c++17 -O3 -flto -I${CPP_API} -I${CPP_API}/google/protobuf -I/usr/local/include -I${ABSL} --js-library ${CPP_API}/proxy_wasm_intrinsics.js  $*.cc ${CPP_API}/proxy_wasm_intrinsics.pb.cc ${CPP_CONTEXT_LIB} ${CPP_API}/libprotobuf.a ${ABSL_LIBS} -o $*.wasm

Ownership of the resulting .wasm files

The compiled files may be owned by root. To chown them add the follow lines to the Makefile and docker invocation:

PROXY_WASM_CPP_SDK=/sdk

all: myproject.wasm
  chown ${uid}.${gid} $^

include ${PROXY_WASM_CPP_SDK}/Makefile.base_lite

Invocation file (e.g. build.sh):

#!/bin/bash
docker run -e uid="$(id -u)" -e gid="$(id -g)" -v $PWD:/work -w /work wasmsdk:v2 /build_wasm.sh

Dependencies for building WASM modules:

If you do not wish to use the Docker file, the dependencies can be installed by script (sdk_container.sh), or by hand.

protobuf v3.9.1

You must install the version of protobuf on your build system that matches the libprotobuf.a files (without any patches) so that the generated code matches the .a library. Currently this is based on tag v3.9.1 of https://github.com/protocolbuffers/protobuf.

git clone https://github.com/protocolbuffers/protobuf
cd protobuf
git checkout v3.9.1
git submodule update --init --recursive
./autogen.sh
./configure
make
make check
sudo make install

emscripten

git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk update-tags
./emsdk install 3.1.7
./emsdk activate 3.1.7

source ./emsdk\_env.sh

It is possible later versions will work, e.g.

./emsdk update-tags
./emsdk install latest
./emsdk activate latest

However 3.1.7 is known to work.

Rebuilding the libprotobuf.a files

If want to rebuild the libprotobuf.a files or use a different version see the instructions at https://github.com/kwonoj/protobuf-wasm. Commit 4bba8b2f38b5004f87489642b6ca4525ae72fe7f works for protobuf v3.9.x.

git clone https://github.com/protocolbuffers/protobuf protobuf-wasm
cd protobuf-wasm
git checkout v3.9.1
git clone https://github.com/kwonoj/protobuf-wasm wasm-patches
cd wasm-patches && git checkout 4bba8b2f38b5004f87489642b6ca4525ae72fe7f && cd ..
git apply wasm-patches/*.patch
./autogen.sh
emconfigure ./configure --disable-shared CXXFLAGS="-O3 -flto"
emmake make
cd ..
cp protobuf-wasm/src/.libs/libprotobuf-lite.a ${CPP_API}/libprotobuf-lite.a
cp protobuf-wasm/src/.libs/libprotobuf.a ${CPP_API}/libprotobuf.a

WAVM binaries

git clone [email protected]:WAVM/WAVM.git
cd WAVM
cmake "."
make
sudo make install

Note: ensure /usr/local/bin is in your path

proxy-wasm-cpp-sdk's People

Contributors

110y avatar bianpengyuan avatar gbrail avatar ingwonsong avatar jplevyak avatar lizan avatar martijneken avatar masterlvng avatar mpwarres avatar piotrsikora avatar shikugawa avatar xunzhuo avatar yxue avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

proxy-wasm-cpp-sdk's Issues

[Question] Using gRPC bidi streams

Any example for how to use gRPC bidi streams from a wasm filter ? As I understand we can make gRPC calls from either Context or RootContext.

Tag v0.1.0 fails to build

Hi,

I built the wasmsdk:v2 successfully using the v0.1.0 tag. However it it fails to build the sample at:
https://github.com/envoyproxy/envoy-wasm/blob/master/examples/wasm/envoy_filter_http_wasm_example.cc

with the following error:
`Adding directories to PATH:
PATH += /root/emsdk
PATH += /root/emsdk/upstream/emscripten
PATH += /root/emsdk/node/12.18.1_64bit/bin

Setting environment variables:
EMSDK = /root/emsdk
EM_CONFIG = /root/emsdk/.emscripten
EM_CACHE = /root/emsdk/upstream/emscripten/cache
EMSDK_NODE = /root/emsdk/node/12.18.1_64bit/bin/node
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
make: Nothing to be done for 'all'.`

envoy-release/v1.15 works, however the the generated WASM fails to load in istio v1.6.8 with ABI version unsupported error.

WASM built with one revision prior to the ABI version getting bumped to 0.2.0 passes the abi version check however fails for an unsupported proxy function call.

Is there a specific revision that works with istio? Also is there a way to know which proxy hooks are included in each ABI versions?

Thanks.

WasmData::pairs() unstable

What I suspect is happening is that the output of pairs() stops being safe after the WasmData goes out of scope and its destructor calls ::free().

To reproduce, add to the doc's myproject.cc onRequestHeaders() the following code:

  auto request_headers = getHeaderMapPairs(WasmHeaderMapType::RequestHeaders)->pairs();
  for (int i = 0; i < request_headers.size(); i++) {
    auto pair = request_headers[i];
    char buffer[256]; // (I know this is unsafely short for arbitrary headers)
    sprintf(buffer, "I-A. context id: %d: header %s=%s",
      id(),
      pair.first.data(),
      pair.second.data());
    logInfo(buffer);
  }

  // Allocate a std::string from header std::string_view s
  logInfo("context id: " + std::to_string(id()) + ": first header size " + std::to_string(request_headers[0].first.size()) + "=" + std::to_string(request_headers[0].second.size()));

  for (int i = 0; i < request_headers.size(); i++) {
    auto pair = request_headers[i];
    char buffer[256];
    sprintf(buffer, "I-B. context id: %d: header %s=%s",
      id(),
      pair.first.data(),
      pair.second.data());
    logInfo(buffer);
  }

The output from the I-A loop is correct, the output from the I-B loop is corrupted.

This problem goes away if I create request_headers using two lines, keeping the WasmData in scope.

  auto request_header_pairs = getHeaderMapPairs(WasmHeaderMapType::RequestHeaders);
  auto request_headers = request_header_pairs->pairs();

WASM is not support int64_t?

I get timestamp by gettimeofday function, tv_sec * 10 is out of bound.

[2021-03-22 20:34:35.539][40124][info][wasm] [source/extensions/common/wasm/context.cc:1178] wasm log: now.tv_sec=1616470475, now.tv_usec=539285, now.tv_sec*10=-1015164434

Consolidate docker based build to use bazel

Currently the build container used for docker based build is fragile since it copies dependencies like protobuf, emscripten toolchain, and abseil etc, and needs user to write tedious make file and build scripts. Instead we should consolidate container based build to also use bazel, which means provide a docker container which has all dependencies to build wasm module with emscripten tool chain. The container could be pushed by postsubmit and used by other downstream extension development.

@PiotrSikora wdyt?

error: use of undeclared identifier 'setBuffer'

I want to replace response body using setBuffer

inline WasmResult setBuffer(WasmBufferType type, size_t start, size_t length, std::string_view data,

But when I compile my code with wasme build cpp, I get the following message

filter.cc:80:7: error: use of undeclared identifier 'setBuffer'
      setBuffer(BufferType::HttpResponseBody, 0, body_buffer_length, "replace");

Question: WasmRuntimeValues in Envoy

As envoy source code shows:

image

How can I run wasm in Wasmtime, Wavm.. instead of v8 engine in Envoy ?

I try to edit envoy config to change runtime field to wasmtime or wavm, but it seems not to work right.

Is it now Envoy just support v8 to run wasm or something else?

Building fails by giving the error: Argument list too long compilation terminated.

I have read past closed issues related to this error but nothing worked out. Can anyone please guide me to solve this problem?

ERROR: /home/<user>/.cache/bazel/_bazel_root/3ec64ea2d722c04dac7c46c2d723e18b/external/envoy/source/exe/BUILD:58:17: C++ compilation of rule '@envoy//source/exe:main_common_lib' failed (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 1447 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 1447 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
gcc: fatal error: cannot execute '/usr/lib/gcc/x86_64-linux-gnu/9/cc1plus': execv: Argument list too long
compilation terminated.
Target //:envoy failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 5.283s, Critical Path: 3.48s
INFO: 6 processes: 6 internal.
FAILED: Build did NOT complete successfully

I tried ulimit -s unlimited, updating GCC and clang to 10. Not sure why it's not working.

metric API not working

I'm testing http filter compiled with latest sdk master branch in istio-1.5.8.
The metric methods seems not working.
Here are my related code:

bool AddHeaderRootContext::onConfigure(size_t length) {
  logWarn("onConfigure");
  defineMetric(MetricType::Counter, "test_counter_config", &metric_id);
  defineMetric(MetricType::Gauge, "test_gauge", &gauge_metric);
  defineMetric(MetricType::Histogram, "test_gauge", &histogram_metric);
  return true; 
}
// ...
FilterHeadersStatus AddHeaderContext::onResponseHeaders(uint32_t) {
  logWarn(std::string("onResponseHeaders ") + std::to_string(id()));
  addResponseHeader(root_->header_name_, root_->header_value_);
  auto res = incrementMetric(root_->metric_id, 1);
  uint64_t value = 0;
  getMetric(root_->metric_id, &value);
  replaceResponseHeader("location", "envoy-wasm:" + std::to_string(int(value)));
  logWarn("incrementMetric res:" + std::to_string(int(res)) + ", current value:" + std::to_string(int(value)));
  recordMetric(root_->gauge_metric, 3);
  recordMetric(root_->histogram_metric, 1);
  return FilterHeadersStatus::Continue;
}

I see no metric in promethues, and according to log, the metric counter value is always 0 even after incrementMetric()
below are the logs:

[Envoy (Epoch 0)] [2020-08-14 09:48:38.071][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onCreate 14
[Envoy (Epoch 0)] [2020-08-14 09:48:38.071][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onRequestHeaders 14
[Envoy (Epoch 0)] [2020-08-14 09:48:38.071][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onResponseHeaders 14
[Envoy (Epoch 0)] [2020-08-14 09:48:38.071][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: incrementMetric res:0, current value:0
[Envoy (Epoch 0)] [2020-08-14 09:48:38.071][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDone 14
[Envoy (Epoch 0)] [2020-08-14 09:48:38.071][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onLog 14
[Envoy (Epoch 0)] [2020-08-14 09:48:38.071][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDelete 14
[Envoy (Epoch 0)] [2020-08-14 09:48:39.942][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onCreate 15
[Envoy (Epoch 0)] [2020-08-14 09:48:39.942][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onRequestHeaders 15
[Envoy (Epoch 0)] [2020-08-14 09:48:39.942][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onResponseHeaders 15
[Envoy (Epoch 0)] [2020-08-14 09:48:39.942][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: incrementMetric res:0, current value:0
[Envoy (Epoch 0)] [2020-08-14 09:48:39.942][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDone 15
[Envoy (Epoch 0)] [2020-08-14 09:48:39.942][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onLog 15
[Envoy (Epoch 0)] [2020-08-14 09:48:39.942][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDelete 15
[Envoy (Epoch 0)] [2020-08-14 09:48:41.250][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onCreate 16
[Envoy (Epoch 0)] [2020-08-14 09:48:41.250][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onRequestHeaders 16
[Envoy (Epoch 0)] [2020-08-14 09:48:41.250][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onResponseHeaders 16
[Envoy (Epoch 0)] [2020-08-14 09:48:41.250][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: incrementMetric res:0, current value:0
[Envoy (Epoch 0)] [2020-08-14 09:48:41.250][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDone 16
[Envoy (Epoch 0)] [2020-08-14 09:48:41.251][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onLog 16
[Envoy (Epoch 0)] [2020-08-14 09:48:41.251][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDelete 16
[Envoy (Epoch 0)] [2020-08-14 09:48:42.625][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onCreate 17
[Envoy (Epoch 0)] [2020-08-14 09:48:42.625][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onRequestHeaders 17
[Envoy (Epoch 0)] [2020-08-14 09:48:42.625][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onResponseHeaders 17
[Envoy (Epoch 0)] [2020-08-14 09:48:42.625][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: incrementMetric res:0, current value:0
[Envoy (Epoch 0)] [2020-08-14 09:48:42.626][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDone 17
[Envoy (Epoch 0)] [2020-08-14 09:48:42.626][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onLog 17
[Envoy (Epoch 0)] [2020-08-14 09:48:42.626][31][warning][wasm] [external/envoy/source/extensions/common/wasm/context.cc:1092] wasm log http-test8 add_header_root_id: onDelete 17

Compiling for different Envoy versions

Hi, I'm doing some test with the sample code and was able to build and run the wasm filter in the envoy-dev:latest image.
In my use case I'll need to deploy this in an Istio environment. So I'm trying to run it using their Envoy implementation at istio/proxy:1.7.3 repo/image.
So far I understand that Istio was 'beta' supporting wasm filters because their Envoy implementation was targeting the envoy-wasm fork (which was recently merged to envoy upstream).

When I try to run the sample using the istio container I get the following error:

[2020-10-19 21:16:41.016][1][debug][config] [external/envoy/source/extensions/filters/network/http_connection_manager/config.cc:474]     http filter #0
[2020-10-19 21:16:41.022][1][debug][filter] [src/envoy/http/authn/http_filter_factory.cc:43] Called AuthnFilterConfig : createEmptyConfigProto
[2020-10-19 21:16:41.025][1][debug][wasm] [bazel-out/k8-opt/bin/external/envoy/source/extensions/common/wasm/_virtual_includes/wasm_hdr/extensions/common/wasm/wasm_vm.h:45] WasmVm created envoy.wasm.runtime.v8 now active
[2020-10-19 21:16:41.025][1][debug][wasm] [external/envoy/source/extensions/common/wasm/wasm.cc:110] Base Wasm created 1 now active
[2020-10-19 21:16:41.079][1][error][wasm] [external/envoy/source/extensions/common/wasm/wasm.cc:127] Wasm VM failed Failed to initialize Wasm code
[2020-10-19 21:16:41.079][1][debug][wasm] [external/envoy/source/extensions/common/wasm/wasm.cc:168] ~Wasm 0 remaining active
[2020-10-19 21:16:41.081][1][debug][wasm] [bazel-out/k8-opt/bin/external/envoy/source/extensions/common/wasm/_virtual_includes/wasm_hdr/extensions/common/wasm/wasm_vm.h:49] ~WasmVm envoy.wasm.runtime.v8 0 remaining active
[2020-10-19 21:16:41.094][1][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] Listener-local-init-watcher a6a092e4-3314-4819-a610-e71c2faeaa7f destroyed
[2020-10-19 21:16:41.094][1][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] init manager Listener-local-init-manager a6a092e4-3314-4819-a610-e71c2faeaa7f 654504511291682766 destroyed
[2020-10-19 21:16:41.094][1][debug][init] [external/envoy/source/common/init/target_impl.cc:32] target Listener-init-target a6a092e4-3314-4819-a610-e71c2faeaa7f destroyed
[2020-10-19 21:16:41.096][1][critical][main] [external/envoy/source/server/server.cc:101] error initializing configuration '/home/andrey/work/iheart/github-orgs/teams/ingress/ihm-wasm-oidc-filter/test/envoy.yaml': Unable to create Wasm HTTP filter my_plugin

As mentioned in the readme file, it is possible to build the plugin using a specific envoy version. In this case I'd need to use istio/proxy version (I guess).
But, which folder should be used?

how to make http call in stream context

Hi all,
I want to do some check operation by sending outgoing http request in onRequestHeaders, just like the mixer filter
However, Context dosen't have httpCall method currently, only RootContext have.
So how could I make it?

Is there detailed API documentation

Is there detailed API documentation? Such as "proxy_wasm_externs.h" and these functions "virtual FilterStatus onNewConnection() { return FilterStatus::Continue; }
virtual FilterStatus onDownstreamData(size_t, bool) { return FilterStatus::Continue; }
virtual FilterStatus onUpstreamData(size_t, bool) { return FilterStatus::Continue; }
virtual void onDownstreamConnectionClose(CloseType) {}
virtual void onUpstreamConnectionClose(CloseType) {}"

StringView is not declared in proxy_wasm_intrinsics.h.

building this example: https://github.com/envoyproxy/envoy-wasm/tree/master/examples/wasm
proxy-wasm-cpp-sdk revision: 6892046

docker run -v $PWD:/work -w /work  wasmsdk:v2 /build_wasm.sh
Adding directories to PATH:
PATH += /root/emsdk
PATH += /root/emsdk/upstream/emscripten
PATH += /root/emsdk/node/12.18.1_64bit/bin

Setting environment variables:
EMSDK = /root/emsdk
EM_CONFIG = /root/emsdk/.emscripten
EM_CACHE = /root/emsdk/upstream/emscripten/cache
EMSDK_NODE = /root/emsdk/node/12.18.1_64bit/bin/node
em++ -s STANDALONE_WASM=1 -s EMIT_EMSCRIPTEN_METADATA=1 -s EXPORTED_FUNCTIONS=['_malloc'] --std=c++17 -O3 -flto -s WASM_OBJECT_FILES=0 --llvm-lto 1 -DPROXY_WASM_PROTOBUF_LITE=1 -I/sdk -I/usr/local/include --js-library /sdk/proxy_wasm_intrinsics.js myproject.cc /sdk/proxy_wasm_intrinsics_lite.pb.cc /sdk/struct_lite.pb.cc /sdk/proxy_wasm_intrinsics.cc /sdk/libprotobuf-lite.a -o myproject.wasm
myproject.cc:11:44: error: unknown type name 'StringView'
  explicit ExampleRootContext(uint32_t id, StringView root_id) : RootContext(id, root_id) {}
                                           ^
In file included from myproject.cc:2:
In file included from /root/emsdk/upstream/emscripten/system/include/libcxx/string:505:
In file included from /root/emsdk/upstream/emscripten/system/include/libcxx/string_view:176:
In file included from /root/emsdk/upstream/emscripten/system/include/libcxx/__string:57:
In file included from /root/emsdk/upstream/emscripten/system/include/libcxx/algorithm:644:
...

declaring something like

#define StringView std::string_view

Solves the issue.

docs have the same issue

docs/wasm_filter.md
370:void addRequestHeader(std::string_view key, StringView value)
381:void replaceRequestHeader(std::string_view key, StringView value)
440:void addResponseHeader(std::string_view key, StringView value)
451:void replaceResponseHeader(std::string_view key, StringView value)
510:void addRequestTrailer(std::string_view key, StringView value)
521:void replaceRequestTrailer(std::string_view key, StringView value)
578:void addResponseTrailer(std::string_view key, StringView value)
589:void replaceResponseTrailer(std::string_view key, StringView value)
1158:std::vector<std::pair<std::string_view, StringView>> pairs()

How to retrieve downstream local and remote address?

Does wasm cpp have api to retrieve download local and remote address?

For example in lua http filter, request_handle:streamInfo():downstreamLocalAddress(), request_handle:streamInfo():downstreamLocalAddress(), can be used to get such addresses.

Example's root_id

The file README, line 47 registers the example with no root_id parameter. I assume this picks up the default argument, "", but I couldn't get the example working without a root ID. (I was using Istio EnvoyFilter to configure, perhaps mishandling an empty root_id is an Istio flaw, but it was confusing not to see a root ID in the example.

Consider running the spell checker on the docs (e.g. "from the version of Enovy you are going to deploy")

Question: Why don`t we store pre-built image in registry like docker hub?

From what I have tried, the process of building docker image takes so long, and as other issues said, it`s possible to go wrong when building image.

Is there any problem to push a pre-built image to docker hub to manage? If we want to use it to compile C++ to wasm, we can just pull directly from docker hub?

I just pushed to my hub, if i want to use it somewhere else, I just run docker pull bitliu/proxy-wasm-cpp-sdk:v2 to get this image to compile cpp to wasm.

Build wasm in docker image based on main branch failed

docker run -v $PWD:/work -w /work wasmsdk:v2 /build_wasm.sh
Adding directories to PATH:
PATH += /root/emsdk
PATH += /root/emsdk/upstream/emscripten
PATH += /root/emsdk/node/14.15.5_64bit/bin

Setting environment variables:
PATH = /root/emsdk:/root/emsdk/upstream/emscripten:/root/emsdk/node/14.15.5_64bit/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
EMSDK = /root/emsdk
EM_CONFIG = /root/emsdk/.emscripten
EMSDK_NODE = /root/emsdk/node/14.15.5_64bit/bin/node
em++ -s STANDALONE_WASM=1 -s EMIT_EMSCRIPTEN_METADATA=1 -s EXPORTED_FUNCTIONS=['_malloc'] --std=c++17 -O3 -flto -s WASM_OBJECT_FILES=0 --llvm-lto 1 -DPROXY_WASM_PROTOBUF_LITE=1 -I/sdk -I/usr/local/include --js-library /sdk/proxy_wasm_intrinsics.js my_wasm.cc /sdk/proxy_wasm_intrinsics_lite.pb.cc /sdk/struct_lite.pb.cc /sdk/proxy_wasm_intrinsics.cc /sdk/libprotobuf-lite.a utility/*.cc -o my_wasm.wasm
emcc:WARNING: --llvm-lto ignored when using llvm backend
clang-12: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
cache:INFO: generating system asset: crt1.o... (this will be cached in "/root/emsdk/upstream/emscripten/cache/wasm-lto/crt1.o" for subsequent builds)
cache:INFO: - ok
Traceback (most recent call last):
File "/root/emsdk/upstream/emscripten/em++.py", line 14, in
sys.exit(emcc.run(sys.argv))
File "/root/emsdk/upstream/emscripten/emcc.py", line 2256, in run
do_binaryen(target, options, wasm_target)
File "/root/emsdk/upstream/emscripten/emcc.py", line 2703, in do_binaryen
webassembly.add_emscripten_metadata(wasm_target)
File "/root/emsdk/upstream/emscripten/tools/webassembly.py", line 91, in add_emscripten_metadata
toLEB(0) +
File "/root/emsdk/upstream/emscripten/tools/webassembly.py", line 33, in toLEB
assert x >= 0, 'TODO: signed'
AssertionError: TODO: signed
Makefile:17: recipe for target 'my_wasm.wasm' failed
make: *** [my_wasm.wasm] Error 1

And succeed in envoy-release:v1.5 branch.

details of httpcall method

I would like to make calls to an external http server via the proxy. The method 'httpcall' allows one to do it right? could someone give an example how to use it with a real URL? also, this method can be used to make calls to web service outside the cluster right (like httpbin.org for eg)?

Is there an http status code related API

How can I return an HttpResponse in my cpp project, for example, I can further set to return 200 or 403 http status code according to whether the field value contained in the request header matches certain rules, is there any such example? I don't see an API for returning the relevant http status code in the API documentation provided by github

Is there an example of something like envoy wasm rust?

Failed to load Wasm module

[source/extensions/common/wasm/wasm_vm.cc:39] Failed to load Wasm module due to a missing import: env.pthread_rwlock_init

Can I transfrom http request into tcp on HTTP Filter

Can I transfrom http request into tcp on HTTP Filter

Description:
We have an old RPC framework running on production, with a self-designed RPC protocol on TCP. And I've made a transformer to turn gRPC request onto this private RPC request, so that these old services can run in Kubernetes, work on L7, and be controlled by Istio-proxy.
My question is, can I do this transform by WASM with Istio? If I apply a custom wasm on HTTP Filter, how can I turn gPRC
request to TCP bytes(non HTTP format).

By the way, I found ReplaceDownstreamData in Go's APIs (not found in CPP's), but it seems only available in TcpContext, not HttpContext.

Dynamic metadata of type google::protobuf::Struct not working

In my scenario I'm setting up dynamic metadata from ext_authz filter and trying to read from my WASM filter as follows. But the method which is defined to get metadata returns false.

google::protobuf::Struct ext_metadata;
if (!getMessageValue<google::protobuf::Struct>(
           {"metadata", "filter_metadata", "envoy.filters.http.ext_authz"}, &ext_metadata)) {
     LOG_ERROR(std::string("filter_metadata Error ") + std::to_string(id()));
   }

That logs the error because the method getMessageValue returns false. I tried to debug the issue and the following section of logs shows the issue.

envoyproxy-websocket_1  | [2021-03-29 10:56:54.300][29][trace][wasm] [source/extensions/common/wasm/wasm_vm.cc:40] [vm->host] env.proxy_get_property(5458080, 54, 5345664, 5345616)
envoyproxy-websocket_1  | [2021-03-29 10:56:54.300][29][trace][wasm] [source/extensions/common/wasm/wasm_vm.cc:40] [host->vm] malloc(177)
envoyproxy-websocket_1  | [2021-03-29 10:56:54.300][29][trace][wasm] [source/extensions/common/wasm/wasm_vm.cc:40] [host<-vm] malloc return: 5346384
envoyproxy-websocket_1  | [2021-03-29 10:56:54.300][29][trace][wasm] [source/extensions/common/wasm/wasm_vm.cc:40] [vm<-host] env.proxy_get_property return: 0
envoyproxy-websocket_1  | [2021-03-29 10:56:54.300][29][trace][wasm] [source/extensions/common/wasm/wasm_vm.cc:40] [vm->host] env.proxy_log(2, 5449512, 92)

I just tried the following to check whether a value exists in the ext_authz metadata and it returns true. Also the trace shows the metadata.

auto buf1 = getProperty<std::string>({"metadata", "filter_metadata", "envoy.filters.http.ext_authz"});
  if (buf1.has_value()) {
    LOG_INFO("Metadata exist");
  }

how to read file in wasm

In the wasm filter ,I want to read some configurations from file.
I wrote some code in the onConfigure part like this

    std::string data;
    std::ifstream infile;
    infile.open("/configurations.json");
    infile >> data;;
    infile.close();
    LOG_ERROR("configurations -> " + data);

Howerev, I cannot get any content of the configuration file.

How to read a configuration file in wasm ? Is there a file read API in the proxy-wasm-cpp-sdk?

Ths.

Question: out-of-tree wasm modules?

Hi there. I've been having some fun with this project lately, thanks for that!

Perusing the docs, I've stumbled upon a section that spiked my interest:

TODO: add an example about out of tree WASM module example

What is that about? 🤞 I'm hoping it's about having one wasm module coming from somewhere, and wrapping that in a proxy-wasm-compatible way in a new wasm module (because that's something I'd like to achieve).

Can you please shed some light on this for me? Am I completely misguided, or are there any pointers related to this that you could share? 😃

docker build Dockerfile-sdk error

When I run the docker build command, I get the following error:
......
Cloning into 'protobuf'...
fatal: unable to access 'https://github.com/protocolbuffers/protobuf/': gnutls_handshake() failed: Error in the push function.
The command '/bin/sh -c ./sdk_container.sh' returned a non-zero code: 128

I've tried several solutions, but it still reports errors. Can someone help me? Or provide me with a built docker image. Thanks.

httpCall crash

Hi.

the conditions below using httpCall API, crach occurs.

  • When service exists but Pod does not exist
$ kubectl get service -o yaml test-service
apiVersion: v1
kind: Service
metadata:
  name: test-service
  labels:
    app: test-server
spec:
  selector:
    app: test-server
  type: NodePort
  ports:
    - name: http2
      port: 8080
      nodePort: 32530
      targetPort: 9090

$ kubectl get pod test-service
Error from server (NotFound): pods "test-service" not found

The backtrace is as follows


2021-01-07T04:20:30.141218Z	warning	envoy wasm	[external/envoy/source/extensions/common/wasm/context.cc:1110] wasm log outbound|8080||test-service.default.svc.cluster.local
2021-01-07T04:20:30.141362Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] Caught Segmentation fault, suspect faulting address 0x148
2021-01-07T04:20:30.141385Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
2021-01-07T04:20:30.141392Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:92] Envoy version: a20ed1b7b98d8a7a8480c5100c0752578e31d659/1.14.4/Clean/RELEASE/BoringSSL
2021-01-07T04:20:30.141897Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #0: __restore_rt [0x7fad273038a0]
2021-01-07T04:20:30.154680Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #1: Envoy::Http::AsyncRequestImpl::onComplete() [0x55b896955ccf]
2021-01-07T04:20:30.165000Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #2: Envoy::Http::AsyncStreamImpl::encodeData() [0x55b8969556c2]
2021-01-07T04:20:30.174887Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #3: Envoy::Http::Utility::sendLocalReply() [0x55b896a744ee]
2021-01-07T04:20:30.184720Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #4: Envoy::Http::AsyncStreamImpl::sendLocalReply() [0x55b896956975]
2021-01-07T04:20:30.200819Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #5: Envoy::Router::Filter::sendNoHealthyUpstreamResponse() [0x55b89695e98a]
2021-01-07T04:20:30.211038Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #6: Envoy::Router::Filter::decodeHeaders() [0x55b89695ce9c]
2021-01-07T04:20:30.221311Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #7: Envoy::Http::AsyncStreamImpl::sendHeaders() [0x55b896955a47]
2021-01-07T04:20:30.231357Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #8: Envoy::Http::AsyncRequestImpl::initialize() [0x55b8969549e7]
2021-01-07T04:20:30.241355Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #9: Envoy::Http::AsyncClientImpl::send() [0x55b89695444b]
2021-01-07T04:20:30.253443Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #10: Envoy::Extensions::Common::Wasm::Context::httpCall() [0x55b8959aacd2]
2021-01-07T04:20:30.263850Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #11: Envoy::Extensions::Common::Wasm::Exports::http_call() [0x55b8959c2b45]
2021-01-07T04:20:30.284883Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #12: Envoy::Extensions::Common::Wasm::ConvertFunctionWordToUint32<>::convertFunctionWordToUint32() [0x55b8959cea06]
2021-01-07T04:20:30.300589Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #13: Envoy::Extensions::Common::Wasm::V8::V8::registerHostFunctionImpl<>()::{lambda()#1}::operator()() [0x55b8959f96f2]
2021-01-07T04:20:30.310606Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #14: Envoy::Extensions::Common::Wasm::V8::V8::registerHostFunctionImpl<>()::{lambda()#1}::__invoke() [0x55b8959f9597]
2021-01-07T04:20:30.322101Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #15: wasm::FuncData::v8_callback() [0x55b895a04e8f]
2021-01-07T04:20:30.322529Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #16: [0x39fdc0a41647]
2021-01-07T04:20:30.323159Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #17: [0x39fdc0712f74]
2021-01-07T04:20:30.323801Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #18: [0x39fdc07b275f]
2021-01-07T04:20:30.324442Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #19: [0x39fdc075c935]
2021-01-07T04:20:30.325075Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #20: [0x39fdc0895348]
2021-01-07T04:20:30.325714Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:98] #21: [0x16f6000835df]
2021-01-07T04:20:30.335913Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #22: v8::internal::Execution::CallWasm() [0x55b895add963]
2021-01-07T04:20:30.351839Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #23: wasm::Func::call() [0x55b895a041f3]
2021-01-07T04:20:30.362223Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #24: Envoy::Extensions::Common::Wasm::V8::V8::getModuleFunctionImpl<>()::{lambda()#1}::operator()() [0x55b8959ecc40]
2021-01-07T04:20:30.373415Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #25: Envoy::Extensions::Common::Wasm::Context::onRequestBody() [0x55b8959addb9]
2021-01-07T04:20:30.383636Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #26: Envoy::Extensions::Common::Wasm::Context::decodeData() [0x55b8959b03dc]
2021-01-07T04:20:30.394001Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #27: Envoy::Http::ConnectionManagerImpl::ActiveStream::decodeData() [0x55b8969ca131]
2021-01-07T04:20:30.403919Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #28: Envoy::Http::ConnectionManagerImpl::ActiveStream::decodeData() [0x55b8969c9f08]
2021-01-07T04:20:30.415622Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #29: Envoy::Http::Http2::ConnectionImpl::onFrameReceived() [0x55b8969e4589]
2021-01-07T04:20:30.428600Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #30: nghttp2_session_on_data_received [0x55b896bb109c]
2021-01-07T04:20:30.444517Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #31: nghttp2_session_mem_recv [0x55b896bb309d]
2021-01-07T04:20:30.455400Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #32: Envoy::Http::Http2::ConnectionImpl::dispatch() [0x55b8969e3c1a]
2021-01-07T04:20:30.465503Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #33: Envoy::Http::Http2::ServerConnectionImpl::dispatch() [0x55b8969e7660]
2021-01-07T04:20:30.475754Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #34: Envoy::Http::ConnectionManagerImpl::onData() [0x55b8969c4a52]
2021-01-07T04:20:30.490730Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #35: Envoy::Network::FilterManagerImpl::onContinueReading() [0x55b896763883]
2021-01-07T04:20:30.502170Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #36: Envoy::Network::ConnectionImpl::onReadReady() [0x55b89675f9c5]
2021-01-07T04:20:30.512263Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #37: Envoy::Network::ConnectionImpl::onFileEvent() [0x55b89675ed1d]
2021-01-07T04:20:30.522525Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #38: Envoy::Event::FileEventImpl::assignEvents()::$_0::__invoke() [0x55b896759d46]
2021-01-07T04:20:30.532490Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #39: event_process_active_single_queue [0x55b896ba2f4b]
2021-01-07T04:20:30.542948Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #40: event_base_loop [0x55b896ba17de]
2021-01-07T04:20:30.559206Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #41: Envoy::Server::WorkerImpl::threadRoutine() [0x55b89674db84]
2021-01-07T04:20:30.569170Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #42: Envoy::Thread::ThreadImplPosix::ThreadImplPosix()::$_0::__invoke() [0x55b896c579c3]
2021-01-07T04:20:30.569285Z	critical	envoy backtrace	[bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:96] #43: start_thread [0x7fad272f86db]
ActiveStream 0x55b89951db00, stream_id_: 17940336530524321975, state_.has_continue_headers_: 0, state_.is_head_request_: 0, state_.decoding_headers_only_: 0, state_.encoding_headers_only_: 0
request_headers_: 
  ':authority', 'test.domain.com:32530'
  ':method', 'PUT'
  ':path', '/namf-comm/v1/ue-contexts/nfsimulator'
  ':scheme', 'http'
  'content-type', 'application/json'
  'content-length', '5624'
  'accept-encoding', 'gzip'
  'user-agent', 'Go-http-client/2.0'
  'x-forwarded-proto', 'http'
  'x-request-id', '5173ce30-091d-4402-9ffb-533748f70aaf'
  'x-envoy-decorator-operation', 'test.domain.com:32530/*'
  'x-envoy-peer-metadata-id', 'sidecar~10.244.1.38~nfsim-server-8bcf66cf4-hqhbg.default~default.svc.cluster.local'
request_trailers_: null
response_headers_: null
response_trailers_: null
&stream_info_: 
  StreamInfoImpl 0x55b89951dc48, protocol_: 2, response_code_: null, response_code_details_: null, health_check_request_: 0, route_name_: 
2021-01-07T04:20:30.585441Z	info	transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2021-01-07T04:20:30.585478Z	info	sds	resource:default connection is terminated: rpc error: code = Canceled desc = context canceled
2021-01-07T04:20:30.585503Z	info	sds	resource:ROOTCA connection is terminated: rpc error: code = Canceled desc = context canceled
2021-01-07T04:20:30.585543Z	error	sds	Remote side closed connection
2021-01-07T04:20:30.585547Z	error	sds	Remote side closed connection
2021-01-07T04:20:30.586437Z	error	Epoch 0 exited with error: signal: segmentation fault
2021-01-07T04:20:30.586458Z	info	No more active epochs, terminating

It can be solved by modifying it like the code below. Please review.
envoy/source/extensions/common/wasm.context.cc

WasmResult Context::httpCall(absl::string_view cluster, const Pairs& request_headers,
                             absl::string_view request_body, const Pairs& request_trailers,
                             int timeout_milliseconds, uint32_t* token_ptr) {

  --- skip ----

#ifdef __FIX_ENVOY_CORE__ /* by eunbin */
  handler.context_ = this;
  handler.token_ = token;
  *token_ptr = token;
#endif

  // set default hash policy to be based on :authority to enable consistent hash
  Http::AsyncClient::RequestOptions options;
  options.setTimeout(timeout);
  Protobuf::RepeatedPtrField<HashPolicy> hash_policy;
  hash_policy.Add()->mutable_header()->set_header_name(Http::Headers::get().Host.get());
  options.setHashPolicy(hash_policy);
  auto http_request = clusterManager()
                          .httpAsyncClientForCluster(cluster_string)
                          .send(std::move(message), handler, options);
  if (!http_request) {
    http_request_.erase(token);
    return WasmResult::InternalFailure;
  }

#ifdef __FIX_ENVOY_CORE__ /* by eunbin */
    handler.request_ = http_request;
#else
  handler.context_ = this;
  handler.token_ = token;
  handler.request_ = http_request;
  *token_ptr = token;
  return WasmResult::Ok;
#endif
}

Does envoy or envoy wasm c++ sdk support caching of http request and response body

I deployed Envoy wasm plugin in Istio environment,  and use wasm c++ sdk to operate on http requests,I found that the current onRequestBody method does not seem to have a cache operation for the request body, what I want is to have this cache operation so that the request or response body can be accumulated to a certain size before being detected and processed.for example, I can set a buffer size of 10K, only when the request payload accumulated to 10K can trigger the processing logic.
Is there any consideration for this in the future ? or Envoy already has this configuration, looking forward to your reply.
Also,Does wasm currently support breakpoint debugging inside the envoy wasm sandbox using a gdb-like approach?

Envoy 1.14.5 compatibility

Can you provide some details on what branch/tag/commit points to the latest commit for Envoy 1.14.5? My understanding is that the envoy-release/v1.15 is for Envoy 1.15 and the SDK built from this branch is not compatible with Envoy 1.14.5

How to update content-length header after modifying response body via setBuffer(WasmBufferType::HttpResponseBody) ?

Hi, I have a json response like {"foo": "bar"}. It's content-length is 14
I added a wasm to wrap the json response to sth like {"code": 0, "err": "", data: {"foo": "bar"}}. It's content-length is 44.
I tried calling addResponseHeader("content-length", "44") and replaceResponseHeader("content-length", "44") in onResponseBody. Neither worked. And curl complains: curl: (18) transfer closed with 30 bytes remaining to read`.

How can I update content-length after response body is modified?
Thanks.

modify response headers values based on response body contents

Is it possible to modify response header values based on response body contents? For example, if I match a certain pattern in onResponseBody I want to replace :status header value from 200 to 401 or 403 value.

According to documentation, Header API is only effective from onRequestHeader/onResponseHeader. So my task seems to be impossible to implement. Or maybe you can point to some kind of workaround?

Cheers,
Alexander.

Exceptions thrown and caught within WebAssembly make Envoy drop the connection

When the code inside a wasm module throws an exception, and catches it within the same execution so that it does not cross the boundary between wasm and Envoy it drops the connection without a response.

I have created a reproducer for Istio Proxy 1.8.1 at https://github.com/robertpanzer/wasm-extensions
The README explains how to test it.

Expected behavior:
A wasm module should be able to throw and catch exceptions within Context::onRequestHeaders as long as onRequestHeaders does not return exceptionally.

Observed behavior:
When a method throws an exception that is transitively called by onRequestHeaders, and the exception is caught by some other method in that call stack, Envoy drops the downstream connnection without providing any response.

core occurs when No-Content is received after httpCall

hi. core occurs when 204 No-Content is received after httpCall.

For what reason? Please tell me the solution.

The core information is as follows

tanks.

[Envoy (Epoch 0)] [2020-05-22 01:54:01.119][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:83] Caught Segmentation fault, suspect faulting address 0x0
[Envoy (Epoch 0)] [2020-05-22 01:54:01.119][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:70] Backtrace (use tools/stack_decode.py to get line numbers):
[Envoy (Epoch 0)] [2020-05-22 01:54:01.119][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:71] Envoy version: 73f240a29bece92a8882a36893ccce07b4a54664/1.13.1-dev/Clean/RELEASE/BoringSSL
[Envoy (Epoch 0)] [2020-05-22 01:54:01.121][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #0: __restore_rt [0x7efdadd2c890]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.156][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #1: Envoy::Http::AsyncRequestImpl::onComplete() [0x55b0d9a9cb6c]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.181][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #2: Envoy::Http::AsyncStreamImpl::encodeHeaders() [0x55b0d9a9c375]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.207][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #3: Envoy::Router::Filter::onUpstreamHeaders() [0x55b0d9aa8698]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.233][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #4: Envoy::Router::Filter::UpstreamRequest::decodeHeaders() [0x55b0d9aaa7d6]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.258][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #5: Envoy::Http::Http1::ConnPoolImpl::StreamWrapper::decodeHeaders() [0x55b0d9a25b8b]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.283][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #6: Envoy::Http::StreamDecoderWrapper::decodeHeaders() [0x55b0d9a8c32c]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.308][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #7: Envoy::Http::Http1::ClientConnectionImpl::onMessageComplete() [0x55b0d9b17016]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.333][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #8: Envoy::Http::Http1::ConnectionImpl::onMessageCompleteBase() [0x55b0d9b1445e]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.358][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #9: Envoy::Http::Http1::ConnectionImpl::$_8::__invoke() [0x55b0d9b184dd]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.384][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #10: http_parser_execute [0x55b0d9beb052]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.411][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #11: Envoy::Http::Http1::ConnectionImpl::dispatchSlice() [0x55b0d9b12f9a]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.439][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #12: Envoy::Http::Http1::ConnectionImpl::dispatch() [0x55b0d9b12d7f]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.464][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #13: Envoy::Http::CodecClient::onData() [0x55b0d9a8b4a8]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.489][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #14: Envoy::Http::CodecClient::CodecReadFilter::onData() [0x55b0d9a8c1fd]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.515][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #15: Envoy::Network::FilterManagerImpl::onContinueReading() [0x55b0d98bc5db]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.542][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #16: Envoy::Network::ConnectionImpl::onReadReady() [0x55b0d98b904a]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.569][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #17: Envoy::Network::ConnectionImpl::onFileEvent() [0x55b0d98b8b1d]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.596][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #18: Envoy::Event::FileEventImpl::assignEvents()::$_0::__invoke() [0x55b0d98b35f0]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.623][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:75] #19: event_process_active_single_queue [0x55b0d9be49ab]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.649][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/virtual_includes/backtrace_lib/server/backtrace.h:75] #20: event_base_loop [0x55b0d9be323e]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.675][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/virtual_includes/backtrace_lib/server/backtrace.h:75] #21: Envoy::Server::WorkerImpl::threadRoutine() [0x55b0d98a9278]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.700][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/virtual_includes/backtrace_lib/server/backtrace.h:75] #22: Envoy::Thread::ThreadImplPosix::ThreadImplPosix()::$0::invoke() [0x55b0d9daf953]
[Envoy (Epoch 0)] [2020-05-22 01:54:01.700][32][critical][backtrace] [bazel-out/k8-opt/bin/external/envoy/source/server/virtual_includes/backtrace_lib/server/backtrace.h:75] #23: start_thread [0x7efdadd216db]
AsyncClient 0x55b0e04bb690, stream_id
: 12133954153073262449
&stream_info
:
StreamInfoImpl 0x55b0e04bb830, protocol
: 1, response_code
: 200, response_code_details
: via_upstream, health_check_request
: 0, route_name
:
2020-05-22T01:54:02.415268Z info sds node:router192.168.135.53istio-ingressgateway-5f884b4dbd-bf6q4.istio-systemistio-system.svc.cluster.local-2 resource:ROOTCA connection is terminated: rpc error: code = Canceled desc = context canceled
2020-05-22T01:54:02.415321Z error sds Remote side closed connection
2020-05-22T01:54:02.415857Z info Envoy proxy is NOT ready: failed to get server info: Get http://127.0.0.1:15000/stats?usedonly&filter=server.state: dial tcp 127.0.0.1:15000: connect: connection refused
2020-05-22T01:54:02.415903Z info transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2020-05-22T01:54:02.416120Z info sds node:router
192.168.135.53istio-ingressgateway-5f884b4dbd-bf6q4.istio-systemistio-system.svc.cluster.local-1 resource:default connection is terminated: rpc error: code = Canceled desc = context canceled
2020-05-22T01:54:02.416176Z error sds Remote side closed connection
2020-05-22T01:54:02.418683Z error Epoch 0 exited with error: signal: segmentation fault (core dumped)
2020-05-22T01:54:02.418720Z info No more active epochs, terminating

How to set Environment variables for Envoy-Filter?

Hi.

I want to use environment variables in EnvoyFilter, is there a way to set it?

I checked the environment variable with the code below but couldn't find anything.


extern char **environ;
bool testRootContext::onStart(size_t ul_config_size)
{
for (int for_i = 0; environ[for_i] != NULL; for_i++) {
LOG_ERROR(std::string("ENV==== "), std::string(environ[for_i]));
}
return true;
}


Let me know if there is a way to get the environment variable.

thanks.

httpCall should be able to call by host

Httpcall and grpc call have very different APIs.

HttpCall requires an upstream cluster, where are grpc calls can define their own. It is very convenient to define cluster with the plugin config so as to not rely on out of band clusters.

Simplegrpcall already supports this.

/docs/wasm_filter.md API Update

It seems that proxy_wasm_api.h has been updated since /docs/wasm_filter.md pushed. Part of the docs left behind. Is it ok to update /docs/wasm_filter.md by contributing?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.