jaykchen/a-test

a-test's Issues

This is terrible

wasiTests Fail on ubuntu20.04 (codespace)

Description
wasiTests Fail.

Current State
@L-jasmine ➜ /workspaces/WasmEdge/build (fix/c_plugin) $ LD_LIBRARY_PATH=$(pwd)/lib/api ctest

Test project /workspaces/WasmEdge/build
Start 1: wasmedgeAOTCoreTests
1/23 Test #1: wasmedgeAOTCoreTests ............. Passed 31.00 sec
Start 2: wasmedgeAOTCacheTests
2/23 Test #2: wasmedgeAOTCacheTests ............ Passed 0.01 sec
Start 3: wasmedgeAOTBlake3Tests
3/23 Test #3: wasmedgeAOTBlake3Tests ........... Passed 0.01 sec
Start 4: wasmedgeMixcallTests
4/23 Test #4: wasmedgeMixcallTests ............. Passed 0.03 sec
Start 5: wasmedgeCommonTests
5/23 Test #5: wasmedgeCommonTests .............. Passed 0.00 sec
Start 6: wasmedgeLoaderFileMgrTests
6/23 Test #6: wasmedgeLoaderFileMgrTests ....... Passed 0.00 sec
Start 7: wasmedgeLoaderASTTests
7/23 Test #7: wasmedgeLoaderASTTests ........... Passed 0.00 sec
Start 8: wasmedgeExecutorCoreTests
8/23 Test #8: wasmedgeExecutorCoreTests ........ Passed 1.08 sec
Start 9: wasmedgeThreadTests
9/23 Test #9: wasmedgeThreadTests .............. Passed 1.07 sec
Start 10: wasmedgeAPIUnitTests
10/23 Test #10: wasmedgeAPIUnitTests ............. Passed 0.10 sec
Start 11: wasmedgeAPIVMCoreTests
11/23 Test #11: wasmedgeAPIVMCoreTests ........... Passed 1.12 sec
Start 12: wasmedgeAPIStepsCoreTests
12/23 Test #12: wasmedgeAPIStepsCoreTests ........ Passed 1.08 sec
Start 13: wasmedgeAPIAOTCoreTests
13/23 Test #13: wasmedgeAPIAOTCoreTests .......... Passed 12.53 sec
Start 14: wasmedgeExternrefTests
14/23 Test #14: wasmedgeExternrefTests ........... Passed 0.01 sec
Start 15: wasmedgePluginUnittests
15/23 Test #15: wasmedgePluginUnittests .......... Passed 0.01 sec
Start 16: wasiSocketTests
16/23 Test #16: wasiSocketTests .................. Passed 0.01 sec
Start 17: wasiTests
17/23 Test #17: wasiTests ........................***Failed 4.00 sec
Start 18: wasmedgeHostMockTests
18/23 Test #18: wasmedgeHostMockTests ............ Passed 0.00 sec
Start 19: expectedTests
19/23 Test #19: expectedTests .................... Passed 0.00 sec
Start 20: spanTests
20/23 Test #20: spanTests ........................ Passed 0.00 sec
Start 21: poTests
21/23 Test #21: poTests .......................... Passed 0.00 sec
Start 22: wasmedgeMemLimitTests
22/23 Test #22: wasmedgeMemLimitTests ............ Passed 0.00 sec
Start 23: wasmedgeErrinfoTests
23/23 Test #23: wasmedgeErrinfoTests ............. Passed 0.00 sec

96% tests passed, 1 tests failed out of 23

Total Test time (real) = 52.12 sec

The following tests FAILED:
17 - wasiTests (Failed)
Errors while running CTest
Expected
Environment
Hardware Architecture: x86_64
Operating system: Ubuntu 20.04 (codespace)
The following information is optional. Please provide them only if you have built from source.

C++ Compiler version: clang-12
CMake version:
CMake flags: -DCMAKE_BUILD_TYPE=RELEASE -DWASMEDGE_BUILD_TESTS=ON
Boost version:
Steps to Reproduce

follow https://wasmedge.org/book/en/contribute/build_from_src/linux.html#for-ubuntu-2004

export CC=clang-12
export CXX=clang++-12
make build && cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE -DWASMEDGE_BUILD_TESTS=ON .. && make
LD_LIBRARY_PATH=$(pwd)/lib/api ctest
Description
wasiTests Fail.

Current State
@L-jasmine ➜ /workspaces/WasmEdge/build (fix/c_plugin) $ LD_LIBRARY_PATH=$(pwd)/lib/api ctest

Test project /workspaces/WasmEdge/build
Start 1: wasmedgeAOTCoreTests
1/23 Test #1: wasmedgeAOTCoreTests ............. Passed 31.00 sec
Start 2: wasmedgeAOTCacheTests
2/23 Test #2: wasmedgeAOTCacheTests ............ Passed 0.01 sec
Start 3: wasmedgeAOTBlake3Tests
3/23 Test #3: wasmedgeAOTBlake3Tests ........... Passed 0.01 sec
Start 4: wasmedgeMixcallTests
4/23 Test #4: wasmedgeMixcallTests ............. Passed 0.03 sec
Start 5: wasmedgeCommonTests
5/23 Test #5: wasmedgeCommonTests .............. Passed 0.00 sec
Start 6: wasmedgeLoaderFileMgrTests
6/23 Test #6: wasmedgeLoaderFileMgrTests ....... Passed 0.00 sec
Start 7: wasmedgeLoaderASTTests
7/23 Test #7: wasmedgeLoaderASTTests ........... Passed 0.00 sec
Start 8: wasmedgeExecutorCoreTests
8/23 Test #8: wasmedgeExecutorCoreTests ........ Passed 1.08 sec
Start 9: wasmedgeThreadTests
9/23 Test #9: wasmedgeThreadTests .............. Passed 1.07 sec
Start 10: wasmedgeAPIUnitTests
10/23 Test #10: wasmedgeAPIUnitTests ............. Passed 0.10 sec
Start 11: wasmedgeAPIVMCoreTests
11/23 Test #11: wasmedgeAPIVMCoreTests ........... Passed 1.12 sec
Start 12: wasmedgeAPIStepsCoreTests
12/23 Test #12: wasmedgeAPIStepsCoreTests ........ Passed 1.08 sec
Start 13: wasmedgeAPIAOTCoreTests
13/23 Test #13: wasmedgeAPIAOTCoreTests .......... Passed 12.53 sec
Start 14: wasmedgeExternrefTests
14/23 Test #14: wasmedgeExternrefTests ........... Passed 0.01 sec
Start 15: wasmedgePluginUnittests
15/23 Test #15: wasmedgePluginUnittests .......... Passed 0.01 sec
Start 16: wasiSocketTests
16/23 Test #16: wasiSocketTests .................. Passed 0.01 sec
Start 17: wasiTests
17/23 Test #17: wasiTests ........................***Failed 4.00 sec
Start 18: wasmedgeHostMockTests
18/23 Test #18: wasmedgeHostMockTests ............ Passed 0.00 sec
Start 19: expectedTests
19/23 Test #19: expectedTests .................... Passed 0.00 sec
Start 20: spanTests
20/23 Test #20: spanTests ........................ Passed 0.00 sec
Start 21: poTests
21/23 Test #21: poTests .......................... Passed 0.00 sec
Start 22: wasmedgeMemLimitTests
22/23 Test #22: wasmedgeMemLimitTests ............ Passed 0.00 sec
Start 23: wasmedgeErrinfoTests
23/23 Test #23: wasmedgeErrinfoTests ............. Passed 0.00 sec

96% tests passed, 1 tests failed out of 23

Total Test time (real) = 52.12 sec

The following tests FAILED:
17 - wasiTests (Failed)
Errors while running CTest
Expected
Environment
Hardware Architecture: x86_64
Operating system: Ubuntu 20.04 (codespace)
The following information is optional. Please provide them only if you have built from source.

C++ Compiler version: clang-12
CMake version:
CMake flags: -DCMAKE_BUILD_TYPE=RELEASE -DWASMEDGE_BUILD_TESTS=ON
Boost version:
Steps to Reproduce

follow https://wasmedge.org/book/en/contribute/build_from_src/linux.html#for-ubuntu-2004

export CC=clang-12
export CXX=clang++-12
make build && cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE -DWASMEDGE_BUILD_TESTS=ON .. && make
LD_LIBRARY_PATH=$(pwd)/lib/api ctest

Tell me how much does it cost to chat 500 tokens

good afternoon

This library completely failed my project

please fix it asap

new issue

recheck

you are mentioned @jaykchen

good

fake issue

nothing

Response pauses indefinitely after 360MB

I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }

find out how zapier integration works

to get @jaykchen mention

l;j;kjl

long issue to be truncated

Description
I am currently working for the WasmEdge/WasmEdge#2356 issue in WasmEdge. I have encountered some unexpected behaviors while using the Wasi-NN plugin. I have created a reproducible example with all required descriptions at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing .

Current State
The code contains a rust folder with a simple reproducible example for the LayoutLMv2ModelForTokenClassification using dummy inputs to demonstrate a bug encountered
durng inference of the traced torchscript model created using it.

The cpp_torchscript folder within this contains a similar reproducible example using the dummy inputs and inferencing the TorchScript model from the TorchScript C++ API
directly. This is done in a similar fashion as to my understanding of what the WASI-NN plugin would do with it.

The inputs expected by the model are described at https://github.com/sarrah-basta/wasmedge_ai_testing/blob/main/layoutlmv2_with_wasi_ocr/README.md. However, currently for dummy inputs, I have focused on the creating tensors identical to the following shapes :

For index 0 Datatype: long int Shape: [1, 364]
For index 1 Datatype: long int Shape: [1, 364, 4]
For index 2 Datatype: float Shape: [1, 3, 224, 224]
For index 3 Datatype: float Shape: [1, 364]
For index 4 Datatype: long int Shape: [1, 364]

Changes made to the Wasi-NN plugin files within WasmEdge
The original error I encountered was as follows :

[2023-04-16 20:59:08.292] [error] [WASI-NN] Only F32 inputs and outputs are supported for now.
thread 'main' panicked at 'called Result::unwrap() on an Err value: NnErrno { code: 1, name: "INVALID_ARGUMENT", message: "" }', src/main.rs:107:9
stack backtrace:
[2023-04-16 20:59:08.293] [error] execution failed: unreachable, Code: 0x89
[2023-04-16 20:59:08.293] [error] In instruction: unreachable (0x00) , Bytecode offset: 0x001a8568
[2023-04-16 20:59:08.293] [error] When executing function name: "_start"
This error is being caused due to this check in the source code plugins/wasi-nn.

Thus, I made a few changes to the source code file to make sure I was creating tensors of the correct types. This diff can be found here

Bug Found
Following these changes, when trying to inference the model, the following error was encountered :

erminate called after throwing an instance of 'std::runtime_error'
what(): The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9015.py", line 17, in forward
_0 = self.classifier
_1 = self.dropout
_2 = (self.layoutlmv2).forward(input_ids, bbox, attention_mask, input, images, )

seq_length = ops.prim.NumToTensor(torch.size(input_ids, 1))
_3 = int(seq_length)
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9012.py", line 71, in forward
_39 = [position_ids0, visual_position_ids]
position_ids1 = torch.cat(_39, 1)
_40 = (_14).forward(input_ids, )

_41 = (_13).forward(position_ids0, )
_42 = torch.slice(bbox, 0, 0, 9223372036854775807, 1)
File "code/torch/torch/nn/modules/sparse/___torch_mangle_8540.py", line 9, in forward
def forward(self: torch.torch.nn.modules.sparse.___torch_mangle_8540.Embedding,
input_ids: Tensor) -> Tensor:
inputs_embeds = torch.embedding(self.weight, input_ids, 0, False, False)
~~~~~~~~~~~~~~~ <--- HERE
return inputs_embeds
Traceback of TorchScript, original code (most recent call last):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/functional.py(1913): embedding
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/sparse.py(145): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(751): _calc_text_embeddings
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(907): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(1233): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(934): trace_module
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(733): trace
/tmp/ipykernel_17852/2995848706.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3460): run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3400): run_ast_nodes
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3221): run_cell_async
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/async_helpers.py(129): _pseudo_sync_runner
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3016): _run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(2961): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/zmqshell.py(531): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/ipkernel.py(411): do_execute
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(729): execute_request
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(406): dispatch_shell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(499): process_one
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(510): dispatch_queue
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/events.py(81): _run
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(1844): _run_once
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(563): run_forever
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/tornado/platform/asyncio.py(215): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelapp.py(711): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/traitlets/config/application.py(992): launch_instance
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel_launcher.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(85): _run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(192): _run_module_as_main
RuntimeError: index out of range in self

Aborted (core dumped)
Since this is a error thrown from the TorchScipt API, I then tried reproducing the code in a similar fashion directly in C++.

Expected
Running the cpp code file torchscript_inference.cpp inputs of the same shapes to the Torchscript API results in a correct output with the same model and vocabulary files, and prints the resultant output tensor of shape [ CPUFloatType{1,364,7} ]}.

Environment
Hardware Architecture: (x86_64)
Operating system: (Ubuntu 22.04)
C++ Compiler version: gcc 11.3.0 cmake
CMake version: 3.22.1
CMake flags: -DCMAKE_BUILD_TYPE=Release -DWASMEDGE_PLUGIN_WASI_NN_BACKEND="PyTorch"
Steps to reproduce
Change the source code of the plugins/wasi_nn/wasinfunc.cpp file according to the given [diff](https://drive.google.com/file/d/1puiRnrsVwWWztsteSbttxU7U04t3H5i7/view?usp=sharing).
Build WasmEdge from source with the Wasi-NN plugin with Pytorch Backend following instructions at https://wasmedge.org/book/en/contribute/build_from_src/plugin_wasi_nn.html#build-wasmedge-with-wasi-nn-pytorch-backend
Clone the Rust crate at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing/rust
Download the .pt file of the model from [this drive link](https://drive.google.com/file/d/1jDlaVoSBIhVkkPi5iNG2m5l0ieqtB-j9/view?usp=sharing)
Compile and execute using WasmEdge as follows :
Compile the source code to WebAssembly:
cargo build --target=wasm32-wasi --release
The output WASM file will be at target/wasm32-wasi/release/rust.wasm
To speed up the processing, we can enable the AOT mode in WasmEdge with:

wasmedgec target/wasm32-wasi/release/rust.wasm out.wasm
Execute the WASM with the wasmedge with PyTorch supporting:
Always need to enable environment variable in new terminal

This is the worst product ever!

mention to test gh slack official integration

@jaykchen so

Unexpected Behaviour inferencing a model with Wasi-NN plugin using Pytorch Backend

Description
I am currently working for the WasmEdge/WasmEdge#2356 issue in WasmEdge. I have encountered some unexpected behaviors while using the Wasi-NN plugin. I have created a reproducible example with all required descriptions at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing .

Current State
The code contains a rust folder with a simple reproducible example for the LayoutLMv2ModelForTokenClassification using dummy inputs to demonstrate a bug encountered
durng inference of the traced torchscript model created using it.

The cpp_torchscript folder within this contains a similar reproducible example using the dummy inputs and inferencing the TorchScript model from the TorchScript C++ API
directly. This is done in a similar fashion as to my understanding of what the WASI-NN plugin would do with it.

The inputs expected by the model are described at https://github.com/sarrah-basta/wasmedge_ai_testing/blob/main/layoutlmv2_with_wasi_ocr/README.md. However, currently for dummy inputs, I have focused on the creating tensors identical to the following shapes :

For index 0 Datatype: long int Shape: [1, 364]
For index 1 Datatype: long int Shape: [1, 364, 4]
For index 2 Datatype: float Shape: [1, 3, 224, 224]
For index 3 Datatype: float Shape: [1, 364]
For index 4 Datatype: long int Shape: [1, 364]

Changes made to the Wasi-NN plugin files within WasmEdge
The original error I encountered was as follows :

[2023-04-16 20:59:08.292] [error] [WASI-NN] Only F32 inputs and outputs are supported for now.
thread 'main' panicked at 'called Result::unwrap() on an Err value: NnErrno { code: 1, name: "INVALID_ARGUMENT", message: "" }', src/main.rs:107:9
stack backtrace:
[2023-04-16 20:59:08.293] [error] execution failed: unreachable, Code: 0x89
[2023-04-16 20:59:08.293] [error] In instruction: unreachable (0x00) , Bytecode offset: 0x001a8568
[2023-04-16 20:59:08.293] [error] When executing function name: "_start"
This error is being caused due to this check in the source code plugins/wasi-nn.

Thus, I made a few changes to the source code file to make sure I was creating tensors of the correct types. This diff can be found here

Bug Found
Following these changes, when trying to inference the model, the following error was encountered :

erminate called after throwing an instance of 'std::runtime_error'
what(): The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9015.py", line 17, in forward
_0 = self.classifier
_1 = self.dropout
_2 = (self.layoutlmv2).forward(input_ids, bbox, attention_mask, input, images, )

seq_length = ops.prim.NumToTensor(torch.size(input_ids, 1))
_3 = int(seq_length)
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9012.py", line 71, in forward
_39 = [position_ids0, visual_position_ids]
position_ids1 = torch.cat(_39, 1)
_40 = (_14).forward(input_ids, )
~~~~~~~~~~~~ <--- HERE
_41 = (_13).forward(position_ids0, )
_42 = torch.slice(bbox, 0, 0, 9223372036854775807, 1)
File "code/torch/torch/nn/modules/sparse/___torch_mangle_8540.py", line 9, in forward
def forward(self: torch.torch.nn.modules.sparse.___torch_mangle_8540.Embedding,
input_ids: Tensor) -> Tensor:
inputs_embeds = torch.embedding(self.weight, input_ids, 0, False, False)
~~~~~~~~~~~~~~~ <--- HERE
return inputs_embeds
Traceback of TorchScript, original code (most recent call last):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/functional.py(1913): embedding
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/sparse.py(145): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(751): _calc_text_embeddings
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(907): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(1233): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(934): trace_module
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(733): trace
/tmp/ipykernel_17852/2995848706.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3460): run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3400): run_ast_nodes
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3221): run_cell_async
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/async_helpers.py(129): _pseudo_sync_runner
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3016): _run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(2961): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/zmqshell.py(531): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/ipkernel.py(411): do_execute
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(729): execute_request
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(406): dispatch_shell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(499): process_one
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(510): dispatch_queue
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/events.py(81): _run
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(1844): _run_once
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(563): run_forever
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/tornado/platform/asyncio.py(215): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelapp.py(711): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/traitlets/config/application.py(992): launch_instance
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel_launcher.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(85): _run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(192): _run_module_as_main
RuntimeError: index out of range in self

Aborted (core dumped)
Since this is a error thrown from the TorchScipt API, I then tried reproducing the code in a similar fashion directly in C++.

Expected
Running the cpp code file torchscript_inference.cpp inputs of the same shapes to the Torchscript API results in a correct output with the same model and vocabulary files, and prints the resultant output tensor of shape [ CPUFloatType{1,364,7} ]}.

Environment
Hardware Architecture: (x86_64)
Operating system: (Ubuntu 22.04)
C++ Compiler version: gcc 11.3.0 cmake
CMake version: 3.22.1
CMake flags: -DCMAKE_BUILD_TYPE=Release -DWASMEDGE_PLUGIN_WASI_NN_BACKEND="PyTorch"
Steps to reproduce
Change the source code of the plugins/wasi_nn/wasinfunc.cpp file according to the given [diff](https://drive.google.com/file/d/1puiRnrsVwWWztsteSbttxU7U04t3H5i7/view?usp=sharing).
Build WasmEdge from source with the Wasi-NN plugin with Pytorch Backend following instructions at https://wasmedge.org/book/en/contribute/build_from_src/plugin_wasi_nn.html#build-wasmedge-with-wasi-nn-pytorch-backend
Clone the Rust crate at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing/rust
Download the .pt file of the model from [this drive link](https://drive.google.com/file/d/1jDlaVoSBIhVkkPi5iNG2m5l0ieqtB-j9/view?usp=sharing)
Compile and execute using WasmEdge as follows :
Compile the source code to WebAssembly:
cargo build --target=wasm32-wasi --release
The output WASM file will be at target/wasm32-wasi/release/rust.wasm
To speed up the processing, we can enable the AOT mode in WasmEdge with:

wasmedgec target/wasm32-wasi/release/rust.wasm out.wasm
Execute the WASM with the wasmedge with PyTorch supporting:
Always need to enable environment variable in new terminal

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/path/to/pytorch/installation:/path/to/installed/wasmedge/library
This command requires the TorchScript model created by tracing the example inputs over the HuggingFaces model. Add the path to the downloaded .pt file :

wasmedge --dir .:. out.wasm layoutlmv2-fintuned-funsd-jit.pt
For comparison, compile the file located in this [repository](https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing) at cpp_torchscript/torchscript_inference.cpp with the build command :
g++ torchscript_inference.cpp -I/path/to/libtorch/include/ -L/path/to/libtorch/lib/ -ltorch_cpu -lc10 -Wl,-rpath=/path/to/libtorch/lib/
Possible Solutions
As can be seen in the error, the Torchscript model calls some Python code, ad from the paths it can be seen it refers an anaconda environment i have made.
A similar https://github.com/huggingface/transformers/issues/12763 I found in the huggingfaces model suggested this might have something to do with versions of Python or transformers.

It is true that the path to Pytorch I use while executing Wasi-NN OR the CPP code provided here is different from the one in the anaconda environment.
So, I am not able to understand if this was the issue, how is the pure TorchScript API running correctly.

I am not well-versed as to what the TorchScript AI does in the background and how a traced model may react with the PyTorch backend in the Wasi-NN plugin as almost all the examples I have seen are of a scripted Pytorch model. Thank you to anyone who is able to help!

This is very damaging to my project

failure executing Deploy static content to Pages run #5

trigger

The code is unbearably bad

please change asap

failure executing Deploy static content to Pages run #7

There is a minor issue

I found some typos here and there.

failure executing Deploy static content to Pages run #3

the performance is too poor

failure executing Deploy static content to Pages run #4

guug

new issue @you

@jaykchen can't you find it

Response pauses indefinitely after 360MB

I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }
I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }
I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }
I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }

Statically link LLVM 15 or greater on MacOS needs external libraries

Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build
Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build

will this be too long? 38xx tokens

Description
I am currently working for the WasmEdge/WasmEdge#2356 issue in WasmEdge. I have encountered some unexpected behaviors while using the Wasi-NN plugin. I have created a reproducible example with all required descriptions at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing .

Current State
The code contains a rust folder with a simple reproducible example for the LayoutLMv2ModelForTokenClassification using dummy inputs to demonstrate a bug encountered
durng inference of the traced torchscript model created using it.

The cpp_torchscript folder within this contains a similar reproducible example using the dummy inputs and inferencing the TorchScript model from the TorchScript C++ API
directly. This is done in a similar fashion as to my understanding of what the WASI-NN plugin would do with it.

The inputs expected by the model are described at https://github.com/sarrah-basta/wasmedge_ai_testing/blob/main/layoutlmv2_with_wasi_ocr/README.md. However, currently for dummy inputs, I have focused on the creating tensors identical to the following shapes :

For index 0 Datatype: long int Shape: [1, 364]
For index 1 Datatype: long int Shape: [1, 364, 4]
For index 2 Datatype: float Shape: [1, 3, 224, 224]
For index 3 Datatype: float Shape: [1, 364]
For index 4 Datatype: long int Shape: [1, 364]

Changes made to the Wasi-NN plugin files within WasmEdge
The original error I encountered was as follows :

[2023-04-16 20:59:08.292] [error] [WASI-NN] Only F32 inputs and outputs are supported for now.
thread 'main' panicked at 'called Result::unwrap() on an Err value: NnErrno { code: 1, name: "INVALID_ARGUMENT", message: "" }', src/main.rs:107:9
stack backtrace:
[2023-04-16 20:59:08.293] [error] execution failed: unreachable, Code: 0x89
[2023-04-16 20:59:08.293] [error] In instruction: unreachable (0x00) , Bytecode offset: 0x001a8568
[2023-04-16 20:59:08.293] [error] When executing function name: "_start"
This error is being caused due to this check in the source code plugins/wasi-nn.

Thus, I made a few changes to the source code file to make sure I was creating tensors of the correct types. This diff can be found here

Bug Found
Following these changes, when trying to inference the model, the following error was encountered :

erminate called after throwing an instance of 'std::runtime_error'
what(): The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9015.py", line 17, in forward
_0 = self.classifier
_1 = self.dropout
_2 = (self.layoutlmv2).forward(input_ids, bbox, attention_mask, input, images, )

seq_length = ops.prim.NumToTensor(torch.size(input_ids, 1))
_3 = int(seq_length)
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9012.py", line 71, in forward
_39 = [position_ids0, visual_position_ids]
position_ids1 = torch.cat(_39, 1)
_40 = (_14).forward(input_ids, )

_41 = (_13).forward(position_ids0, )
_42 = torch.slice(bbox, 0, 0, 9223372036854775807, 1)
File "code/torch/torch/nn/modules/sparse/___torch_mangle_8540.py", line 9, in forward
def forward(self: torch.torch.nn.modules.sparse.___torch_mangle_8540.Embedding,
input_ids: Tensor) -> Tensor:
inputs_embeds = torch.embedding(self.weight, input_ids, 0, False, False)
~~~~~~~~~~~~~~~ <--- HERE
return inputs_embeds
Traceback of TorchScript, original code (most recent call last):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/functional.py(1913): embedding
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/sparse.py(145): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(751): _calc_text_embeddings
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(907): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(1233): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(934): trace_module
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(733): trace
/tmp/ipykernel_17852/2995848706.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3460): run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3400): run_ast_nodes
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3221): run_cell_async
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/async_helpers.py(129): _pseudo_sync_runner
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3016): _run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(2961): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/zmqshell.py(531): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/ipkernel.py(411): do_execute
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(729): execute_request
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(406): dispatch_shell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(499): process_one
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(510): dispatch_queue
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/events.py(81): _run
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(1844): _run_once
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(563): run_forever
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/tornado/platform/asyncio.py(215): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelapp.py(711): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/traitlets/config/application.py(992): launch_instance
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel_launcher.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(85): _run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(192): _run_module_as_main
RuntimeError: index out of range in self

Aborted (core dumped)
Since this is a error thrown from the TorchScipt API, I then tried reproducing the code in a similar fashion directly in C++.

Expected
Running the cpp code file torchscript_inference.cpp inputs of the same shapes to the Torchscript API results in a correct output with the same model and vocabulary files, and prints the resultant output tensor of shape [ CPUFloatType{1,364,7} ]}.

Environment
Hardware Architecture: (x86_64)
Operating system: (Ubuntu 22.04)
C++ Compiler version: gcc 11.3.0 cmake
CMake version: 3.22.1
CMake flags: -DCMAKE_BUILD_TYPE=Release -DWASMEDGE_PLUGIN_WASI_NN_BACKEND="PyTorch"
Steps to reproduce
Change the source code of the plugins/wasi_nn/wasinfunc.cpp file according to the given [diff](https://drive.google.com/file/d/1puiRnrsVwWWztsteSbttxU7U04t3H5i7/view?usp=sharing).
Build WasmEdge from source with the Wasi-NN plugin with Pytorch Backend following instructions at https://wasmedge.org/book/en/contribute/build_from_src/plugin_wasi_nn.html#build-wasmedge-with-wasi-nn-pytorch-backend
Clone the Rust crate at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing/rust
Download the .pt file of the model from [this drive link](https://drive.google.com/file/d/1jDlaVoSBIhVkkPi5iNG2m5l0ieqtB-j9/view?usp=sharing)
Compile and execute using WasmEdge as follows :
Compile the source code to WebAssembly:
cargo build --target=wasm32-wasi --release
The output WASM file will be at target/wasm32-wasi/release/rust.wasm
To speed up the processing, we can enable the AOT mode in WasmEdge with:

wasmedgec target/wasm32-wasi/release/rust.wasm out.wasm
Execute the WASM with the wasmedge with PyTorch supporting:
Always need to enable environment variable in new terminal

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/path/to/pytorch/installation:/path/to/installed/wasmedge/library
This command requires the TorchScript model created by tracing the example inputs over the HuggingFaces model. Add the path to the downloaded .pt file :

wasmedge --dir .:. out.wasm layoutlmv2-fintuned-funsd-jit.pt
For comparison, compile the file located in this [repository](https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing) at cpp_torchscript/torchscript_inference.cpp with the build command :
g++ torchscript_inference.cpp -I/path/to/libtorch/include/ -L/path/to/libtorch/lib/ -ltorch_cpu -lc10 -Wl,-rpath=/path/to/libtorch/lib/
Possible Solutions
As can be seen in the error, the Torchscript model calls some Python code, ad from the paths it can be seen it refers an anaconda environment i have made.
A similar https://github.com/huggingface/transformers/issues/12763 I found in the huggingfaces model suggested this might have something to do with versions of Python or transformers.

It is true that the path to Pytorch I use while executing Wasi-NN OR the CPP code provided here is different from the one in the anaconda environment.
So, I am not able to understand if this was the issue, how is the pure TorchScript API running correctly.

I am not well-versed as to what the TorchScript AI does in the background and how a traced model may react with the PyTorch backend in the Wasi-NN plugin as almost all the examples I have seen are of a scripted Pytorch model. Thank you to anyone who is able to help!

mention you!

@jaykchen

first issue

failure executing Deploy static content to Pages run #6

failure executing Deploy static content to Pages run #1

nothing special

god

test on remake

nothing serious

failure executing Deploy static content to Pages run #2

new issue, create new notify

@jaykchen

newnewnwe

@jaykchen

another issue

Response pauses indefinitely after 360MB

I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }
I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }
I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }
I'm hitting a problem when downloading this file with reqwest and rusttls: https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb

The download stops after around 360MB, and the response.chunk() future never completes.

I've put a full reproduction here: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb

The bug only occurs when using the reqwest feature rusttls-tls. I think this means that it only happens if the request is made using HTTP/2.

The bug also only occurs when downloading the gzip-compressed version of the file.

Steps to reproduce:
Clone the following repo: https://github.com/mstange/reqwest-with-rusttls-stops-after-360mb
Inside the local clone, run cargo run --release
Wait for a 640MB download to complete.
Expected results:
The program should complete and print Done! Downloaded 639588034 bytes in total.

Actual results:
The program stops downloading after around 360MB. The last printed line is usually something like Downloaded 361462551 bytes.
And then the response.chunk() future just never completes.

Workarounds
Any of the following changes make the download complete successfully:

Change https:// into http:// in the URL.
Remove the Accept-Encoding: gzip header.
Edit Cargo.toml to use "default-tls" instead of "rustls-tls" (causes HTTP/1.1 to be used)
The file can be downloaded successfully with curl -o chrome.dll.pdb --compressed "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb", which also uses HTTP/2. It decompresses to a 3GB file.

System configuration
I'm hitting this bug on macOS 13.1, with Rust 1.67.0.

Full code
src/main.rs:

#[tokio::main(flavor = "current_thread")]
async fn main() {
eprintln!("Downloading chrome.dll.pdb...");
let url = "https://chromium-browser-symsrv.commondatastorage.googleapis.com/chrome.dll.pdb/93B17FC546DE07D14C4C44205044422E1/chrome.dll.pdb";

let client_builder = reqwest::Client::builder();
let client = client_builder
    .no_gzip()
    .no_brotli()
    .no_deflate()
    .build()
    .unwrap();
let builder = client.get(url);
let builder = builder.header("Accept-Encoding", "gzip");
let mut response = builder.send().await.unwrap();

let mut downloaded_len = 0;
let mut last_reported_len = 0;
while let Ok(Some(chunk)) = response.chunk().await {
    downloaded_len += chunk.len();
    if downloaded_len >= last_reported_len + 1_000_000 {
        eprintln!("Downloaded {downloaded_len} bytes.");
        last_reported_len = downloaded_len;
    }
}

eprintln!("Done! Downloaded {downloaded_len} bytes in total.");

}
Cargo.toml:

[package]
name = "reqwest-with-rusttls-stops-after-360mb"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.11.14", default-features = false, features = [
"stream",
"rustls-tls",
# "default-tls",

] }
tokio = { version = "1.25.0", default-features = false, features = [
"macros",
"rt"
] }

oh new

@jaykchen why

am watching lots of reasons

to check @jaykchen if it works at all

LLvM15 issue

Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build
Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build
Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build
Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build
Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build
Description
Current State
The LLVM 15 and LLVM 16 on MacOS needs external dependencies when statically linking.

No external dependency on LLVM-14:

$ otool -L /opt/homebrew/opt/llvm@14/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@14/lib/libLLVM.dylib (compatibility version 1.0.0, current version 14.0.6)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.32.0)
After LLVM-15:

$ otool -L /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm@15/lib/libLLVM.dylib (compatibility version 1.0.0, current version 15.0.7)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

$ otool -L /opt/homebrew/opt/llvm@16/lib/libLLVM.dylib
/opt/homebrew/opt/llvm@16/lib/libLLVM.dylib:
/opt/homebrew/opt/llvm/lib/libLLVM.dylib (compatibility version 1.0.0, current version 16.0.1)
/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 30.0.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/opt/z3/lib/libz3.4.12.dylib (compatibility version 4.12.0, current version 4.12.1)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/opt/homebrew/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.4)
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
With this, the undefined references occurred when statically linking LLVM-15 or greater with WasmEdge.

undef: _ZSTD_compress
undef: _ZSTD_compressBound
undef: _ZSTD_decompress
undef: _ZSTD_getErrorName
undef: _ZSTD_isError
Undefined symbols for architecture arm64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_decompress", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_getErrorName", referenced from:
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef, llvm::SmallVectorImpl&, int) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, unsigned char*, unsigned long&) in libLLVMSupport.a(Compression.cpp.o)
llvm::compression::zstd::decompress(llvm::ArrayRef, llvm::SmallVectorImpl&, unsigned long) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture arm64
Expected
Statically link LLVM 15 or LLVM 16 successfully.

Environment
Hardware Architecture: aarch64 (M2 pro)
Operating system: MacOS 13.2.1
Steps to Reproduce
cmake -DWASMEDGE_LINK_LLVM_STATIC=On -GNinja -Bbuild .
cmake --build build

zapier will notify me over slack

@jaykchen

watching all activities now

@jaykchen watch

code can be a little bit more efficient if you do like this

here are the changes I propose

This is horrible

How can you being not able to address such a big problem

notify me on slack

Unexpected Behaviour inferencing a model with Wasi-NN plugin using Pytorch Backend

Description
I am currently working for the WasmEdge/WasmEdge#2356 issue in WasmEdge. I have encountered some unexpected behaviors while using the Wasi-NN plugin. I have created a reproducible example with all required descriptions at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing .

Current State
The code contains a rust folder with a simple reproducible example for the LayoutLMv2ModelForTokenClassification using dummy inputs to demonstrate a bug encountered
durng inference of the traced torchscript model created using it.

The cpp_torchscript folder within this contains a similar reproducible example using the dummy inputs and inferencing the TorchScript model from the TorchScript C++ API
directly. This is done in a similar fashion as to my understanding of what the WASI-NN plugin would do with it.

The inputs expected by the model are described at https://github.com/sarrah-basta/wasmedge_ai_testing/blob/main/layoutlmv2_with_wasi_ocr/README.md. However, currently for dummy inputs, I have focused on the creating tensors identical to the following shapes :

For index 0 Datatype: long int Shape: [1, 364]
For index 1 Datatype: long int Shape: [1, 364, 4]
For index 2 Datatype: float Shape: [1, 3, 224, 224]
For index 3 Datatype: float Shape: [1, 364]
For index 4 Datatype: long int Shape: [1, 364]

Changes made to the Wasi-NN plugin files within WasmEdge
The original error I encountered was as follows :

[2023-04-16 20:59:08.292] [error] [WASI-NN] Only F32 inputs and outputs are supported for now.
thread 'main' panicked at 'called Result::unwrap() on an Err value: NnErrno { code: 1, name: "INVALID_ARGUMENT", message: "" }', src/main.rs:107:9
stack backtrace:
[2023-04-16 20:59:08.293] [error] execution failed: unreachable, Code: 0x89
[2023-04-16 20:59:08.293] [error] In instruction: unreachable (0x00) , Bytecode offset: 0x001a8568
[2023-04-16 20:59:08.293] [error] When executing function name: "_start"
This error is being caused due to this check in the source code plugins/wasi-nn.

Thus, I made a few changes to the source code file to make sure I was creating tensors of the correct types. This diff can be found here

Bug Found
Following these changes, when trying to inference the model, the following error was encountered :

erminate called after throwing an instance of 'std::runtime_error'
what(): The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9015.py", line 17, in forward
_0 = self.classifier
_1 = self.dropout
_2 = (self.layoutlmv2).forward(input_ids, bbox, attention_mask, input, images, )
~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
seq_length = ops.prim.NumToTensor(torch.size(input_ids, 1))
_3 = int(seq_length)
File "code/torch/transformers/models/layoutlmv2/modeling_layoutlmv2/___torch_mangle_9012.py", line 71, in forward
_39 = [position_ids0, visual_position_ids]
position_ids1 = torch.cat(_39, 1)
_40 = (_14).forward(input_ids, )
~~~~~~~~~~~~ <--- HERE
_41 = (_13).forward(position_ids0, )
_42 = torch.slice(bbox, 0, 0, 9223372036854775807, 1)
File "code/torch/torch/nn/modules/sparse/___torch_mangle_8540.py", line 9, in forward
def forward(self: torch.torch.nn.modules.sparse.___torch_mangle_8540.Embedding,
input_ids: Tensor) -> Tensor:
inputs_embeds = torch.embedding(self.weight, input_ids, 0, False, False)
~~~~~~~~~~~~~~~ <--- HERE
return inputs_embeds
Traceback of TorchScript, original code (most recent call last):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/functional.py(1913): embedding
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/sparse.py(145): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(751): _calc_text_embeddings
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(907): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.py(1233): forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(934): trace_module
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/torch/jit/_trace.py(733): trace
/tmp/ipykernel_17852/2995848706.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3460): run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3400): run_ast_nodes
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3221): run_cell_async
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/async_helpers.py(129): _pseudo_sync_runner
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(3016): _run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/IPython/core/interactiveshell.py(2961): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/zmqshell.py(531): run_cell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/ipkernel.py(411): do_execute
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(729): execute_request
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(406): dispatch_shell
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(499): process_one
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelbase.py(510): dispatch_queue
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/events.py(81): _run
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(1844): _run_once
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/asyncio/base_events.py(563): run_forever
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/tornado/platform/asyncio.py(215): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel/kernelapp.py(711): start
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/traitlets/config/application.py(992): launch_instance
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/site-packages/ipykernel_launcher.py(17):
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(85): _run_code
/home/sarrah/anaconda3/envs/wasm_py_testing/lib/python3.8/runpy.py(192): _run_module_as_main
RuntimeError: index out of range in self

Aborted (core dumped)
Since this is a error thrown from the TorchScipt API, I then tried reproducing the code in a similar fashion directly in C++.

Expected
Running the cpp code file torchscript_inference.cpp inputs of the same shapes to the Torchscript API results in a correct output with the same model and vocabulary files, and prints the resultant output tensor of shape [ CPUFloatType{1,364,7} ]}.

Environment
Hardware Architecture: (x86_64)
Operating system: (Ubuntu 22.04)
C++ Compiler version: gcc 11.3.0 cmake
CMake version: 3.22.1
CMake flags: -DCMAKE_BUILD_TYPE=Release -DWASMEDGE_PLUGIN_WASI_NN_BACKEND="PyTorch"
Steps to reproduce
Change the source code of the plugins/wasi_nn/wasinfunc.cpp file according to the given diff.
Build WasmEdge from source with the Wasi-NN plugin with Pytorch Backend following instructions at https://wasmedge.org/book/en/contribute/build_from_src/plugin_wasi_nn.html#build-wasmedge-with-wasi-nn-pytorch-backend
Clone the Rust crate at https://github.com/sarrah-basta/wasmedge_ai_testing/tree/main/wasinn_testing/rust
Download the .pt file of the model from this drive link
Compile and execute using WasmEdge as follows :
Compile the source code to WebAssembly:
cargo build --target=wasm32-wasi --release
The output WASM file will be at target/wasm32-wasi/release/rust.wasm
To speed up the processing, we can enable the AOT mode in WasmEdge with:

wasmedgec target/wasm32-wasi/release/rust.wasm out.wasm
Execute the WASM with the wasmedge with PyTorch supporting:
Always need to enable environment variable in new terminal

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/path/to/pytorch/installation:/path/to/installed/wasmedge/library
This command requires the TorchScript model created by tracing the example inputs over the HuggingFaces model. Add the path to the downloaded .pt file :

wasmedge --dir .:. out.wasm layoutlmv2-fintuned-funsd-jit.pt
For comparison, compile the file located in this repository at cpp_torchscript/torchscript_inference.cpp with the build command :
g++ torchscript_inference.cpp -I/path/to/libtorch/include/ -L/path/to/libtorch/lib/ -ltorch_cpu -lc10 -Wl,-rpath=/path/to/libtorch/lib/
Possible Solutions
As can be seen in the error, the Torchscript model calls some Python code, ad from the paths it can be seen it refers an anaconda environment i have made.
A similar huggingface/transformers#12763 I found in the huggingfaces model suggested this might have something to do with versions of Python or transformers.

It is true that the path to Pytorch I use while executing Wasi-NN OR the CPP code provided here is different from the one in the anaconda environment.
So, I am not able to understand if this was the issue, how is the pure TorchScript API running correctly.

I am not well-versed as to what the TorchScript AI does in the background and how a traced model may react with the PyTorch backend in the Wasi-NN plugin as almost all the examples I have seen are of a scripted Pytorch model. Thank you to anyone who is able to help!