smallcloudai / refact-lsp Goto Github PK
View Code? Open in Web Editor NEWLSP server for Refact, suitable for Sublime Text, and other editors
License: BSD 3-Clause "New" or "Revised" License
LSP server for Refact, suitable for Sublime Text, and other editors
License: BSD 3-Clause "New" or "Revised" License
run pkg_config fail:
pkg-config exited with status code 1
PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags openssl
The system library openssl
required by crate openssl-sys
was not found.
The file openssl.pc
needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
The PKG_CONFIG_PATH environment variable is not set.
HINT: if you have installed the library, try setting PKG_CONFIG_PATH to the directory containing openssl.pc
.
--- stderr
thread 'main' panicked at /home/macbook/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssl-sys-0.9.98/build/find_normal.rs:190:5:
Could not find directory of OpenSSL installation, and this -sys
crate cannot
proceed without this knowledge. If OpenSSL is installed and this crate had
trouble finding it, you can set the OPENSSL_DIR
environment variable for the
compilation process.
Make sure you also have the development packages of openssl installed.
For example, libssl-dev
on Ubuntu or openssl-devel
on Fedora.
If you're in a situation where you think the directory should be found
automatically, please open a bug at https://github.com/sfackler/rust-openssl
and include information about your system as well as this message.
$HOST = x86_64-unknown-linux-gnu
$TARGET = x86_64-unknown-linux-gnu
openssl-sys = 0.9.98
note: run with RUST_BACKTRACE=1
environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
Compiling openssl-sys v0.9.98
Compiling serde v1.0.195
Compiling futures-util v0.3.30
Compiling tracing v0.1.40
Compiling tokio v1.35.1
error: failed to run custom build command for openssl-sys v0.9.98
Caused by:
process didn't exit successfully: /home/macbook/refact-lsp/target/debug/build/openssl-sys-03b09a3967e9bcac/build-script-main
(exit status: 101)
--- stdout
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR
X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR unset
cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
OPENSSL_LIB_DIR unset
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR
X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR unset
cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
OPENSSL_INCLUDE_DIR unset
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR
X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR unset
cargo:rerun-if-env-changed=OPENSSL_DIR
OPENSSL_DIR unset
cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG
cargo:rerun-if-env-changed=OPENSSL_STATIC
cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
run pkg_config fail:
pkg-config exited with status code 1
PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags openssl
The system library openssl
required by crate openssl-sys
was not found.
The file openssl.pc
needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
The PKG_CONFIG_PATH environment variable is not set.
HINT: if you have installed the library, try setting PKG_CONFIG_PATH to the directory containing openssl.pc
.
--- stderr
thread 'main' panicked at /home/macbook/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssl-sys-0.9.98/build/find_normal.rs:190:5:
Could not find directory of OpenSSL installation, and this -sys
crate cannot
proceed without this knowledge. If OpenSSL is installed and this crate had
trouble finding it, you can set the OPENSSL_DIR
environment variable for the
compilation process.
Make sure you also have the development packages of openssl installed.
For example, libssl-dev
on Ubuntu or openssl-devel
on Fedora.
If you're in a situation where you think the directory should be found
automatically, please open a bug at https://github.com/sfackler/rust-openssl
and include information about your system as well as this message.
$HOST = x86_64-unknown-linux-gnu
$TARGET = x86_64-unknown-linux-gnu
openssl-sys = 0.9.98
note: run with RUST_BACKTRACE=1
environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
nor openai nor huggingface send [DONE] at the end of the stream, we should expect this when refact server act the same
We have an LSP function that does code completion, but it's a custom function.
Here's the specification from Microsoft:
So we want to keep the custom function, to keep existing plugins working, but also add all the necessary bits and pieces for 3.18 inline completion support.
When I add HTTP_PROXY env variable server works incorrect
ERROR refact_lsp::global_context:85: failed to load caps: error sending request for url (http://127.0.0.1:8008/coding_assistant_caps.json): error trying to connect: tcp connect error: Network is unreachable (os error 101)
problem appears on refact server
2024-04-15T18:31:37.412472Z ERROR refact_lsp::cached_tokenizers:90: failed to download tokenizer: failed to get response: HTTP status client error (401 Unauthorized) for url (https://huggingface.co/claude-instant-1.2/resolve/main/tokenizer.json)
2024-04-15T18:31:37.614502Z INFO refact_lsp::cached_tokenizers:48: downloading tokenizer from https://huggingface.co/claude-instant-1.2/resolve/main/tokenizer.json
main, 40b4bd7
2024-04-22T17:09:15.562125Z WARN refact_lsp::scratchpads::chat_utils_rag:383: range in search results is outside of file lines that actually exist /Users/valaises/PycharmProjects/data-collection/github/scripts/s1_api_grabber/gh_utils.py:0-59
2024-04-22T17:09:15.564576Z ERROR refact_lsp:70: Panic occurred: PanicInfo { payload: Any { .. }, message: Some(attempt to subtract with overflow), location: Location { file: "src/scratchpads/chat_utils_rag.rs", line: 385, col: 46 }, can_unwind: true, force_no_backtrace: false }
colorize_if_more_useful(linevec, omsg.line1-1, omsg.line2, &"nosymb".to_string(),
because omsg.line1 is a size and is 0
Somehow sometimes the file tokenizer.json is empty. To reproduce you can replace correct tokenizer.json with empty file.
Actual result:
Plugin says nothing about problem
LSP log:
ERROR refact_lsp::http::utils:31: /code-completion returning "400 Bad Request"
2023-12-12T22:06:20.910329Z ERROR refact_lsp::custom_error:54: client will see {"detail":"failed to load tokenizer: EOF while parsing a value at line 1 column 0"}
If you delete empty file manually it can help, but not always. In some cases it's continue to recreate empty tokenizer.json file (no connection to server with file?) and to fix this problem you need to find correct tokenizer.json and replace empty file.
Expected result:
Plugin working. If necessary, the plugin will independently solve problems with tokenizer.json (or inform the user about the problem in clear text about the error and how to fix it)
Tested plugins:
VS Code v2.0.108
JB 1.2.8
On Apple cpu, we still use x86 binary for refact-lsp, you can take built here:
https://github.com/smallcloudai/refact-lsp/actions (main branch)
We build it on github because it's the easiest and trusted way to build a binary
But they don't have an easy way to build for apple chips (we tried)
[to a user] So, what I think your "refact-lsp": error=86, Bad CPU type in executable" means, your mac refuses to run x86 for some reason. It works for other people, and it's just as fast, and the binary size is 15M vs 16M, and I type this on my MacBook Air with M1
If you can dig into this deeper, that would be fantastic! Like what do you (disable something?) have to do to get this error?
We should set a limit on how many tokens are available for RAG in chat. rn it takes all available tokens
Want:
found some problems with tokenizer loading:
first request to any model causes problem with tokenizer, error looks like this {"detail":"Tokenizer: failed to load tokenizer: EOF while parsing an object at line 6253 column 15"}
we should give path to tokenizer's data with model directly without "template": not all models stored in huggingface (3rd-party gpt model for example)
Failed to load tokenizer: EOF while parsing a value at line 1 column 0 when trying to use completion with wizardcoder/15b, llama2/7b, llama2/13b
2024-02-27T08:14:27.345588Z INFO refact_lsp::cached_tokenizers:128: using tokenizer "/home/svakhreev/.cache/refact/tokenizers/Refact/1.6B/vllm/tokenizer.json"
2024-02-27T08:14:27.369785Z INFO refact_lsp::scratchpads::completion_single_file_fim:179: single file FIM prompt 1984 tokens used < limit 1984
2024-02-27T08:14:27.369796Z INFO refact_lsp::http::routers::v1::code_completion:104: prompt 1.824527ms
2024-02-27T08:14:27.736596Z INFO refact_lsp::restream:75: forward to endpoint 366.00ms, url was http://0.0.0.0:8008/v1/completions
thread 'tokio-runtime-worker' panicked at src/completion_cache.rs:78:40:
byte index 5000 is not a char boundary; it is inside '标' (bytes 4998..5001) of ` selected_path = self.train_lines[self.train_labels[:] == c]
image_indexes = random.sample(range(0, len(selected_path)), 3)
#------------------------------------------#
# 取出两张类似的图片
# 对`[...]
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Use attached file with completion endpoint to reproduce the problem
dataloader.txt
error[E0635]: unknown feature `stdsimd`
--> /Users/danny/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-index-0.9.0/src/lib.rs:17:30
|
17 | #![cfg_attr(nightly, feature(stdsimd))]
| ^^^^^^^
For more information about this error, try `rustc --explain E0635`.
error: could not compile `lance-index` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
rustc 1.80.0-nightly (87293c958 2024-05-08)
It's not clear to me how to use it in neovim - any hints?
Find definitions of symbols near the cursor, attach file snippets from other files.
To compare user completion scores with and without RAG, we need to add information whether AST was used when completion was accepted.
Tried completion and get problem:
Log:
2023-11-17T08:50:44.863769Z INFO refact_lsp::lsp:211: file:///home/fupfv/refact/refact_data_pipeline/filters_fim_v2.py changed, save time: 12.383µs
2023-11-17T08:50:44.863798Z INFO refact_lsp::lsp:218: file:///home/fupfv/refact/refact_data_pipeline/filters_fim_v2.py changed, telemetry time: 1.73µs
2023-11-17T08:50:44.863880Z INFO refact_lsp::lsp:243: asked for completion
2023-11-17T08:50:44.931428Z INFO refact_lsp::http_server:253: 127.0.0.1:38810 POST /v1/code-completion body_bytes=283
2023-11-17T08:50:44.931489Z ERROR refact_lsp::custom_error:41: client will see {"detail":"Model 'Refact/1.6B/vllm' not found. Server has these models: []"}
Tried release plugin 1.0.52 - working.
Docker version: smallcloud/refact_self_hosting_enterprise nightly efd6631418cb
there's a 20s delay until files are "old enough" to get vectorized
2024-04-22T17:18:00.155762Z INFO refact_lsp::vecdb::vectorizer_service:67: 0 files to process, 257 files too new
2024-04-22T17:18:10.157983Z INFO refact_lsp::vecdb::vectorizer_service:67: 0 files to process, 257 files too new
2024-04-22T17:18:20.160417Z INFO refact_lsp::vecdb::vectorizer_service:67: 257 files to process, 0 files too new
2024-04-22T17:18:20.770521Z INFO refact_lsp::vecdb::vectorizer_service:104: have 257 unprocessed files
Which is OK, when user constantly updates the file, programming in it. But when IDE connects to Refact-lsp sending files list there should be no cooldown
We faced similar problem before but lloks like it's still here.
How to reproduce:
Expectations:
Reality:
New users who just installed plugin have no errors and completions.
Obviously birds match the project style much better, I suggest to replace frog with a bird.
Per the instructions in the Cargo Book best practices guide, Rust applications should always commit the Cargo.lock file in git to ensure a reproducible build.
This file is missing.
We are interested of course in a full working plugin for VS classic, but before going there, we need a minimal example.
We talked to C# experts who tried to decompile MS IntelliSense and Codeium, without much luck. They both too big, not clear what is going on, too many DLLs.
We need a minimal example of inline completion that we can start with. It would be great if it worked with the current LSP server, especially LSP 3.18 InlineCompletion as in #1 , but multiline "hello\nworld" will do, too.
The code must be short, clear. Create a project that compiles into VS classic plugin, demonstrates multiline gray text.
vecdb indexing is stuck forever when indexing a project with some python packages in it, node_modules, etc. Guess, vecdb search quality won't be great either when some many garbage in it
In this project there's an LSP server for Refact. It almost works well, but it adds text to a popup in Sublime, doesn't show the gray text.
The mini_auto_complete
option in Sublime replaces popup with a single line gray text, but it's not the right thing either.
The right thing would be multiline gray text, and it so happens the upcoming LSP standard version 3.18 has the official support for it, called InlineCompletion.
This issue/bounty is for implementing it correctly in Sublime.
Some tricky parts:
Showing gray text and popup might conflict, the popup has its strengths too. So the decision about which suggestion to show might be tricky, sure it's shouldn't show both at the same time.
Grey text should not increase visible line counters, for example a code suggestion at line 100 that has 5 lines shouldn't make the next line 106 before it's accepted.
They should accept the changes to implement this at Sublime and Sublime LSP plugin.
A discussion at Sublime forum, they also have Discord.
example:
Ooops database is broken!
Last error message: {}
You can report this issue here:
https://github.com/smallcloudai/refact-lsp/issues
Also you can run this to erase your db:
/path1/path2/refact-lsp --rag-db-erase
After that restart this LSP server or your IDE.
Server gives default model with lora Refact/1.6B:run:checkpoint
If you directly ask server for this model it will work, but refact-lsp still asks for model without a lora Refact/1.6B
Looks like we're missing this postfix somewhere
File list is the same. Results from preview don't pass through post processing, which causes inconsistency
refact-lsp version: 40b4bd7
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.