dinglufe / segment-anything-cpp-wrapper Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
When running the Run function in the loadImage function, it is always slow, is there any solution? Is the process a must every time a new picture is loaded? Can it be reused to reduce pre-treatment time?
I want to export mobile sam model to a new size 1024*xx. I hope xx can be a smaller number than 720. to speed up the inference speed.
But In export_pre_model.py, error coours at " import mobile_sam as SAM". Where is the mobile_sam.
hi there, what are the main differences right now for the automatic mode vs the original segment anything demo?
On the original demo, there is a method to turn a box into a mask. I think would be a nice addon here as well.
I have downloaded the sam hq models and executed this exporter script. This error occurs because the original segement_anything is imported.
If I change the code of the exporter to segement_anything_hq, the script runs without error, but does not export any models.
Could you please upload the exported models to github?
s:\Sources\Master\segment-anything-cpp-wrapper>python export_pre_model.py
Traceback (most recent call last):
File "s:\Sources\Master\segment-anything-cpp-wrapper\export_pre_model.py", line 56, in
sam = SAM.sam_model_registrymodel_type
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\znoopy2k\AppData\Local\Programs\Python\Python311\Lib\site-packages\segment_anything\build_sam.py", line 15, in build_sam_vit_h
return _build_sam(
^^^^^^^^^^^
File "C:\Users\znoopy2k\AppData\Local\Programs\Python\Python311\Lib\site-packages\segment_anything\build_sam.py", line 106, in _build_sam
sam.load_state_dict(state_dict)
File "C:\Users\znoopy2k\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Sam:
Unexpected key(s) in state_dict: "mask_decoder.hf_token.weight", "mask_decoder.hf_mlp.layers.0.weight", "mask_decoder.hf_mlp.layers.0.bias", "mask_decoder.hf_mlp.layers.1.weight", "mask_decoder.hf_mlp.layers.1.bias", "mask_decoder.hf_mlp.layers.2.weight", "mask_decoder.hf_mlp.layers.2.bias", "mask_decoder.compress_vit_feat.0.weight", "mask_decoder.compress_vit_feat.0.bias", "mask_decoder.compress_vit_feat.1.weight", "mask_decoder.compress_vit_feat.1.bias", "mask_decoder.compress_vit_feat.3.weight", "mask_decoder.compress_vit_feat.3.bias", "mask_decoder.embedding_encoder.0.weight", "mask_decoder.embedding_encoder.0.bias", "mask_decoder.embedding_encoder.1.weight", "mask_decoder.embedding_encoder.1.bias", "mask_decoder.embedding_encoder.3.weight", "mask_decoder.embedding_encoder.3.bias", "mask_decoder.embedding_maskfeature.0.weight", "mask_decoder.embedding_maskfeature.0.bias", "mask_decoder.embedding_maskfeature.1.weight", "mask_decoder.embedding_maskfeature.1.bias", "mask_decoder.embedding_maskfeature.3.weight", "mask_decoder.embedding_maskfeature.3.bias".
Hello, I'm trying to generate the onnx preprocessing models and I'm having the following error:
File "export_pre_model.py", line 84, in forward
return self.predictor.get_image_embedding(), torch.stack(self.predictor.interm_features, dim=0)
AttributeError: 'SamPredictor' object has no attribute 'interm_features'
I tested with the vit_b and vit_h models from https://github.com/facebookresearch/segment-anything/ and the error appear for both.
I installed the segment_anything library direct from the Meta github with pip.
Thanks for your Segment Anything CPP Wrapper.
Here is the Segment Anything CPP Wrapper for macOS.
This code is originated from Segment Anything CPP Wrapper and implemented on macOS app RectLabel. We customized the original code so that getMask() uses the previous mask result called as low_res_logits and retain the previous mask array for undo/redo actions.
Please let us know your opinion.
I meet the a problem , when i run the decoder in C++ ,my masks output shape is [1,4,0,0] , Like this
facebookresearch/segment-anything#294
masks out shape: 1 4 0 0
iou_predictions out shape: 1 4
low_res_masks out shape: 1 4 256 256
Do you know how to solve it?
Thanks
What should I do if the input size of the modified image is 640x480, and the obtained mask cannot coincide with the original image?
I was trying to run the program in VsCode and followed the steps as specified. I have installed vcpkg and the opencv folders are already in it.
But, when I run test.cpp, the following error is shown
"opencv2/opencv.hpp: No such file or directory".
Can you please advise how to solve?
Thank you,
Mareeta
I want to run the mobile sam at resolution 320x240. Any suggestions?
with export_pre_model.py, it still require one side to be 1024.
How to change the image size, still only 1024x720?I see that the recognition result is still different from the demo on the official website, I wonder if it's because of this reason?
help!!! I use it as this,but get error:not enough space: expected 4194304, got 0. My memory is enough.What's wrong?
thresholdMat = sam.getMask({500,400});
if change cuda ,loading model is fail,what's wrong?my cuda version is 11.7. GPU 1050TI
Hello, thank you for your efforts. Is it possible for you to support EdgeSam?
Thank you in advance!
MobileSAM & FastSAM are "fast", but there is also another direction: https://github.com/SysCV/sam-hq — it would be wonderful if you could add support for it as well :)
Can you add efficeientsam?
Can you show all the test project code.I cann't run in my compute after I create a new project and and test.cpp into project.
How can we support multiple points, and negative points?
Inference by cpu,the image size is 2000*2000,It takes a long time to run "sam.loadImage(image)",about3minutes,why is it happen?How can I improve the running speed?thanks for your answer!
bash ./vcpkg install opencv:x64-windows gflags:x64-windows onnxruntime-gpu:x64-windows
An error occurred mounting one of your file systems. Please run 'dmesg' for more details.
你好,请问您了解这是因为啥吗
I want that the model segments based on an input string, e.g. "red cars".
It seems that this is not yet supported in this implementation.
If I find time, I could try to add this. But I need a starting point.
Can you give some advice?
Thanks for sharing this repo. It really helps.
Every time I import a new image into the code, it loads particularly slowly, especially when executing the sam.getMask() function. But when I select different points on the same image, it is processed quickly. I wonder why? Is it because the code has to execute the decoder model every time a new image is loaded?
compare: When I enter the same bounding box but use a different photo, the code takes about 11200ms to process it;
When I use the same photo but different points,the code takes about 50ms to process it。
Preprocess device:cpu(i5-13400) Sam device:cpu(i5-13400)
When both I use CPUs for Preprocess and Sam, I wanted to ask if there was any way to reduce processing time?
it seems that the effect of mobile sam is significantly inferior to original sam , fastSAM maybe a better choice.
CMake Error at build/vcpkg/scripts/buildsystems/vcpkg.cmake:855 (_find_package):
Could not find a package configuration file provided by "OpenCV" with any
of the following names:
OpenCVConfig.cmake
opencv-config.cmake
Add the installation prefix of "OpenCV" to CMAKE_PREFIX_PATH or set
"OpenCV_DIR" to a directory containing one of the above files. If "OpenCV"
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
CMakeLists.txt:6 (find_package)
大佬能否留个联系方式
Progress: 0% 2023-07-26 17:04:50.4252918 [E:onnxruntime:test, cuda_call.cc:119 onnxruntime::CudaCall] CUDNN failure 4: CUDNN_STATUS_INTERNAL_ERROR ; GPU=0 ; hostname=MSI ; expr=cudnnFindConvolutionForwardAlgorithmEx( GetCudnnHandle(context), s_.x_tensor, s_.x_data, s_.w_desc, s_.w_data, s_.conv_desc, s_.y_tensor, s_.y_data, 1, &algo_count, &perf, algo_search_workspace.get(), max_ws_size);
2023-07-26 17:04:50.4346841 [E:onnxruntime:, sequential_executor.cc:494 onnxruntime::ExecuteKernel] Non-zero status code returned while running Conv node. Name:'/mask_downscaling/mask_downscaling.0/Conv' Status Message: CUDNN failure 4: CUDNN_STATUS_INTERNAL_ERROR ; GPU=0 ; hostname=MSI ; expr=cudnnFindConvolutionForwardAlgorithmEx( GetCudnnHandle(context), s_.x_tensor, s_.x_data, s_.w_desc, s_.w_data, s_.conv_desc, s_.y_tensor, s_.y_data, 1, &algo_count, &perf, algo_search_workspace.get(), max_ws_size);
Thank you for providing such good learning materials. I tried to use the export_pre_model.py file you provided to export the onnx model, and found that the input tensor format is uint8 instead of float32. Use the same method to convert the encoder on the official SAM model. , the input format of the obtained onnx model is float32, is this correct?
I've export onnx model ''mobile_sam.onnx" use script provided in mobile sam project,call it use v1.4.1,error msg as follows:
Now click on the image (press q/esc to quit; press c to clear selection; press a to run automatic segmentation) Ctrl+Left click to select foreground, Ctrl+Right click to select background, Middle click and drag to select a region 2023-07-12 10:52:46.2316412 [E:onnxruntime:, sequential_executor.cc:368 onnxruntime::SequentialExecutor::Execute] Non-zero status code returned while running Reshape node. Name:'Reshape_197' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\cpu\tensor\reshape_helper.h:41 onnxruntime::ReshapeHelper::ReshapeHelper gsl::narrow_cast<int64_t>(input_shape.Size()) == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,6,256}, requested shape:{1,10,8,32}
Sorry for offtop. I was trying to find a windows cli to run stable diffusion onnx models and accidentally found your repo.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.