kevmo314/magic-copy

Magic Copy is a Chrome extension that uses Meta's Segment Anything Model to extract a foreground object from an image and copy it to the clipboard.

o shit waddup

magic-copy's Issues

Docker instructions in README

docker run --gpus all -p 8000:8000 segment-anything

throws the following error

docker: Error response from daemon: No command specified.

I'm sure there's an obvious command to add to the end of that line but I've never used Docker so I don't know what command that would be. The included Dockerfile doesn't specify a command either.

Upgrade to sam-hq

Would it be possible to implement the new sam-hq model? It seems to perform better in the selection of fine details.

https://github.com/SysCV/sam-hq

Can't use on local files

Steps to reproduce:

1. Open file from local file system, e.g. file:///C:/Users/home/Pictures/test.png
2. Right click on the image and bring up Magic Copy
3. Once magic copy is up, notice that clicking on any part of the image does not extract any objects

Undo Button Not Functional For First Mask

As titled. The undo button does not seem to work for me for the first mask created.

OS: Windows
Browser: Chrome latest stable build

Manifest unreadable error when trying to install

I tried to install the extension on chrome by following these steps.

Alternatively, the extension can be installed manually:

Download the repository as a ZIP file.
Extract the ZIP file.
In Chrome, go to chrome://extensions/, enable "Developer mode", and click "Load unpacked".
Select the folder where the extension was extracted.

After enabling developer mode and selecting "Load unpacked", I get this error:

Failed to load extension
File ~\Downloads\magic-copy-main
Error Manifest file is missing or unreadable
Unable to load manifest.

I've tried deleting, downloading and extracting the zip file again, but that didn't work. When loading unpacked I'm selecting the folder where the files were extracted, not the zip file.
I'm using Chrome version 112.0.5615.50 64 bits (latest).
Any ideas of what could be causing this?

"background.service_worker is currently disabled" in Firefox

I get zip from releases

Are there any plans for Firefox support?

Are there any plans for Firefox support?

Release on Firefox Add-ons Store

This issue will close when the extension is available on the Firefox Add-ons Store. If you would like to be notified, hit the "Subscribe" button in the sidebar.

support locally hosted version?

can you add support for a locally run instance of segment anything?

Permission 'file:///*' is unknown or URL pattern is malformed.

Permission 'file:///*' is unknown or URL pattern is malformed.
it can not work.

No build script in package.json causing execution errrors in command line and Dockerfile

"scripts": { "watch:chrome": "webpack --config webpack/webpack.chrome.dev.js --watch", "build:chrome": "webpack --config webpack/webpack.chrome.prod.js", "watch:firefox": "webpack --config webpack/webpack.firefox.dev.js --watch", "build:firefox": "webpack --config webpack/webpack.firefox.prod.js", "watch:figma": "webpack --config webpack/webpack.figma.dev.js --watch", "build:figma": "webpack --config webpack/webpack.figma.prod.js" },

The above has no "build" script. So when you run npm run build from the README you get an error npm ERR! Missing script: "build". And when you run docker build -t segment-anything . you get the same error because the Dockerfile runs npm run build

The obvious quick fix is to rename your browser of choice to build:

"scripts": { "watch": "webpack --config webpack/webpack.chrome.dev.js --watch", "build": "webpack --config webpack/webpack.chrome.prod.js", "watch:firefox": "webpack --config webpack/webpack.firefox.dev.js --watch", "build:firefox": "webpack --config webpack/webpack.firefox.prod.js", "watch:figma": "webpack --config webpack/webpack.figma.dev.js --watch", "build:figma": "webpack --config webpack/webpack.figma.prod.js" },

But I'm wondering if there's a more elegant solution? Like mapping build to build:chrome for instance on initial setup.

Sorry if anything I said is imprecise or there's an obvious solution, I'm new to a lot of these tools

Fix CORS restrictions on Firefox

For some reason, Firefox seems to be very unhappy with the cross-origin requests out of the background script from the extension. I've posted about this here: https://discourse.mozilla.org/t/bypassing-cors-restrictions-on-background-scripts/113839/2

But I have no idea how to fix this after a few hours of trying different approaches. I don't think we can reliably do the fetch in the content script either.

This appears to be the last remaining issue for Firefox support.

why chrome extension did't work?

is the default Segment Anything Model endpoint unaccessible?

Option to crop to selected mask

Hey! First off, love the implementation and thanks for your work. Glad to see local mode implemented.

When selecting smaller parts of a larger image, the overall image size remains the same. It'd be great to be able to trim the transparent edge pixels so the resulting file doesn't need further cropping or editing in an external program. Making it an option would be best for people who want to keep the same image size for easy overlaying/compositing.

Release on Chrome Web Store

This issue will close when the extension is available on the Chrome Web Store. If you would like to be notified, hit the "Subscribe" button in the sidebar.

Suggestion - Integrate MobileSAM into the pipeline for lightweight and faster inference

Reference: https://github.com/ChaoningZhang/MobileSAM

Our project performs on par with the original SAM and keeps exactly the same pipeline as the original SAM except for a change on the image encode, therefore, it is easy to Integrate into any project.

MobileSAM is around 60 times smaller and around 50 times faster than original SAM, and it is around 7 times smaller and around 5 times faster than the concurrent FastSAM. The comparison of the whole pipeline is summarzed as follows:

Best Wishes,

Qiao

Support negative regions

Right clicking on the image should add a negative marker.

Suggestion: SAM-HQ + Negative

Will you please consider to upgrade to SAM-HQ it is much more accurate.
Also adding a Negative for small segments that are unwanted will be very appreciated.

Thanks ahead and please keep up the good work!💙

Uncaught (in promise) Error: Could not establish connection. Receiving end does not exist.

Uncaught (in promise) Error: Could not establish connection. Receiving end does not exist.

iframes are also triggered

It seems like Magic Copy triggers other iframes in addition to the main body. We should track the contextmenu events and only trigger on the iframe that contains the image matching the source url.

buildcrx.sh

how to use this file? According to the process, the Google plugin cannot be loaded

The compiled source code cannot run properly. ort.min.js:1802 wasm streaming compile failed: LinkError: WebAssembly.instantiate(): Import #36 module="a" function="K": function import requires a callable

Hello, I downloaded the source code of your open-source project magic-copy from releases/tag/v1.2.0 and installed dependencies locally. After running npm run watch and adding the plugin to Google Chrome, I found that it doesn't work properly. It shows an error like the one in the screenshot below.

However, if I download the compiled magic-copy-chrome.zip from releases/tag/v1.2.0 and install it directly in the browser, it works fine. I also checked the format of onnxruntime-web inside sandbox.js and found that they are different. Could this be an issue with the version of onnxruntime-web? I tried upgrading to the latest version of onnxruntime-web, but the issue persists.