samrawal/emacs-secondmate

An open-source, mini imitation of GitHub Copilot for Emacs.

• 👋 Hi, I’m @samrawal!
• 🩺 I'm currently an M.D. student at the Carle Illinois College of Medicine at the University of Illinois, Urbana-Champaign.
• 🤖 I'm also working on Natural Language Processing research applied to neurology records:
  • 🌵 Clinical AI software is at @cactiML and cactiml.com
• 💻 More about me at samrawal.com.
• ✏️ I write at samrawal.substack.com.
• 🐦 I'm on Twitter @samarthrawal.

emacs-secondmate's Issues

Provide installation instructions for the Python dependencies

It's customary to provide a requirements.txt file to install Python dependencies. From what I can tell this would be Flask and https://huggingface.co/transformers/installation.html.

Missing license

Please add one, otherwise this isn't open-source software and hinders contribution.

Allow option of using HuggingFace Accelerated Inference API

If the user has access to the HuggingFace Accelerated Inference API [0], allow them to configure appropriate API Tokens and inference the model via the API rather than having to run the Language Model server locally.

[0] https://api-inference.huggingface.co/docs/python/html/index.html

Support launching server automatically from Emacs

There isn't really a reason why the server couldn't be launched automatically from Emacs. This requires figuring out a few things though:

• Locate a suitable Python interpreter.
• Locate the Python script relatively to the package file.
• Query the user for connection options if necessary (unless they set them explicitly).
• Pass those options to the Python script and launch it asynchronously.

The design for this can be lifted from other interaction packages, such as CIDER or Sly. See also #3.

Another thing worth thinking about is whether this really needs to use a HTTP service. If this is designed to run on localhost, talking with a subprocess makes a lot more sense and avoids security issues. I wrote a bunch before, though for interaction via Scheme rather than Emacs Lisp.

Make nano port

Please!

Could you explain how to get started with this/install via a package manager in the README?

Hi, I love the project and would like to try it out. However, as an emacs noob I'm not sure how to get this started with something like use-package.

Additionally, does the plugin automatically launch the server, or would you have to do that manually? If so, could it be made to be automatic?

fine tuning?

Hey there,

thanks for sharing your work! Are you planning to fine-tune the model?

Contacting host...

Hi, I am at the last stage of the setup and emacs just gets stuck on Contacting localhost:9900 and the terminal outputs Setting pad_token_id to eos_token_id:50256 for open-end generation but nothing happens.
What am I meant to do?

Convert Emacs code to an Emacs package

• Remove top-level key binding commands and instead document what commands users should bind in their init file.
• Add package header and footer.
• Make sure that the package adheres to MELPA's coding guidelines.

See also #3.

Not enough memory error running server.py

device is set to "cuda" and changing it to "cpu" returns the same error

python server.py

Traceback (most recent call last):
  File "A:\xxxxxxxxx\emacs-secondmate\serve\server.py", line 10, in <module>
    model = AutoModelForCausalLM.from_pretrained(modelname)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\auto\auto_factory.py", line 395, in from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\modeling_utils.py", line 1179, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 905, in __init__
    self.transformer = GPTNeoModel(config)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 708, in __init__
    self.h = nn.ModuleList([GPTNeoBlock(config, layer_id=i) for i in range(config.num_layers)])
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 708, in <listcomp>
    self.h = nn.ModuleList([GPTNeoBlock(config, layer_id=i) for i in range(config.num_layers)])
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 542, in __init__
    self.mlp = GPTNeoMLP(inner_dim, config)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\transformers\models\gpt_neo\modeling_gpt_neo.py", line 521, in __init__
    self.c_fc = nn.Linear(embed_dim, intermediate_size)
  File "C:\Users\xxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\torch\nn\modules\linear.py", line 81, in __init__
    self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
RuntimeError: [enforce fail at ..\c10\core\CPUAllocator.cpp:79] data. DefaultCPUAllocator: not enough memory: you tried to allocate 104857600 bytes.