abo-abo/lpy

Minimal Python IDE for GNU Emacs

lpy

Deceptively minimalistic Python IDE for GNU Emacs.

Setup

Install from MELPA and enable lpy-mode.

Introduction

Q: How many keys does it take to define/redefine a Python function? A: One - e.

Arrow navigation

Q: Does this also apply to classes, import statements, for loops and plain one liners? A: Of course.

Q: How many keys does it take to move to the next Python function? A: One - j.

Q: Does this also apply to classes, import statements, for loops and plain one liners? A: Of course.

Q: So, how do I eval a statement, move to the second statement, and eval the second statement? A: eje.

Q: How to move back to the first statement? A: k.

Q: How do I descent into a compound statement, like def? A: l.

Q: How do I ascent back? A: h.

Q: Back to eval, how do I eval something that’s not a compound statement? A: Mark as much legal code as you like with a region and press e.

Q: Any shortcut for marking a symbol in order to eval it? A: M-m.

Semantic navigation

Q: How can I select a top-level symbol in the current file? A: g.

Q: How do I follow to the current symbol’s definition? A: M-..

Q: How do I go back from there? A: =M-,=.

Q: Does =M-,= also bring me back after g? A: Yes.

Eval in more detail

Q: Where do I see the result of the eval? A: In the minibuffer.

Q: What if the result of the operation is None? A: Then (ok) is printed to confirm that the operation went through.

Q: What if an uncaught exception is thrown? A: The exception will be displayed in the minibuffer, highlighted in red.

Q: Is it possible to be productive without ever seeing the REPL buffer? A: Yes.

Q: How do I see the REPL buffer? A: C-c C-z.

Q: Can I have more than one REPL session? A: Yes, use xp to select session or start a new one.

Q: Does pressing e on a variable assignment output nothing because the result is None? A: No, magic is used to make it print the new value of the variable.

Q: Where else is e magical? A: e on __file__ will print the actual file name.

Q: Any more? A: e on a single line return statement will print the return result.

Q: How do I insert the result of the eval into the buffer? A: E.

Q: How do I insert the result of the eval into the buffer as a comment? A: 2e.

Q: Howto easily select things to eval? A: M-m to get into special. Then + and - to expand/shrink selection with expand-region.

Debugging

Q: How do I step into an expression? A: xj.

Q: Where does this work? A: Function or method call, possibly with the result being assigned to a variable.

Q: What does that do exactly? A: Evaluate the function arguments, jump to the function definition, and store the function arguments into global variable with appropriate names.

Q: So if all I have is sys.argv I can use xj (combined with j, l and e) to step into my program as deep as I want? A: Yes.

Q: What if I just want to place a breakpoint to speed things up? A: To place a breakpoint, throw any exception. Then, enter lp.pm() into the REPL.

Q: lp.pm() sounds weird, how can I see how it works? A: Enter into the REPL: os.path.realpath(42) which will raise an exception a few functions down the line. Then enter lp.pm().

Q: Wait, so this is like pdb? A: Yes, very similar.

Q: So why go through the trouble? A: To get global variable context and proceed with e and j.

Q: Got it. Anyway, I see File "/usr/lib/python2.7/posixpath.py", line 61, Frame [3/3]:, what does it mean? A: realpath called _joinrealpath, which called isabs, which raised an Exception.

Q: Three functions (realpath, _joinrealpath, and isabs), that’s what 3/3 means. And I’m in isabs now? A: Yes.

Q: How do I open the source code for isabs? A: It’s a link: use the mouse or ace-link or next-error.

Q: I’m at the definition of isabs now, was its argument value stored somewhere? A: Yes, if you eval s, you get 42 - the value that propagated from os.path.realpath(42).

Q: isabs is boring. How do I go up the stack? A: Enter up into the REPL.

Q: I entered up, and now the frame is 2/3 and I’m at def _joinrealpath(path, rest, seen). Does this mean I can eval path, rest, and seen since they were propagated from the os.path.realpath(42) call? A: Yes.

Q: And entering up again will bring me to 1/3 and realpath? A: Yes.

Q: How do I go back down stack? A: Enter dn.

Notebooks

Q: Is this like IPython? A: Yes.

Q: But it’s different how? A: You use Emacs instead of a browser, and the cells are self-contained in comments.

Q: Sounds nice, but I’m not sold yet? A: It’s like Org-mode embedded in Python code.

Q: So I can fold / unfold each cell? A: Yes.

Q: And eval/reeval it with e? A: Yes.

Q: How about organizing cells in a hierarchy? A: Also possible.

Q: And e works on the hierarchy as well? A: Yes.

Q: Are cells actually called different or something? A: Yes, they’re outlines.

Q: How do I make an outline named Includes? A: Enter #* Includes.

Q: And like in Org-mode, the amount of stars is that outline’s level? A: Yes.

Q: Do M-left and M-right work like in Org-mode for promotion/demotion? A: Yes.

Q: How to fold/unfold an outline? A: i.

Q: How do I fold/unfold all outlines? A: I.

Q: How do I make a table of contents? A: 2I.

Q: When I press e on an outline it evaluates itself and the result is echoed; how do I make the result insert itself into the buffer instead? A: End the outline name in : (semicolon)

Q: How do I clean up all inserted results? A: M-x lpy-clean.

Q: Any more neat stuff about outlines? A: Yes, outlines are structured statements and parents to the top-level statements.

Q: So j / k and h / l, and even e treat outlines as statements? Neat. A: Yeah.

Completion

Q: How do I get completion at point? A: Press C-M-i.

Q: Is this static completion or does it depend on the REPL state? A: Both. The static one is more convenient and is tried first. But the dynamic one is very reliable, since it knows exactly on which type of object you’re operating.

Q: But dynamic completion won’t work unless my current object has a value in the REPL? A: Correct.

Q: What’s used for static completion? A: Jedi.

Inline hints

Q: How do I look up the function arguments of the current function? A: Toggle C-2.

Q: What about the docstring? A: Toggle C-1.

lpy's Issues

Support for python-ts-mode?

Now that treesitter is in Emacs proper, it sounds reasonable to incorporate it.

Link for context: https://archive.casouri.cc/note/2023/tree-sitter-in-emacs-29/index.html

unexpected eval behavior

Eval following expression with 2e:

(5.94 + 4.36) / 2

expected behavior:

(5.94 + 4.36) / 2
#  => 5.15

actual behavior:

(5.94 + 4.36)
#  => 5.15
/ 2

The / 2 was below the output after the evaluation. Any ideas?

Space auto-inserted around '=' in keyword arguments

When assigning values to keyword arguments in a function call lpy automatically inserts spaces around the = operator. This goes against pep8 recommendation.

name 'lp' is not defined on pspecial-lispy-eval

When calling pspecial-lispy-eval in certain places (variable assignment within functions), I get the following:

NameError: name ’lp’ is not defined
Traceback (most recent call last):
  File "<string>", line 19, in __PYTHON_EL_eval
  File "/home/usr/try.py", line 2, in <module>
    x = 5

Interestingly enough, the following fixes it:

(define-key lpy-mode-map (kbd "e") (lambda ()
                                       (interactive)
                                       (pspecial-lispy-eval)))

which I don't really understand why.

Hotkeys default to self-insert-command

I love lispy and use it every day, and would love to try lpy. But I have encountered a problem that I don't know quite how to debug myself. None of the hotkeys (like 'e') work when called without a region. Evaluation with region works fine. Lispy works well, and I have tried lpy with both lispy-mode on and off.

I would love to provide more information regarding this problem, but I am not sure how to even start. Any ideas?

EDIT: I am on native emacs 28.5, and it seems loading LPY with emacs -q does not solve the issue. C-1 and C-2 work correctly.

EDIT 2: It seems pressing 'e' on the first character of the line (instead of before the first paren) evaluates it, which contradicts the documentation (in lpy.el). Is this expected behavior?

eval outline doesn't work on some corner cases

eval doesn't work on following outline

# * Title 1

mylist = []
mylist.append([
    'a',
    'b',
    'c'
])

Here is the traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jw/work/examples/debug.py", line 5
    ’c’
      ^
SyntaxError: invalid syntax
  File "<stdin>", line 1
    ])
    ^
SyntaxError: invalid syntax

But it does work when reformatting the code as follows:

# * Title 1

mylist = []
mylist.append(['a', 'b', 'c'])

Wrong value on evaluate region, different result when paste into python process buffer

Try this code snippet:

class Robot:
    __counter = 0
    def __init__(self):
        type(self).__counter += 1
    @classmethod
    def RobotInstances(cls):
        return cls, Robot.__counter

print(Robot.RobotInstances())
x = Robot()
print(x.RobotInstances())
y = Robot()
print(x.RobotInstances())
print(Robot.RobotInstances())

If the snippet is evaluated by paste into a python process buffer, the result is correct. If first select it and eval with "e", the result is wrong.

lpy-slurp and lpy-barf print weird characters

Specifically,

^[x

Allow switching to standard Python shell

Currently lpy filters only processes with particular names starting with lispy-python-* - this limits the ability to attach to shells started with other Python frameworks that can be used in parallel.

"j" "k" at special position can not cross outline

Since we already have "J" and "K" for moving outline positions, maybe let "j" and "k" work as lispy for elisp buffer.

Outline imcompatible with PEP8 stype E265

The outline in lpy is
#* Level 1
However this does not comply with pep8 E265: Block comments should have a space after #.
Does it make sense to change it to something like
# * Level 1
to please the linter? Thanks.

Make worf-goto also work for lpy

Hi, Oleh

Could you please consider adding worf-goto function to lpy? Currently I use counsel-outline, but it is not as visually appealing as worf-goto. Thanks!

Performance issue with C-2

It seems C-2 is much slower than C-1 on my Windows laptop.

In my understanding, this is because C-2 starts a new python process every time. Why not use the existing process like C-1 does? How to improve the performance?

lpy-soap adds whitespace in quoted strings

For instance, when writing a date "XX-YY-ZZ" spaces will be added after typing '-', which is unexpected since Python formatting rules do not apply to the string itself.

Very hard to understand

First I struggle with the concept that many things only work depending where your cursor is, and for some commands it makes sense for other not.

The biggest struggle and I am still not sure if I am correct was to realize that lispy-mode must be activated so that lpy-mode functions properly? I needed probably 1 hour to guess / try+error that alone.

Then maybe for a lispy-user all that stuff makes a lot of sense but for me that years ago looked over lispy with the result that it's not good enough / does not compete with paredit even with a far stretch after 30 mins trying to use it, didn't really get how I am supposed to use it.

I mean I am a paredit user so I know as example slurf / barf, something I enjoy and that seems to be implemented, but not as interactive commands instead slurf seems to be hidden in "split" which seems to work kind of, but then barf I have no idea how to make it work and looking over the source for minutes didn't make it that much clearer.

And then the shortcuts not only do you have to know where the cursor must exactly be or it just ignores your commands but you have to know which 20 shortcuts do what you want... that are not very intuitive and named in a way that also makes not much sense not to mention documentation.

Well I just looked at lispy-mode and that would probably better for me, but I still send this report back as feedback especially (if I am correct) to not mention that you need lispy mode active to make this mode work is really a big trap.

Combining lpy and lispy

Hi abo-abo - when I activate lispy-mode and lpy together, most lpy functionality ceases to function. Is there a way you recommend to work with both modes in the same buffer?

Thanks!

Traceback on wrong line

Whenever I get a traceback, I don't get any helpful information. I always get the error referring to the first line:

Traceback (most recent call last):
  File "<string>", line 8, in __PYTHON_EL_eval
  File "/home/pollock/.pyenv/versions/3.9.0/lib/python3.9/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
  File "/home/pollock/try.py", line 1
    import numpy as np

Is this expected behavior?

Support async eval

Hi, Oleh

Could you please consider adding async eval to lpy? Or if you don't have time to do that, could you please point a direction for me? I wan't to help but I'm not skilled at elisp or python.

Can not eval decorated function or class in place

For example(point before @):

def honirific(cls):
    class HonirificCls(cls):
        def full_name(self):
            return "Ms." + super(HonirificCls, self).full_name()

    return HonirificCls


@honirific
class Name(object):
    def __init__(self, first_name, last_name):
        self.first_name = first_name
        self.last_name = last_name

    def full_name(self):
        return " ".join([self.first_name, self.last_name])

Cleanup only current outline / statement output

Handy feature request (to extend "C" that cleans up the entire buffer)

Evaluate doesn't work for statement that needs user input

For example:

while True:
    try:
        n = input("Please enter an integer: ")
        n = int(n)
        break
    except ValueError:
        print("No valid integer! Please try again ...")
print("Great, you successfully entered an integer!")

The desired behavior is to switch to the python process buffer and wait for user input

lpy hangs when eval outline

When I try to eval outline with the following example, lpy hangs:

#* Outline

print("test") # when there is a comment at end of the line, this will hang

If I delete the comment at end of the line, the issue doesn't persist.

Incorrect evaluation of list comprehension

Hi,

I found following evaluation gives wrong results:

mylist = [1, 2, 3]
mylist2 = [
    x for x in mylist
    if True
]

After evaluation, mylist2 is empty while the correct results should be [1, 2, 3].

Edit:

I found this works:

mylist = [1, 2, 3]
mylist2 = [
    x for x in mylist if True
]

[request] Release tags?

Hello,

It'd be nice to have release tags, for downstream distributions such as GNU Guix. Currently we take a snapshot of the latest master branch (commit) some times, but that isn't ideal.

Thanks for your consideration!

decorators

Python decorators with '@' syntax don't seem to evaluate.
for example:

def up_down(func):
    def wrap(w):
        old_word = func(w)
        new_word = ""
        for i in range(len(old_word)):
            if i == 0 or i %2 == 0:
                new_word += old_word[i].upper()
            else:
                new_word += old_word[i].lower()
        return new_word
    return wrap

@up_down
def greet(name):
    return "my oh my, {}".format(name)

raises a syntax error:

  File "<ipython-input-504-2c7b5957ecab>", line 2
    import codecs, os;__pyfile = codecs.open(’’’/tmp/py922Z80’’’, encoding=’’’utf-8’’’);__code = __pyfile.read().encode(’’’utf-8’’’);__pyfile.close();os.remove(’’’/tmp/py922Z80’’’);exec(compile(__code, ’’’/tmp/py922Z80’’’, ’exec’));

How to make lpy work with evil?

I googled but found nothing on this topic. Any suggestions? Thanks!

middleware sometimes not loaded

Borrowed from :

  (defun elpy-shell--ensure-shell-running ()
    "Ensure that the Python shell for the current buffer is running.
  
  If the shell is not running, waits until the first prompt is visible and
  commands can be sent to the shell."
    (with-current-buffer (process-buffer (elpy-shell-get-or-create-process))
      (let ((cumtime 0))
        (while (and (when (boundp 'python-shell--first-prompt-received)
                      (not python-shell--first-prompt-received))
                    (< cumtime 3))
          (sleep-for 0.1)
          (setq cumtime (+ cumtime 0.1)))))
    (elpy-shell-get-or-create-process))

Finally I have found the solution.

  (defun lispy--python-poetry-name ()
    (let* ((venv-name (if conda-project-env-path
                          (car (last (file-name-split conda-project-env-path)))
                        "default"))
           (project (project-current)))
      (if project
          (concat venv-name "-" (car (last (file-name-split (project-root project)))))
        venv-name)))
  
  (defun lispy--python-proc (&optional name)
    (let* ((proc-name (or name (lispy--python-proc-name)))
           (project (project-current))
           (process (get-process proc-name)))
      (if (process-live-p process)
          process
        (let* ((python-shell-font-lock-enable nil)
               (inferior-python-mode-hook nil)
               (buffer
                (let ((python-shell-completion-native-enable nil)
                      (default-directory (or (and project (project-root project))
                                             default-directory)))
                  (python-shell-make-comint (python-shell-parse-command) proc-name nil nil))))
  
          (with-current-buffer buffer
            (let ((cumtime 0))
              (while (and (when (boundp 'python-shell--first-prompt-received)
                            (not python-shell--first-prompt-received))
                          (< cumtime 3))
                (sleep-for 0.1)
                (setq cumtime (+ cumtime 0.1)))))
  
          (setq lispy--python-middleware-file
                (if (file-name-absolute-p lispy-python-middleware-file)
                    lispy-python-middleware-file
                  (expand-file-name "lispy-python.py" lispy-site-directory)))
          (setq lispy--python-init-file lispy-python-init-file)
          (setq process (get-buffer-process buffer))
          (unless (process-live-p process)
            (pop-to-buffer buffer)
            (user-error "Could not start %s" python-binary-name))
          (with-current-buffer buffer
            (python-shell-completion-native-turn-on)
            (setq lispy-python-buf buffer)
            (lispy-python-middleware-reload)))
        process)))

Consider calling functions on region for functions defined as pspecial-

lpy--insert-or-call could call the function on the region (e.g - enabling calling python-shell-send-region on the current region rather than the statement)

A warning about lpy-switch-to-shell

when C-c C-z, the emacs shows me this:
/anaconda3/bin/ipython:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
#!/anaconda3/bin/python

And the ipython is always started with cell [2] instead of [1], this is really upsetting.

Add to MELPA

Please add the packge to MELPA :)

Feature request: built lpy upon anaconda mode

I found goto definition requires semantic-directory.el, there are many overlap in functionalities in many packages. Anaconda-mode is a small and quite nice package.

Didn't output timeit results

2e doesn't output timing results

Take following code snippet for example:

import timeit
timeit.timeit(
    "obj.method()", """
class SomeClass:
    def method(self):
        pass
obj= SomeClass()
""")

It should output:

#=> 0.21603588491238312

However, the number was not showing.

lpy-right fails in file without headings

lpy-right fails for me because outline-back-to-heading fails in zo-bnd-subtree with "Before first heading"

Allow customization of evaluated block format

This is a feature request.

For instance,

# #+begin_example
# =>
... 
# #+end_example

which would allow outorg to export the results in html or other formats nicely.

(much more advanced: if it were possible to choose a special output format for different types (e.g if an object is a vega-lite object, then somehow allow the right evaluation so that outorg could export to an svg using for example emacs-vega-view).

lpy-forward-sexp-function missing during function evaluation

Example snippet:

def a():
    return 1 + 1

a()

To reproduce, put cursor before a() and evaluate. When I put the function lpy-forward-sexp-function
back from an old version. Everything seems to be OK again. Why remove lpy-forward-sexp-function in commit

dfa6437

Soap dependency

Where can the soap package be found? I couldn't find a package that provides soap and defines soap-command.

Always keeping parentheses balanced

How would I configure such that parentheses are always balanced and behave like in lispy?

Does lispy employ a separate package to do this? I don’t want to enable lispy just for this feature.

lpy-parens doesn't follow python style

If you try to write:

a = math.sin(5)

It produces the following instead:

a = math.sin (5) # Note the space before (

This space is not a convention in Python. See https://www.python.org/dev/peps/pep-0008/.

lispy-describe-inline doesn't work when parentheses are present

For example, the following case does work:

But when parentheses are present, it doesn't work any longer:

or

void-function worf--bounds-subtree

Steps to reproduce

Open a python buffer and press "l" on any line. For example:

import sys

Result

Debugger entered--Lisp error: (void-function worf--bounds-subtree)
  (worf--bounds-subtree)
  (cdr (worf--bounds-subtree))
  (re-search-forward regex (cdr (worf--bounds-subtree)) t)
  (and (re-search-forward regex (cdr (worf--bounds-subtree)) t) (if (lispy--in-comment-p) t (setq success t) nil))
  (while (and (re-search-forward regex (cdr (worf--bounds-subtree)) t) (if (lispy--in-comment-p) t (setq success t) nil)))
  (let* ((cur-offset (if (bolp) 0 (1+ (current-column)))) (new-offset (+ cur-offset 4)) (regex (format "^%s[^ \n]" (make-string new-offset 32))) (pt (point)) success) (while (and (re-search-forward regex (cdr (worf--bounds-subtree)) t) (if (lispy--in-comment-p) t (setq success t) nil))) (if success (backward-char) (goto-char pt)) (if (or (bolp) (looking-at " ")) nil (backward-char 1)))
  (cond ((lpy-outline-p) (if (zo-right 1) nil (if (re-search-forward "^\\sw" (cdr (worf--bounds-subtree)) t) (progn (backward-char))))) ((lpy-line-left-p) (let* ((cur-offset (if (bolp) 0 (1+ (current-column)))) (new-offset (+ cur-offset 4)) (regex (format "^%s[^ \n]" (make-string new-offset 32))) (pt (point)) success) (while (and (re-search-forward regex (cdr (worf--bounds-subtree)) t) (if (lispy--in-comment-p) t (setq success t) nil))) (if success (backward-char) (goto-char pt)) (if (or (bolp) (looking-at " ")) nil (backward-char 1)))) ((eolp) (let ((lvl (lpy-lvl))))) (t (self-insert-command 1)))
  lpy-right(1)
  call-interactively(lpy-right)
  (cond ((region-active-p) (call-interactively (quote lpy-right))) ((lispy--in-string-or-comment-p) (call-interactively (quote self-insert-command))) ((or (lpy-line-left-p) (and (lispy-bolp) (not (memq last-command (quote (self-insert-command soap-command newline)))) (or (looking-at lispy-outline-header) (looking-at lispy-outline)))) (call-interactively (quote lpy-right))) (t (setq this-command (quote self-insert-command)) (call-interactively (quote self-insert-command))))
  pspecial-lpy-right()
  call-interactively(pspecial-lpy-right nil nil)
  command-execute(pspecial-lpy-right)

Work-around

I was able to work-around this by the following steps:

1. M-: (require 'worf)
2. M-x worf-mode ;; echoes "Worf mode enabled"
3. M-x worf-mode ;; echoes "Worf mode disabled"