General feedback about improvements to Interactive Window Experience and Workflow

[taoluo]

General feedback about improvements to Interactive Window Experience and Workflow

[ftpronk]

It would also be very nice if that IPython console could be coupled to the debugger. It's a feature that makes for very nice debugging in pudb and in spyder-ide.

[GPhilo]

The original request came on Nov 3, 2016. I really hope it will be implemented soon, VS Code is fantastic under many aspects, but not having an iPython console is really a turn down for me. I've been trying out many IDEs, but none so far can replace Spyder because of how addicted I got to "run piece of code, check in the console a few values, run the next line, check some other values, repeat". Yes, technically that is already doable, but the ease of having a console with history, completion, magic functions, etc. just doesn't compare to what is now the Jupyter extension. Fingers crossed!

[chenwang]

Hydrogen on Atom offers a really good experience for data science and ipython. I do hope that something similar could come to VScode.

[jrwrigh]

I've already said my support in the referenced feature request, but I figured I'd throw my hat here as well. The IPython console offers a really great interface for interacting with scripts and data while also working really well with a REPL based workflows.

[rathsandeep]

Holding on to spyder for only this. If this is implemented, I will switch to vscode. Spyder lacks some of other features a good code editor should have. I end up switching between vscode and spyder when I am doing python+something else

[stereobutter]

@yozlet Thank you for sharing your workflow. I also was thinking about replacing .ipynb files with plain .py and the VS code comment cells, especially since version control for notebooks is virtually non existent.

About the PIW as history of evaluation order and why I personally don't like the way it currently works

One thing I argue is that most of the issues people take with notebooks (and why a history of evaluation order may be desirable to them) really comes down to where the individual user is coming from.
A lot of people come/came from Matlab or similar environments, where the workflow is that you write a script with a lot variables and imperative code that is run in a top to bottom manner. The potentially arbitrary order of execution in Notebooks does not really mesh well with this style of writing code (because rerunning pieces of code messes up the kernel state). You can often identify these users by their liberal usage of the Clear All Outputs/Run All Cells commands in jupyter.

I practically never run my notebooks top to bottom during experimentation, I rather view the notebook as a space where I state (and explain, by text and examples) some computational facts i.e. importing/declaring classes, functions and assignments that I then compute with. This means that when I want to change some computational fact I go to the place of its definition, change it and evaluate only the individual cell(s) (instead of making some change and rerunning the whole notebook). Also I tend to write python in a functional style (avoiding mutable state). Both these changes to how I (and also other people I have talked to) work with notebooks is not well reflected by the side by side .py file with cells + PIW as history. Basically it boils down to these two use cases:

1. People with an imperative/script-oriented workflow want the PIW to reflect the temporal dependencies of their code evaluation (what was evaluated in which order).

2. People with a more declarative/functional workflow want the PIW to reflect the logical grouping of their code (where did I define something) like a plain old jupyter notebook does (thus my requests for editing/saving/loading notebooks in the PIW). For knowing about the evaluation order the cell numbers (that increase with each evaluation) are usually enough.

Currently the PIW does only work for the first use case in my option, since it is basically a classical console. Come to think of it my initial requests in this thread are really a case of the XY problem - I asked for specific features/solutions but didn't really explain the problem (which I try to rectified with this post).

Feature request/proposal that might actually serve the use case instead of just tacking on features in hope of solving my issues:

Instead of making the PIW into something it is not indented to be (and breaking a use case a lot of people care about), I think it would be better to offer basically two ways of working with cells in .py
files in VS Code:

1. The PIW as it is for a more imperative/script oriented workflow
2. An option of sending the cells of a .py file to a jupyter kernel and showing the output in-place, right in the editor (for a somewhat related feature request see microsoft/vscode#3220, also there is hydrogen and a VS Code related feature request). This way one could basically use the .py file with comment cells as a 1:1 replacement for jupyter notebooks (with much better editor support/version control/etc. and no more moving code from experimental .ipynb to .py files for refactoring). The jupyter notebook (.ipynb or .html) then just becomes an export format that can be shared with end users (this is were a preview feature like the one on the marketplace comes in handy).

[GPhilo]

I think the current Interactive Window ("PIW" in the rest of the post) is an excellent result, but I see a few reasons why this thread still keeps getting requests for things that were already addressed.

• The window is designed notebook-first: the docs mention "cells" everywhere, while a person used to Spyder's console (which I believe is very close to the idea people have in mind when asking for an "IPython console") natively reasons line-by-line as if using a standard python interactive console.
  Using the PIW like this is possible, but perhaps not immediately clear to the user.

• At least for my use-case, the PIW takes way too much space. The fact that by default it opens in a split-window is great when running cell-by-cell or when mixing cells and plots, but if I'm just looking for a better python interactive console with completion and other fancy features (or, really, if I'm not interested in inline plots or inspecting variables via the GUI), the space is just wasted.
  What I would love to see is a configuration option that reduces the PIW GUI to the basic console view, ideally showing as one of the consoles in the "Terminal" view, but it could also be a new tab called "Python interactive" (for all I tried, I couldn't find a way to drag the view down next to the Terminal view). The general behaviour would be the same (perhaps also allowing swapping the shift-enter/enter commands to be closer to how the python interactive window works?), but the wasted screen area would be much less.

Together with the above, I would maybe add a subsection in the docs that shows how the PIW can be used as a plain "python console with advanced features" and mention this possibility quite early in the docs (this to address the fact that as-of-now users may think the PIW is notebook-only).

[GPhilo]

I agree. The feature request was for a "QTConsole-like experience" and we got notebooks 2.0.

[shea-parkes]

I think a lot of folks would be fine with "do it like Spyder and RStudio do it" (I sure would be). I'm actually very curious why that wasn't the initial form of implementation. They (Spyder and RStudio) represent a familiar and successful interface to a very wide audience.

[rchiodo]

Adding @neo-anderson's comment from another issue. Seems relevant to this discussion
https://github.com/Microsoft/vscode-python/issues/3919#issuecomment-454538174

There are many reasons why Jupyter+Hydrogen+pweave+Atom setup works well for me as a data scientist. It makes it possible to recreate the RMarkdown workflow for python along with the advantages of the powerful Atom editor.

• The source file is a plain markdown file with code blocks denoted using ``` string, which is consistent with the standard markdown syntax. This means that I can preview the markdown file in any markdown preview application without making any changes.
• You can run entire code blocks or select the statements to run and execute the selection only. This is a huge time saver when compared to the Jupyter notebook way of creating blocks even for running a single statement. For example, if I quickly want to see the shape of a dataframe, I can just type df.shape inside the code block, run it (Cmd+Enter/Shfit+Enter) and delete it after seeing the result.
• Inline output display. The output from Jupyter is displayed as a frame/bubble under the statement that was run. Clicking this frame would copy the result into the clipboard. Handy.
• If I feed the source markdown file into pweave, I can export it into a pdf/html/md file with the output of the code blocks inserted. This feature is a recreation of the knitr+Sweave feature in R Studio. Extremely useful for creating reports. This is one of the main reasons I switched to Atom. I use pweave to create a markdown file with code blocks replaced with the output from Jupyter and then I use pandoc to convert it into a pdf using themes/templates. It offers a lot of customization options for each code block too. For example, hide/show the code block in the exported report, hide/show the output results in the report, skip evaluation of code blocks, etc. Examples: echo=False, eval=True, results="hidden"
• Ability to maximize the code and markdown preview tabs and quickly switch between them using shortcut keys. "Maximize editor group" feature in VS Code will not maximize the tab to the full width of the window for some reason.

[smortaz]

@ftpronk - ironically jupyter/ipython does not have any debugging support. one of our devs is working w their team to add debugging to jupyter - stay tuned. hopefully that'll land and enable better vsc/jupyter integration. if interested in following the progress, see: jupyter/debugger#9

[rchiodo]

It would be nice to summarize what people are asking for. I think these are?

1. iPython console that is interactable.
2. iPython console that doesn't go away or that the lifetime can be controlled
3. iPython console that starts up fast

Anything else?

[jflam]

To me, this is what I’d like to see:

1. If your cursor is positioned on a line and there is no selection, send just that line to the interactive window.
2. If there is an active selection, send the selection to the interactive window.
3. Bonus points: if your cursor is positioned on a line and there is no selection, and that line sits within a block of code, send that block to the interactive window. E.g., your cursor is in the body of a for statement. This saves creating a selection for a common case, and where it may not make sense to send that one line of code without context. But having a second command “Send block” might be a good way to force the user to specify their intent.

Would love to hear feedback on this. Thx!

[mcdevitts]

@u2berggeist I do mean IPython... as in it would be great to have an IPython console running inside vscode like @GPhilo suggested. I wasn't referring to Jupyter's QtConsole.

What I meant was that plenty of tools these days are ignoring IPython integration and opting instead for Jupyter Notebook. Jupyter Notebooks are great, but they require a completely different work flow.

[jflam]

@GPhilo and @mcdevitts would you be interested in reviewing a proposal I have for this feature? If so, please hit contact me at jflam at {the company that makes VS Code}. Thx!

[bitagoras]

I am a little bit unhappy with the current solution. In my optinion the way on how we interactively work with Python should go in a slightly other direction. I programmed many years ago a hybrid of a simple editor and a console. Up to now, I could not find any comparable in the internet, so I recently wrote it again as an extension for Notepad++:
https://github.com/bitagoras/PyPadPlusPlus
You can have a look at the animation to see the idea behind this concept. What is special in PyPadPlusPlus compared to other interactive Python environments:

• The editor acts as the console input and you can execute single lines of code
• There is only one pure output console
• This means, there is no mixture of code and output and there is no redundant copy of the code which wasts some of the available area of your window and gives no additional information.
• The code blocks are optional. You can always execute single lines or any selected part of the code. With Ctrl+Enter you can step through your code and run it line-by-line. When you start at a #%% comment, you go through your code block-by-block. It depends on where you start. This is totaly flexible and intuitive.
• you can execute single lines or selections with the mouse middle button. With only one hand you can choose and run any part of the code within one second.
• Auto completion and code inspection is based on runtime information of the object.
• A matplotlib event handler is included, so you can plot many interactive windows from the same script.
• There is no Jupyter kernel required, so the startup is very fast.

I wrote a first simple version of this editor in 2004 and since then I used it continuously in science and industry until now. I controlled complex experimental setups in a physics lab with it. There I used Python code-lines like clickable buttons to control the experimental setup: a motor, the superconducting magnet coils, a cryostat and many other devices. The control was easy to use like a GUI and flexible like a text file. The program could grow while I was using it. This control produced great scientiffic results. The same workflow was also very useful for data analyses, numerical calculations and algorithm development.

I also use sometimes Jupyter, when I want to demonstrate mathematical concepts to others. But most of the time I need pure Python code, where my editor concept gives me the best flexibility. I realy would like to make this available in VS Code. In the Hydrogen plugin of Atom is something that goes in a similar direction. Could this maybe also fit to the concept of vscode-python?

[ronglums]

Great discussion here! Reopening the issue and we will look into this.

[rchiodo]

@shea-parkes can you enumerate what's missing from 'do it like Spyder and RStudio do it'?

I think this is the current list:

• Entry into the terminal without having to put new code into my python file
• Selection based execution from a python file
• Variable explorer

@dhirschfeld the debugger iPython repl sounds like a great addition too.

Is there anything else?

[brettcannon]

Just to help incentivize people to leave 👍 vote on the top comment, if you look at the issues with the most 👍 votes you will see this is number 6, but number for has nearly 50 votes. So while I appreciate the comments asking for this, it's those votes that really matter when we prioritize our work (and what people say in the survey that pops up for people who are randomly selected to fill one out).

[jflam]

Yes - I had some students at ICL create a prototype of this concept, and it's available on the VS Code marketplace today:

https://marketplace.visualstudio.com/items?itemName=neuron.neuron-IPE

We're definitely going to build something like this.

[brettcannon]

We have started our first cut of this experience in microsoft/vscode-python#3034.

[mcdevitts]

@bitagoras Your points miss a major use-case many of us are arguing for: introspection and manipulation of objects after the code has been run. In the case of ndarrays, why should I re-run my entire code or a specific "cell" when all I want is to examine the shape of the array, and play with various permutations of the data to make it work.

Finally, this ability is even built into the python interpreter with the -i command. All we're asking for is an ipython shell be opened in the same way. Heck, I'd be okay if this was selectable option. I completely disagree with your comment that a shell has no place in a modern editor or IDE. Both Python Tools in Visual Studio and PyCharm have integrated shells for debugging / introspection / manipulation after the script has been run. I would even point out that interactive consoles are available in JupyterLab and for very similar reasons like we've been mentioning:

Link a code console to a notebook kernel to explore code interactively without cluttering up the notebook with temporary scratch work.

[mcdevitts]

@rchiodo Spyder, I think, has the best implementation of an integrated IPython REPL. You can choose to run code or your file in a new REPL which then stays open for tinkering, or you can run your file in an already open and existing IPython REPL. Both use cases have their advantages and disadvantages, but not being forced into one or the other is a huge plus.

A non-python example of an ever-present REPL is Matlab, and I think a lot of the scientific code community either started in Matlab or plays in Matlab enough to see the utility of an interactive interpreter.

[dhirschfeld]

All I would like is an ipython interactive window - it's much more functional than a plain python repl. This can obviously be done by opening a terminal and typing ipython but it then has no association with the editor.

For the ipython interactive window I would like to be able to send selected lines from the editor to be executed in the current ipython window as well as to be able to execute a whole file in either the current or a new ipython window.

Spyder implements this and can be taken as the canonical example of what I think most people want (at least for a first-cut)

What I would also like, and what PyCharm implements, is to be able to open an ipython window at any point inside the debugger and have the local/global namespace populated with variables from the current stack-frame. For me the ipython repl in the debugger is PyCharm's killer feature.

[matkozak]

@shea-parkes can you enumerate what's missing from 'do it like Spyder and RStudio do it'?

I think this is the current list:

* Entry into the terminal without having to put new code into my python file

* Selection based execution from a python file

* Variable explorer

I just want to support this statement. I've been watching this issue and this is basically what I would like to get out of it; I think the Matlab/RStudio-like behaviour is what most researchers who start using Python for data analysis really miss.

Variable explorer would be nice but it's really an extra. The most important thing is to be able to interact with your variables without necessarily editing your scripts every time. Right now, the Python Interactive window is one environment, opening a Python REPL creates another (opening jupyter console --existing from the terminal is a good workaround for the moment), the selection-based execution creates yet another one and none of them talk to each other.

[d0mf1g]

@rchiodo

Well, an interactive IPython shell/console by itself, without needing the entire Jupyter module, would be a good start. The current extension gives us almost the entire Jupyter package, but judging from the responses here, something like an an IPython + QT Console experience without the notebooks is what's really desired (just the part circled in red):

This doesn't necessarily mean take just the QT console part from Jupyter and bring it to VS Code, but the user experience has to be similar.

I guess the option to choose the extent of the console's functionality (e.g., IPython by itself, or IPython + QT console, or the two + Notebooks) would make everyone happy.

[syagev]

I think the extension is still not addressing one of the most critical original issues - specifically, the ability to type code directly into REPL. Since we can't type in the "Results" tab of the extension we are "forced" to edit code in some "dummy" notebook file whenever we want to try out a few pieces of code. This isn't the most convenient way of working..

Think about this simple use case: I'm working on an existing complex plain python module, and while I'm doing that I need to quickly try out some code without launching the entire project. I can either (1) type it in a terminal, but without code completion, or (2) switch tabs to a "dummy" notebook and type it there - this means that on a decent monitor I can't have both the code file I'm working on and a decent REPL open in the same time.

It might sound picky, but I really think it's a crucial part of doing productive python data science development.

I've posted a related SO question (since I wasn't sure this feature existes and maybe I just missed it): https://stackoverflow.com/questions/53327894/interactive-python-console-with-autocomplete-in-vs-code/53328072#53328072

[mcdevitts]

Being able to automatically open an ipython shell would be fantastic.

To tag onto @GPhilo's comment, it would be even better if we could just execute the current file in a new ipython shell that stays open AFTER execution for further interaction. In other words, this would be dependent upon the ipython shell only, not Juptyer. Being able to plot from that console (these plots don't have to be inline in the terminal) would be key.

[stinos]

^ agreed; Jupyter is all nice, but not everything has to be Jupyter when a plain shell works as well, and often faster. For my workflow which isn't uncommon I think, the behaviour e.g. Spyder has would be best. And ideally improve that (e.g. Spyder has still annoying quirks like Ctrl-C not clearing REPL input, F9 only working in editor not for text from REPL output, ...). So yes that's: open an IPython terminal, run code and/or entire files in it and leave terminal open, maybe even with variables shown in watch window. And preferrably exactly the same behaviour when running under the debugger. Thinking of it, even though I dislike Matlab both as a language and IDE, it does have the REPL part nailed almost perfectly.

[bitagoras]

I would add:

4. Execution of code from the editor with intelligent selection of line(s) and advancing cursor.
5. "Silent" execution without a copy of the executed code in the console.
6. Auto completion in the editor based on run-time information.

[bat67]

I think what we need is like the same way running sections or files in SpyderIDE or MATLAB.

In Spyder we can run files or run sections(seperated by #%%) in the console by pressing shift-enter in editor, and after the file or sections finish running, all the variables remain, so it’s possible for doing some subsequent processing in the console, which is interactable(maybe watching variables, or doing something that is not maturely considered so it shouldn’t be written in this file). What running in the console is just IPython, not jupyter-notebook(in VSCode currently, if doing the same thing it will start Jupyter Server which is slow).

And in MATLAB, it is the same as in Spyder, we can run sections seperated by %%, after running this section, we can do anything we want to these variables in the MATLAB command line window.

But in Spyder, running sections is just like copying the section, and pasting in spyder ipython console to run. This does not happen in MATLAB because it will run section silently and only print the output the program should print. This is somehow like what @bitagoras said in number 5, “Silent execution without a copy of the executed code in the console”.

Maybe in VSCode, we can open a terminal and type ipython to open ipython in bash. And print run xx.py to run this file. After running, variables remain. But this method cannot run sections, only if you copy and paste this sections in ipython console and run manually. This may be the same as Spyder in  some ways, but a little bit tricky.

[ronglums]

@johnwoodill Yes I'm aware of that issue. We'll get to it.

[jflam]

Thanks for the feedback. The current feature is a point-in-time snapshot of where we are right now, and was chosen for engineering expediency. We are definitely thinking about the right way to provide folks with both options as well. Stay tuned!

[stinos]

Heh just today I had to take up some Python work and also spent quite some time in the debug console. And yes it does a pretty good job already. Way better than plain ipdb. For the particular data I was working with the only things I missed:

• Debug->Evaluate doesn't handle indentation and multiple lines well, this is probably the main issue for now
• when showing numpy arrays, it prints the \n chracter instead of actually displaying a new line
• pandas dataframes also don't look as good as they could
• I've never really used Jupyter, but for looking at histograms etc a visualisation really is a must so inline matplotlib would be completely awesome

[rchiodo]

So if we had say a button to open the interactive window in Jupyter, that would be useful? (Which could also be backed by a command and it would just open a jupyter notebook in the browser)

[KaiLicht]

A must have for data science work with VS Code! That would be a great feature.

[smortaz]

don's existing jupyter plug-in for vscode kinda sorta does the same, but it's not complete. we'll get there.

…

On Tue, Feb 13, 2018 at 11:49 AM, Chen Wang ***@***.***> wrote: Hydrogen <https://nteract.io/atom> on Atom offers a really good experience for data science and ipython. I do hope that something similar could come to VScode. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/Microsoft/vscode-python/issues/727#issuecomment-365382824>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtUVAdxlv4zVz0Ltk9DvxmgTPEM2pn_ks5tUec4gaJpZM4R9ydW> .

[brettcannon]

@JackCaster yes

[ronglums]

Please check out the latest Python extension that enables an interactive experience to work with data: https://blogs.msdn.microsoft.com/pythonengineering/2018/11/08/data-science-with-python-in-visual-studio-code/

[jamescasbon]

You can get a repl into the current kernel by running "jupyter console --existing"

This gives you access to variables defined from the notebook cells.

One possibility is to use the %connect_info stuff to automate creating a repl connected to the current notebook.

Plotting doesn't work with this setup, but it is possible to inspect and otherwise poke objects.

[bitagoras]

I would like you to reconsider whether a shell is really the best way of interaction with Python. The main drawback for shells is in my opinion the very inconvenient way of repeating your previoulsly used commands. Imagine your commands have several lines of code. When you're lucky it consists of one indented block which you can scroll back, modify something and repeat it. But if several single lines belong together, e.g if you plot some data, it becomes very annoying. You have to scroll back to every single line, perhaps modify it, and run it in the right order. In my workflow I like to write some code in the editor, run in partially, then I want to plot some intermediate data or write some output to test my program. Then I either discard the plots and tests or I keep it for later usage or maybe make it part of the program or make a separate program out of it. But in the shell they are lost and I have to retype everything into the editor or fishing all the single command lines from the shell. How can this be a desired solution? In the solution I demonstrated with PyPadPlusPlus you have all the advantanges of a shell but none of the described disadvantages. It does not cover all the advantages of Jupyter for presenting results with formated text, rendered equations etc. But it is much more versatile compared to a shell. And Hydrogen, which has a similar mode than PyPadPlusPlus with a separate output area, demonstrates how you can have many properties of the two concepts combined into one plugin.

[ronglums]

We can start with an iPython console area in the Interactive window that is editable and results will be rendered in the Interactive window.

[ftpronk]

The last thing I want to do when debugging python code is to clutter my files with run sections or any other modifier. It's not only cumbersome, but I find that it's simply, visually, standing in the way. To find bugs, you need to be able to read your code, and anything that impedes or impacts code readability is not an acceptable solution.

[nraw]

It's great to see that there's a discussion on this as I'm baffled to see that this kind of support is not available yet. I guess coming from languages such as R and Matlab I'm more used to the interactive way of coding when it comes to data science and while I guess Spyder comes very close to this, it just feels like a very basic tool of which the main benefits could be easily replicated in VSCode.

But honestly, I think I would already be fine by just having the option of "Send line/selection to Debug console" and "Send block to Debug console". Am I missing it and this already exists?

So far everything I've found was either sending it to the terminal and/or the Interactive python window, which just feels like an inferior Jupyter notebook which if I wanted I would just be using Jupyter in the first place.

[stinos]

having to enter cells, how would you want to indicate a cell to run?

As long as I don't have to add # %% all over the place it's all good :P More seriously: what jflam says is pretty much the canonical way. Select text, execute command to send text to terminal. Or without selection, send current line. Not too keen on the idea of automatically making that a block so if that is available it should be a seperate command. It would be pretty handy, but the decision on which scope to run in case of nested scopes might be a point of discussion of course..

edit just to check on what is getting implemented: does this mean we set "terminal.integrated.shell.windows": "/path/to/ipython" and then can send files to it (translated to run in ipython), and text?

[GPhilo]

That would be the best solution indeed

[yozlet]

Further to @mk6ve's comment, the existing VSCode Debug Console does a whole lot of what I want already, especially:

1. Start with existing execution state (i.e. whatever the state of the Python program is when the debugger pauses)
2. Easily type single lines and execute them
3. Recall/edit commands from history
4. (this is the killer for me) Inspection/exploration of expression results

This last one is really useful for my own needs, because I'm usually dealing with nested data structures (objects containing arrays containing dicts, etc.). The tree-like explorer widget that the Debug Console provides for expression results makes it easy to explore, compared to the static text that the Python Interactive Window currently provides. (I know you're working on it!)

To be clear: I'm not talking about exploring all the current variables, just the result of a given expression. Sure, exploring all the variables would be useful too, but I can already (kind of) do that by entering their names.

(Yesterday I had a poke around the Debug Console code, and it looks like there's a lot of interesting stuff there that maybe more extensions could/should be using for REPLs rather than just debugging? I realise that some of the interaction model is quite different from a notebook, and I don't want to conflate the two or devalue the awesome work on the Python Interactive view.)

Caveats:

1. Unlike many Jupyter users, I don't use visualisations much at all. To me, being able to explore data structures in results interactively is much more useful. But I don't know how representative I am of the target user base. (It's also possible that if/when Jupyter widget support lands, there'll be a multitude of widgets that allow this kind of exploration.)
2. @rchiodo, @ronglums and everyone else working on the Data Science project: you're awesome, thank you!

[rchiodo]

Some good news, items 1 and 3 on your list on our backlog. Issues microsoft/vscode-python#4170 and microsoft/vscode-python#4256 respectively. Please feel free to up vote them.

As far as your other requests:

2 - I don't think we're going to support this. The idea for our window is that it's not a notebook, but a history. Although that begs the question why we support exporting to notebooks then. And especially if we implement microsoft/vscode-python#4256. What's your major use case for wanting to edit a cell? Would it be enough if the up arrow allowed you to scroll through the cells you've ever entered before? (Meaning you just want to change a cell and rerun it). Or is it that you want to generate a notebook and you need to fix up something?

4 - microsoft/vscode-python#3525 is almost this, but without the preview part. Not sure we'd want two views of a notebook (the interactive window and a preview pane). Please feel free to enter a new issue for the preview part.

[yozlet]

@SaschaSchlemmer I've been using the Python Interactive Window to write and experiment with data-processing code too. If you use the PIW on its own, like a single Jupyter notebook, it's no good for that. However, if you use it in a two-pane setup - python file on the left, PIW on the right - it might be much more like what you're looking for.

Here's how I do my experiments (or, more likely, one-off data prep jobs) in it:

1. Create a new Python file (I'll call this one my-exp.py) - it's a normal Python file, not a .ipynb file. Put the #%% cell-splitting comment at the top, and hit Save, then click the Run Cell decorator that pops up. Now I have the split pane view described above.
2. I use the Python file like a notebook, with cells separated by #%% splitters. The cell decorators let me easily (re-)run an individual cell or the whole file. But it's also a single runnable file if I want. Plus, because it's a normal VSCode Python file, I get all the syntax highlighting, completion, linting etc. etc. (Much of that is coming to the PIW, but it's not there yet.)
3. I use some of the cells in the Python file for defining functions, others for running the kind of glue code that I'll also want to execute when running the entire file in one go.
4. I mainly use the PIW for running one-off lines of glue code, calling the functions I've defined in the Python file (after I've run the cells with the definitions in) and doing inspection of results. Actually, inspection takes up most of it, so I'm really looking forward to better data/state inspectors landing.

I suspect I'm missing better ways of doing all this, but it seems to work fairly well so far. Thinking about it now, I might go with three panels next time: main Python code, a palette of Python glue code snippets, and the PIW.

Part of why I like VSCode's "Interactive Window" is that it's a definitive history of interpreter state. By cleanly separating input and output, and not allowing you to change history (by editing/rerunning cells in their previous positions), it stays away from Jupyter's most confusing and error-making problem: hidden state. (Note: I am deliberately avoiding the PIW's weird undo/redo buttons. I played with them for a while and I can't tell what they're good for, other than making it easier to clear a load of the log without moving the mouse around.)

This is best explained by Joel Grus in his insightful and hilarious talk, "I Don't Like Notebooks". (Here's the video, it's 40min of talk and 15min of inaudible questions. Alternatively, here's the Google Slides deck which is much quicker to scan through, especially if you raise the notes panel under the slide view.)

In short: Jupyter makes it easy to get started, easy to get instant results, easy to misunderstand the current interpreter state, easy to screw up as a result, and surprisingly difficult to produce good code out the other end. (I was going to elaborate on these points until I looked at Joel's slides again. He explains them far better than I ever could, since he has enough experience to really understand both the data scientist and software engineer perspectives.)

The main thing I'd like for a VSCode Python REPL is to make it useful for both exploratory play and high quality software engineering, somehow making a contiguous bridge between the two. (Good luck, folks.)

[rchiodo]

Right now to use the IPython console you have to work with the Jupyter notebooks.

Not sure why you're saying this? You don't have to use a jupyter notebook. You can use our interactive window:
https://code.visualstudio.com/docs/python/jupyter-support-py

Are you saying this window is too jupytery (for lack of a better term)?

It's very unlikely we'll create yet another terminal that kinda talks to IPython. It wouldn't support remote. It wouldn't support remapping output. Jupyter does all of this for us. Why would we start reinventing Jupyter?

[yozlet]

@GPhilo I mentioned this above, but in case you missed it: the existing Debug Console does much of what you want here, including its position in the "Panels" section with the Terminal. It's much more like a basic REPL console, but has a bunch of useful tricks (including autocompletion, if I remember rightly).

The main problem with the Debug Console is that it's primarily meant for process debugging. As such, it's slightly inconvenient to start: you need to create a Python script which loads up the modules & state you want, then add a breakpoint to the end (manually in VSCode or with the new Python breakpoint() command). Finally, run the script in the debugger. The Debug Console will now be available for use.

[mcdevitts]

100% agree on this. I would also be able to switch over if ipython was fully supported. It's absolutely required for a data science workflow.

[jrwrigh]

@chenwang I didn't know that existed. Having Jupyter outputs appear inline rather than have it appear in a seperate window is fantastic. If we could have something like that on VSCode it'd be great.

[mcdevitts]

Completely agree with @GPhilo. Often people mistake wanting ipython with wanting jupyter.

[jrwrigh]

@mcdevitts Are you sure that we don't mean IPython? We're referring to the interactive console that is still (to my knowledge) called IPython. We're not referring to the Jupyter QtConsole which is a fancier GUI frontend over the IPython console.

[ftpronk]

@smortaz Important priorities for sure. Maybe it would make sense, while working on improving debugging in VS Code, to already take on board the idea of (optionally) using IPython as an integral part of the debugging process, as opposed to it just being an idea which is strapped-on later in the development process? (Just stating the obvious in case it might have been overlooked. I'm not questioning the VS Code planning in any way..)

@u2berggeist I guess they are referring to the ipython functionality common to the interactive console and the QtConsole, as opposed to the jupyter cell evaluation which is currently available in VSCode.

[ftpronk]

@smortaz - Very interesting! I was always a bit surprised that it wasn't used more as debugging tool, but that explains it I guess. So such a development will also end up making jupyter/ipython better tools too. Exciting stuff! Thank you for the link, I'll definitely stay tuned. Keep up the great work!

[janat08]

will this fix where autocomplete works line-by-line basis not recognizing blocks of code?

[GPhilo]

@jflam Are there any updates on this issue?

[JackCaster]

We have started our first cut of this experience in microsoft/vscode-python#3034.

Is microsoft/vscode-python#3034 somehow related to Neuron? I read a recent post on the Microsoft blog that says that

Microsoft has decided to integrate the work that we did in neuron into the Python extension for Visual Studio Code

[yzhang-gh]

I have some similar experiences as @bitagoras shared. In short, the good things are

1. Ability to execute the selected code (or the current line if no seletion) (no matter there is a #%% or not)
2. The "pure output console". IMO, the current output cells with the corresponding codes are probably useful, but most of the time they are not but distractive...
   I have opened a related issue microsoft/vscode-python#3266 (fold the code in the output cell by default)

These two features are already there with the former Jupyter extension (It has a command executeSelectedCode (don't remember the exact name) for feature 1)

But I have to use my own fork since that is unmaintained now.

---

As for

3. Auto completion and code inspection is based on runtime information of the object.
4. A matplotlib event handler is included, so you can plot many interactive windows from the same script.
5. There is no Jupyter kernel required, so the startup is very fast.

they look attractive

[bitagoras]

A folded cell by defaul would be a good improvement. I would go even further and would also hide the first line of code when folded. I am pretty sure that nobody would unfold it ever. Since you cannot modify this code when you want to restart the cell, you have to search the corresponding line in the editor anway.

The currenty concept tries to mimic Jupyter notebook, but it isn't it Jupyter. I don't expect that VS Code will become a replacement for Jupyter. This would require to include all kind of web techniques to make the interactive plots etc. available. PyCharm tried to do this for some time but it was never good enough, so that I used it. And currently the startup time of VS Code to print a simple "hello world" with Python takes 30 seconds at my computer. So, I would apreciate if I this plugin would focus on the properties where Jupyter is not made for, e.g. single line / selection execution, simplicity, efficiency (e.g. best use of the available window space) and fast startup.

[jrwrigh]

@jflam So if the issue has not been fully resolved (as I think we've all agreed on), should it not be re-opened?

[jflam]

@ronglums - can you please re-open and drive this? Thx!

[GPhilo]

@bitagoras I'm not sure I understand your comment. No one here is saying "throw out the editor", the desired coding experience if very different from what you describe. You have your code, possibly with defined sections similarly to what the current data science extension allows, but these sections run in an interactive iPython console. That way, if you need to quickly run some one-liner, or if you want to inspect the value of some variable/expression, you just type it in the console and get the output.

Of course, if you have a major block of code, you'll probably want to have that in the editor (possibly in a different file). Again, the best case scenario would be "select the code, press Shift+Enter and the code runs in the interactive console". How is this inconvenient?
To get a better idea of what I'm talking about here, look up how Spyder's console works. That's essentially what I personally would love to see implemented in vscode.

[bitagoras]

Don't get me wrong. I am just talking about the interactive addon to the editor.

Sure, "select the code, press Shift+Enter and the code runs in the interactive console" is basically it. But there are several things I don't like: The selection is annoying. You have to select any single line before you can execute it. If you did not select the whole line properly, you get an error. You also mess up your output with a copy of your commands and some ">>>" and "...". Why do you need the same commands twice on your screen? And why do you want to enter commands "in" the shell? Your editor should be where you type your commands (the "human" side) and the output is only the output (the "machine" side). I don't see the advantage of this mixture. You can open a new scratch editor if you do not like to enter some commands in your main program. Having a "pure input" area and a "pure output" area seems to me more cleaned up and you never have to switch your cursor. And your output will more look like the output of your finalized Python script when it runs as a whole.

The idea behind the shell was to have an interview-like dialog between the user and the interpreter. This perfectly fits into a system shell like a linux shell. But in a modern editor and IDE such a shell has no right being here any more im my opinion. There I expect something more optimized for this environment.

[bitagoras]

This use-case is included in what I described. After you have runned the whole code (Ctrl+A and Shift+Enter), you can inspect the variables and do some post processing, testing or plotting. Your program never ends until you shut down the kernel.

I know from PyCharm your use-case with the shell. PyCharm is great, you can debug and stop at a break point and then use the shell to investigate the current state of your variables. You also can open a scratch editor and write some analysis or plotting functions and run selected parts in the shell to inspect the current local scope of your function. But this selecting an running code in the shell could be more smooth and without displaying a copy of your code in the shell, because it is a mess. That's basically what I say. I don't want to abolish the shell, but improve it.

I think it is a question of complexity of your data whether a normal shell satifies your needs. In my use-case it is often not sufficient to check some variable contents or find some bugs in the code. I have to find numerical inaccuracies in very complex data. This can be quite challenging and requires mathematical transforms and ploting of maps or interactive 3D animations to investigate the data (I currently develop code for a satellite based navigation system, where it's all about numerical accuracy). My interactive auxillary calculations are too complex to enter it in the shell. My need for an appropriate interactive environment was so great that I developed this year this new Notepad++ addon. It helps me now a lot and it is much better than my old solution and all other Python environment I tried (and I tried everything new the last 14 years). I think that many people who work with complex data or maybe machine learning algorithms have the same need. I just still miss a debugger, therefore I really hope that this could also be realized in VS Code, which is a great and modern editor.

[stinos]

Am I missing it and this already exists?

Yes: while in debug mode, select text, right-click, select Debug: Evaluate

But for the rest the Debug Console isn't the most exciting environment, especially not if you're used to a full R or Matlab or IPython REPL.

[rchiodo]

People seem to be asking for an iPython REPL. Do you have an example of an iPython REPL? Are you talking spyder? What are the parts missing from the VS Code Python Interactive window?

[rchiodo]

Just as far as the Spyder console is concerned (skipping the variable explorer for now), the differences would be:

• Starts up as soon as you open
• Lets you enter text directly into it or from your python code
• Simpler UI

Otherwise it's the same?

Which of those things is most important? Meaning what would you want most out of VS code.

[stinos]

What are the parts missing from the VS Code Python Interactive window?

Basically that it's not interactive in the sense that you cannot type commands and hit enter (I think?). And that it requires cells, making it pretty awkward to use existing files which don't have cells in it. See ftpronk previous comment.
Then again I haven't used it extensively so maybe I'm missing something obvious.

[rchiodo]

I think we're going to be adding the commands pretty soon.

Would a single line be enough? (That's what Spyder does as far as I can tell).

@stinos, For your other response - having to enter cells, how would you want to indicate a cell to run? Spyder has the cell requirement too for shift+enter, but lets you also send a 'cell' with selection. Is this good enough?

[rchiodo]

edit just to check on what is getting implemented: does this mean we set "terminal.integrated.shell.windows": "/path/to/ipython" and then can send files to it (translated to run in ipython), and text?

Meaning you want to replace the normal Python REPL with an iPython one? I guess I'm interested in why/if you want it in the terminal instead of the extra tab we have for the Python Interactive window.

[mcdevitts]

Would a single line be enough? (That's what Spyder does as far as I can tell).

@rchiodo Spyder definitely supports being able to run the entire file in the interactive interpreter. It also does this in a way (I believe using the %run magic command or something similar) where the entire code contents of the file are not dumped into the interpreter.

[rchiodo]

@rchiodo Spyder definitely supports being able to run the entire file in the interactive interpreter. It also does this in a way (I believe using the %run magic command or something similar) where the entire code contents of the file are not dumped into the interpreter.

I was actually interested in what was directly needed in the interpreter. It looks like the %run command runs an entire file from the interpreter (although the file has to be saved). We would of course support all the magics available, so %run should behave the same way.

What I was more interested in was if it was necessary to be able to enter more than one line directly into the terminal? Should it behave like a terminal, or like a jupyter cell?

[stinos]

Meaning you want to replace the normal Python REPL with an iPython one?

If possible, by all means, IPython is just better. Basically what I do now already is have an IPython REPL open and manually copy/paste into it from the editor, then continue typing commands in that REPL for inspecting data/plotting/...

I guess I'm interested in why/if you want it in the terminal instead of the extra tab we have for the Python Interactive window.

Oh but in principle I don't really care if functionality goes in a terminal tab or Python Interactive tab. Though I'm a bit confused, all the Python Interactive window in my VSCode does is show pretty tables etc, but it's not a REPL, I cannot type in it. Are you adding that as well or am I missing something here?

[rchiodo]

Are you adding that as well or am I missing something here?

That's the goal. We plan on adding the ability to type directly into the window very soon. Not sure how the UI is going to look yet, but at very least you'll be able to enter a single line of text and on hitting the enter key it will execute that text and display whatever output it generated. Just like a REPL.

[GPhilo]

@jflam I actually dislike your point 3) and would at least make it disableable. The reason is, often if I'm debugging (especially loops) I'll just set the iteration variable to the first element of the loop and then run the loop body one line at a time checking the intermediate results. Doing this with your 3) enabled as default would require to select the line every time.

Also related to this use-case, please please please strip the white space in front of a line on the "run line" command without selection, the interpreter will keep complaining about wrong indentation otherwise.

[jflam]

@GPhilo thanks for the candid feedback! Especially like the point about stripping the whitespace.

I think for this reason, it's important for us to make "Send block to interactive window" a separate command. I think this should give you the experience you want without breaking your current workflow. Do you agree? Thx!

[dhirschfeld]

Note also that ipython itself take care of stripping not only whitespace but also prompts which makes for easy copy/pasting (important for an interactive repl)
e.g. In [1], >>>, ...

[shea-parkes]

Those are the biggies to me sir. Especially the first two, but the third one is very nice as well.

[JackCaster]

@dhirschfeld the debugger iPython repl sounds like a great addition too.

The iPython REPL for debugging would be a fantastic, and the benefits would not be restricted to the data science use only (as other suggested this could be a settings: do you want the classic REPL or the IPython terminal?), and not related to the Interactive windows running the Jupyter server.
But I do not understand why, in current implementation, the debug console does display PyDev console: using IPython 7.1.1? It does not look like an IPython terminal at all to me. For example, an IPython terminal would display the numpy array np.random.uniform(size=(3, 3)).round(1) as

array([[0.5, 0.8, 0.6],
       [0.2, 0.8, 0.1],
       [0.1, 0.2, 0.3]])

while the debug console now displays

array([[0.9, 0.1, 0.1],\n       [0.7, 0.2, 0.2],\n       [0.9, 0.1, 0.9]])

which is quite clunky if you want to inspect variables while debugging.

[rchiodo]

@DonJayamanne , @karthiknadig or @int19h, can you guys comment on the formatting for the current debugger console? It looks like there might a problem with linefeeds?

[stereobutter]

I tried the jupyter notebook integration of VS Code today and I really like what I saw, however there are a few things still missing for VS Code to replace plain jupyter notebooks in my personal day to day workflow.

My current workflow with VS Code and jupyter is this:

In my working directory I have:

1. One or more .py modules (or packages) where my classes and functions live, e.g. foo.py

2. One or more .ipynb notebooks that I have opened in Chrome side by side with VS Code (and that are connected to a local jupyter notebook server). On the top of such a notebook I usually have something like:

    %load_ext autoreload
    %autoreload 2
    from foo import some_function
   

I use the jupyter notebooks both as a of glorified scratch pad and to run experiments/share my work with data-sciency coworkers. For the scratch pad/experimentation use case that means that a function/class might start out in a jupyter notebook and is moved to .py module in VS Code where I do refactoring, testing etc. The function/class is then imported back into the originating notebook.

Missing features/issues with VS Code and its Python Interactive that keep me from ditching plain jupyter notebooks in my workflow are:

1. No autocompletion in Python Interactive; just having to explicitly type each and every single brace and bracket annoys me to no end.

2. No cell editing in Python Interactive; it looks like a jupyter notebook but behaves more like a plain old console in this regard

3. Open/Save of the Python Interactive Window as .ipynb file. Currently one can only export the current interactive window as a .ipynb file but can't reopen and edit it without importing and exporting again.

4. Missing preview feature for notebooks. It would be nice if there were a workflow for keeping a Python Interactive notebook in sync with a preview from the preview extension.

[stereobutter]

@rchiodo, to give you some context where my request(s) are coming from: For me (as a ML guy) my product is often comprised of two parts

1. An algorithmic prototype that solves some domain issue e.g. detecting cats in images
2. A custom library/toolkit used to build said prototype

Usually I start out just with a jupyter notebook, literally throwing some stuff together to see what sticks.As development processes and I identify some pieces of reusable code I move them from my experimentation notebook to a real editor/IDE for refactoring, testing and documentation. When that is done I import them back into my experimentation notebook. That way I end up with some (hopefully) reusable, documented and tested library code and a notebook that uses this library code to (hopefully) show how to solve the domain issue.

The notebook is often what I share with colleagues (that might take my library code and build a production ready application with it) and end-users (that are interested only in the domain issue, not the library code necessary to solve it).

In my opinion the current state of the notebook support leans more towards notebooks being throw-away scratchpads/consoles and not an integral part of the product that evolves side by side with the library source code. Also the amount of glue/script/plotting/example code and text in the notebook is often not insignificant, so that the tooling for the notebooks really has an impact on the overall developer experience (read happiness) and productivity.

I hope that gives some context as to why I want to be able to edit cells, load/save and preview notebooks among just general improvements like intellisense/auto-complete.

Addendum:

What would make me a very happy developer were if I could literally have a browser window with a plain jupyter notebook right inside VS Code. That would make VS Code a (way better) Jupyter Lab actually.

[rchiodo]

For that last item, why not just have a jupyter notebook open? What does having it inside VS Code buy you?

[dhirschfeld]

I assume the hassle of starting a notebook server? It's the killer feature of nteract for me.

[rchiodo]

@SaschaSchlemmer, there's already an item you can upvote for number 2 - microsoft/vscode-python#4575. There's some experimental features coming to VS Code that should make this possible for us to do in the future.

This discussion is great for giving us the reasoning why we'd want to have both and not just one or the other. Thanks a lot.

[absudabsu]

I like the workflow suggested by @yozlet. I wonder if some of those elements could be re-purposed to make another useful feature, namely...

I'd like to see VS-code have embedded/interactive plots in a separate grid window/cell, like Spyder has. For example, like the debugging variable inspector (I believe this exists in VS-code for javascript), I think it would be very useful to have an embedded "window" where matplotlib (or similar) will get displayed from calls such as plt.plot(...) or plt.show().

In my workflow, I read and write images all the time, usually using plt.savefig and it would be incredibly useful to be able to view and interact with figures directly in the VS-code application. If this was done, for example, in a way compatible with remote-code, I would never need another IDE.

[rchiodo]

@sirgogo is microsoft/vscode-python#3094 similar to what you're asking for? Another window to interact with plots?

[absudabsu]

@rchiodo Maybe I made it confusing by mentioning remote-code.

I'm asking for an embedded "plots" window, something like what is in Matlab, or in this image from the Rodeo IDE (bottom-right) https://pbs.twimg.com/media/DlBAV-gU0AAAP-S.jpg

Its not quite the same as whats in Spyder: https://raw.githubusercontent.com/spyder-ide/spyder/master/img_src/screenshot.png
But a Spyder-like implementation would be a good start, especially if the plots were interactive.

[ronglums]

@sirgogo, the PIW supports plots similarly to Spyder. Is there anything else you like about how Spyder works with plots?

This issue microsoft/vscode-python#4976 covers many good suggestions on this thread regarding plot support - especially about creating a standalone "plots" window like what's in Matlab/R Studio. Please feel free to vote.

[d0mf1g]

Thanks for the feedback. The current feature is a point-in-time snapshot of where we are right now, and was chosen for engineering expediency. We are definitely thinking about the right way to provide folks with both options as well. Stay tuned!

@rchiodo @jflam

Any decision on implementing a standalone IPython interactive console without the full Jupyter notebooks modules/experience?

[rchiodo]

@d0mf1g not sure what you mean by that? Something that loads just ipython? But also supports plots/dataframes/etc?

[rchiodo]

What would that look like? The UI inside of VS code would likely be the same?

Is the reason for this ask to simply get rid of the jupyter dependencies in your python environment?

[d0mf1g]

@rchiodo

Is the reason for this ask to simply get rid of the jupyter dependencies in your python environment?

Sort of. Right now to use the IPython console you have to work with the Jupyter notebooks. As many have noted earlier, notebooks have their own work flows that are different to than just using a terminal.

For starters, could we first get an option to just work with an integrated IPython console without loading the entire Jupyter package? This would be a first big step and wouldn't require much. Console/terminal would look no different than it does now.

https://ipython.readthedocs.io/en/stable/install/install.html#install

And then afterwards maybe you guys can make improvements to it by adding the QT console front end using the IPython kernel:

https://qtconsole.readthedocs.io/en/stable/index.html

[GPhilo]

@yozlet You're right, I was not aware of the Debug console's function, but as you say, it's extremely cumbersome to use (and, for some use cases, running in debug mode is just not an option, performance-wise)

[absudabsu]

I want to use ipython as if on the terminal (i.e. without cells, or notebook-style magic), but also have the ability to plot and view figures without saving them to a file. Right now I run ipython from the console panel, save figures to a file, and use the built-in file explorer to load them up... but its a pain, especially for large figures or where interactive plots are necessary. Moreover, I don't want to reformat my (usually headless) code to be able to call it from a notebook cell and then visualize. This is not an issue when running ipython in the terminal (via the console) on the desktop, but its certainly an issue when you are using the remote-ssh plugin to work on a remote system.

…

On Mon, Oct 21, 2019 at 2:49 PM GPhilo ***@***.***> wrote: @yozlet <https://github.com/yozlet> You're right, I was not aware of the Debug console's function, but as you say, it's extremely cumbersome to use (and, for some use cases, running in debug mode is just not an option, performance-wise) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/microsoft/vscode-python/issues/727?email_source=notifications&email_token=ABKVCINKZ6JWPHAGQEABMLTQPYPWTA5CNFSM4EPXE5LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB3462A#issuecomment-544722792>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKVCIIZNWKAAFJKMRZFCJ3QPYPWTANCNFSM4EPXE5LA> .

[ronglums]

@sirgogo Just want to mention that cell annotations aren't required for the Python file to work with the Interactive window in VS Code. You can use the "Run selection/line in python interactive window" command to run any code selection or line.

[johnwoodill]

@ronglums Yes, you are correct, but as you append more selection/lines to the python interactive window it begins to slow down. See issue microsoft/vscode-python#6604 . This issue makes it hard to work with the python interactive window as a REPL.