spyder-ide/spyder-terminal

Run system terminals inside Spyder. Works on Linux, macOS and Windows.

Spyder-Terminal

Project status

Build status

Copyright © 2017–2020 Spyder Project Contributors

Overview

This is a Spyder plugin for displaying an OS independent virtual terminal inside the main Spyder window. It currently supports both Unix-like and Windows operating systems.

Spyder-Terminal allows you to easily execute any bash command inside Spyder, even ncurses programs like nano or vi:

On Windows you can run console applications such as IPython or powershell:

Installation

To install this plugin, you can use either the conda or pip package managers, as follows:

Using conda:

conda create -n spyder-env -c conda-forge spyder-terminal
conda activate spyder-env
spyder

Using pip (only if you don't use conda!):

pip install spyder-terminal

Note: At the moment it is not possible to use this plugin with the Spyder installers for Windows and macOS. We're working to make that a reality in the future.

Dependencies

This project depends on

• Spyder
• Tornado
• Pexpect (Posix Systems)
• pywinpty (Windows Systems)
• Coloredlogs
• xterm.js
• Terminado

Changelog

Visit our CHANGELOG file to know more about our new features and improvements.

Server implementation

Besides a Qt terminal, spyder-terminal also provides a web-based terminal interface based on Tornado, which allows you to deploy and serve terminals from a Web/Javascript frontend. To deploy only the server, you can execute the following bash script:

# Shell option:

# On Unix systems this can be bash/tcsh/zsh or any Unix shell:
# bash: /usr/bin/env bash

# On Windows systems this might be cmd or powershell:
# cmd: %SystemRoot%\windows\system32\cmd.exe
# powershell: %SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe

python -m spyder_terminal.server --port <PORT> --shell <Path to the terminal backend to execute>

Development and contribution

To start contributing to this project, you need to install the yarn and npm package managers. If you use conda, you can run the following command to do that:

conda install -c conda-forge nodejs yarn

Then, please install the package's dependencies with:

conda create -n spyder-terminal-dev -c conda-forge --file requirements/{conda,conda_win}.txt

depending on your operating system.

Afterwards, you need to run

python setup.py build_static

to build the Javascript components for this plugin.

Finally, in order to run our test suite, please install its required dependencies with

conda install -c conda-forge --file requirements/tests.txt

and use pytest to run the server and client tests for the terminal like this

pytest .

Sponsors

Spyder is funded thanks to the generous support of

And the donations we have received from our users around the world through Open Collective:

More information

Main Website

Download Spyder (with Anaconda)

Spyder Github

Troubleshooting Guide and FAQ

Development Wiki

Gitter Chatroom

Google Group

@Spyder_IDE on Twitter

@SpyderIDE on Facebook

Support Spyder on OpenCollective

spyder-terminal's Issues

Solve security access problem

The tornado backend server should be deployed only on loopback interface.

Prevent running this plugin if Spyder is using PyQt4

This is an important thing to do and something I forgot to mention before. This is because PyQt4 uses a websockets version that is too old for modern standards.

The plugin should deactivate itself and show a warning message in that case.

TypeError: 'Icon' object is not callable

From @wlsdnen on July 26, 2017 12:59

Description

What steps will reproduce the problem?

What is the expected output? What do you see instead?

Please provide any additional information below

File "c:\users\jjw\appdata\local\programs\python\python35\lib\site-packages\spyder_terminal\terminalplugin.py", line 156, in __wait_server_to_start
QMessageBox.Ok)
TypeError: 'Icon' object is not callable

Version and main components

• Spyder Version: 3.2.0
• Python Version: 3.5.3
• Qt Versions: 5.9.1, PyQt5 5.9 on Windows

Dependencies

pyflakes >=0.6.0 :  1.5.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  None (NOK)
numpy >=1.7      :  1.13.1 (OK)
sphinx >=0.6.6   :  1.6.3 (OK)
rope >=0.9.4     :  0.10.5 (OK)
jedi >=0.9.0     :  0.10.2 (OK)
nbconvert >=4.0  :  5.2.1 (OK)
sympy >=0.7.3    :  None (NOK)
cython >=0.21    :  0.26 (OK)
qtconsole >=4.2.0:  4.3.0 (OK)
IPython >=4.0    :  6.1.0 (OK)
pylint >=0.25    :  1.7.2 (OK)

Copied from original issue: spyder-ide/spyder#4817

Don't hard code port number

Instead of hard-coding the port for the tornado app to 8070, it should be selected from the available ports in the system.

For that, we need to use select_port from spyder/utils/misc.py

Execute initial commands only if prompt is available

Incorporate Windows version using pywinpty

This should be a separate project, however it must be added as a dependency.

Tabify terminal next to the IPython console

Since the Python console is gone in Spyder 3.2, we need to tabify the terminal next to the IPython console in 0.2.

This only consists in changing extconsole for ipyconsole in the method called on_first_registration of TerminalPlugin.

Add project skeleton

Based on https://github.com/spyder-ide/spyder-plugin-cookiecutter

TypeError: 'Icon' object is not callable

From @wlsdnen on July 26, 2017 12:59

Description

What steps will reproduce the problem?

What is the expected output? What do you see instead?

Please provide any additional information below

File "c:\users\jjw\appdata\local\programs\python\python35\lib\site-packages\spyder_terminal\terminalplugin.py", line 156, in __wait_server_to_start
QMessageBox.Ok)
TypeError: 'Icon' object is not callable

Version and main components

• Spyder Version: 3.2.0
• Python Version: 3.5.3
• Qt Versions: 5.9.1, PyQt5 5.9 on Windows

Dependencies

pyflakes >=0.6.0 :  1.5.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  None (NOK)
numpy >=1.7      :  1.13.1 (OK)
sphinx >=0.6.6   :  1.6.3 (OK)
rope >=0.9.4     :  0.10.5 (OK)
jedi >=0.9.0     :  0.10.2 (OK)
nbconvert >=4.0  :  5.2.1 (OK)
sympy >=0.7.3    :  None (NOK)
cython >=0.21    :  0.26 (OK)
qtconsole >=4.2.0:  4.3.0 (OK)
IPython >=4.0    :  6.1.0 (OK)
pylint >=0.25    :  1.7.2 (OK)

Copied from original issue: spyder-ide/spyder#4817

Copy and Paste shortcuts are disabled

Terminal content not re-scaling to plugin size

With the latest Qt 5.8.x and latest version of Spyder the terminal doesn't change its size to fit the window it is in. It throws this error while loading (which might just be the case that it is not loading properly):

[...]
spyder_terminal/widgets/terminalgui.py", line 50, in __init__
    self.view.page().contentsChanged.connect(self.contents_modified)
   AttributeError: 'WebPage' object has no attribute 'contentsChanged'

After chasing the error through the code, it seems it has to do with the Qt version. From the Qt documentation the signal from QWebEnginePage, contentsChanged does no longer exist but another one contentsSizeChanged does exist.

I've changed Line 50 of terminalgui.py from

self.view.page().contentsChanged.connect(self.contents_modified)

to:

self.view.page().contentsSizeChanged.connect(self.contents_modified)

and it works fine now. I'm not even sure if it is what it is supposed to do. But either commenting or changing the line above seems to let the WebView load properly and to help it rendering the terminal full size. If not it crashes and it can't load properly.

Each terminal instance should be launched on currently opened path

The current working directory of a terminal should correspond to the cwd defined on Spyder.

Rename plugin to "Terminal"

@goanpeca suggested to avoid long names for plugins, so instead of System terminal this plugin should be named just Terminal.

Create PyPi distribution package and setuptools configuration

1. Create setup.pyaccording to https://packaging.python.org/distributing/
2. Test distribution on pip

Capture all keyboard events when a console is on focus

Right now, some shortcuts are interfering with key presses aimed to the console. For instance, to exit htop we need to press F10, however when that key is pressed, the shortcut that activates the code profiler is activated.

Display xterm terminal inside a QWebView widget

The widget must display the index.html served by the Tornado server.

Write package documentation and description

Extend README.md contents

Use check_compatibility for PyQt4

If users try to use this plugin with PyQt4 we need to show a message telling me that it only works in PyQt5.

Release Spyder Terminal v0.1.1

First bug fix release after v0.1

Release 0.1.2

@andfoy, please release 0.1.2. Important things to do:

1. Close this issue before creating our Changelog. An issue informing about a new release must be created every time so that the rest of the team knows about it.
2. Close the Github milestone before creating our Changelog so that it's created with the right date format.
3. Update our conda-forge feedstock.
4. Move conda-forge packages to the spyder-ide channel once they are uploaded to anaconda.org. Our installation instructions must not include conda-forge because installing from conda-forge is one of the main causes of breakage in Anaconda (when not done in a different environment from root, which almost nobody does).

Redirect server output and errors to a file

Server should start and end with Spyder process

The tornado server should be deployed when Spyder starts (Similarly when the process finishes).

Windows: Close websocket when a console is closed manually

For instance, when an user executes the command exit on a Windows CMD/Powershell console, it should present the same behaviour as on UNIX consoles. i.e., Websocket is closed and the message Pipe closed should be displayed.

Add CI Engines and Minor Tests

Add some tests and integrate them with CircleCI and Travis

Selection is broken as of xterm.js v2.7.0

According to xterm.js#670, selection is now managed by xterm instead of the browser, which may imply that we must redirect mouse input to xterm.js events as of 2.7.0.

clear command does not work

The clear command does not seem to function. I'm not sure if this is a xterm.js bug or a spyder-terminal bug.

What steps will reproduce the problem?

1. Launch terminal

What is the expected output? What do you see instead?

Expected:

[avylove@localhost Project]$      

Actual:

Loading...                              
[avylove@localhost Project]$ cd /home/avram/Desktop/Audrey      
[avylove@localhost Project]$ clear
[avylove@localhost Project]$

Please provide any additional information below
Neither clear nor reset commands seem to do anything. I tried changing the clearTerm() function from to user term.clear(), which removed the clear command from the terminal, but still didn't clear the terminal.

Versions and main components

• Terminal Version: Included
• Spyder Version: 0.1.2
• Python Version: 3.5.3
• Operating system: Fedora 25

Plugin "Options menu" is empty

When you press the little wheel on the top right corner (i.e. the Options menu), it doesn't return anything.

At least we should add there an entry to create a new terminal.

Server doesn't launch on Spyder startup

Description of your problem

The tornado server doesn't seem to launch on Spyder startup

Executing spyder_terminal/server/main.py after Spyder has started, I can connect to a terminal

What steps will reproduce the problem?

1. pip3 install --user spyder-terminal
2. Launch Spyder
3. Open terminal

What is the expected output? What do you see instead?
Should see a terminal, instead see:
This site can’t be reached
127.0.0.1 refused to connect.

Please provide any additional information below

Versions and main components

• Terminal Version: xterm.js included with spyder-terminal
• Spyder Version: 3.1.4
• Python Version: 3.5.3
• Operating system: Fedora 25

Update package dependencies on Windows

Windows version now depends on pywinpty. Also, Universal Wheel creation must be disabled on MANIFEST.in

Make mouse wheel to work

This is kind of annoying too: if you try to browse the contents of the terminal using the mouse wheel, it doesn't work. You have to directly press the scrollbar to do that.

I don't know if it's possible to fix this, but we should definitely give it a try.

Release 0.2

Getting annoying Qt warnings

I'm getting a very annoying Qt warning after starting or resizing terminals:

content-type missing in HTTP POST, defaulting to application/x-www-form-urlencoded. Use QNetworkRequest::setHeader() to fix this problem.

The problem seems the Post calls to the server are sent without a body, and Qt complaints about them.

@andfoy, it should be an easy fix for you :-)

Package missing in current win-64 channels

Description of your problem

What steps will reproduce the problem?

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\myname>conda install spyder_terminal -c spyder-ide
Fetching package metadata ...............

PackageNotFoundError: Package missing in current win-64 channels:

• spyder_terminal

What is the expected output? What do you see instead?

Expect successful install

Please provide any additional information below

Maybe you've only released 32-bit so far?

I'm not sure why it wouldn't be found since I was specifying your custom channel.

Versions and main components

• Terminal Version: not installed
• Spyder Version: 3.1.4
• Python Version: 3.6.1 64bits
• Operating system: Windows 7

Add server tests

Each terminal should be numbered sequentially

Set Terminal title as Terminal 1, 2, ...

Create Conda redistribution package

1. Create conda recipe according to: http://conda.pydata.org/docs/build_tutorials/pkgs.html
2. Test resdistribution using conda

Close automatically a Terminal tab if it was closed from the process itself

Right now, if exit or Ctrl+D is executed to finish a terminal process, we output the message Pipe closed and the tab remains open. We would like to override this behaviour by closing the tab when the process is finished.

Bold font may not be activated on new terminals

According to xterm.js#157, bold fonts are disabled if the isBoldBroken function determines if the current font has no bold variant. Right now it seems that our fallback font, ubuntu-powerline is not loading its bold variant, therefore, bold fonts are disabled by default, even if the font is changed via Qt.

We should bundle plain Ubuntu Mono instead, this font actually works and enables the bold variants.

Release v0.1

First production release of spyder-terminal.

TODO:

• Select ports
• Text overflow
• Terminal names
• Test PyQt 5.8
• Font selection

Create basic terminal shortcuts

The following terminal shortcuts must be defined:

• Copy (Ctrl + C)
• Paste (Ctrl + V)
• Launch a new instance (Ctrl + Shift + T)

Enable AppVeyor CI for Windows testing

Extract Javascript dependencies to a bower.json file

Dependencies should be bundled via bower.

Fix text overflow when commands fill all the terminal space

Required by #29

Implement basic web bash console using tornado and xterm.js

This terminal implementation should be compatible with the xterm.js API, besides, the terminal execution shall be asynchronous and non-blocking. Also it shall support POSIX compatible Operating Systems and Windows command execution.

Terminals should use user-defined fonts

The font should be loaded from Spyder preferences.

Prevent tests to get stuck on AppVeyor

Don't start to create consoles until the server is up and running

@andfoy, you can use this code to keep pinging the server until it's up (at least for 10 secs) and if not, report an error:

https://github.com/spyder-ide/spyder-notebook/blob/master/spyder_notebook/utils/nbopen.py#L65-L70

Error on a slow start

Description of your problem

What steps will reproduce the problem?

1. start spyder on a new winpython on a slow PC

2. I get this error:
   

3. then the Terminal is just a black window and a blinking cursor. starting new Terminal didn't provide a working Terminal

What is the expected output? What do you see instead?

• no failure if possible, some logic to adjust to a slow PC
• guidance to recover the Terminal in that case:
  

Please provide any additional information below

• spyder did 13 "Kernel died, restarting" during this slow starting process

Versions and main components

• Terminal Version: 0.2.0
• Spyder Version: 3.2.1dev of 20170729
• Python Version: 3.5.3 64bit
• Operating system: windows 7 64bit

pip list:

adodbapi (2.6.0.7)
alabaster (0.7.10)
algopy (0.5.5)
altair (1.2.0)
altair-widgets (0.1.2)
astroid (1.5.3)
astroML (0.3)
Babel (2.4.0)
backports-abc (0.5)
backports.weakref (1.0rc1)
baresql (0.7.4)
bcolz (1.1.2)
beautifulsoup4 (4.6.0)
bkcharts (0.2)
blaze (0.10.1)
bleach (2.0.0)
blosc (1.5.1)
bloscpack (0.11.0)
bokeh (0.12.6)
boto3 (1.4.1)
botocore (1.4.36)
Bottleneck (1.2.1)
bqplot (0.10.0a3)
brewer2mpl (1.4.1)
Brotli (0.6.0)
certifi (2017.4.17)
cffi (1.10.0)
chardet (3.0.4)
click (6.7)
cloudpickle (0.3.1)
clrmagic (0.0.1a2)
colorama (0.3.9)
coloredlogs (7.1)
comtypes (1.1.3)
CVXcanon (0.1.1)
cvxopt (1.1.9)
cvxpy (0.4.10)
cx-Freeze (5.0.2)
cycler (0.10.0)
cyordereddict (1.0.0)
Cython (0.26)
cytoolz (0.8.2)
dask (0.15.1)
datashape (0.5.2)
db.py (0.5.3)
decorator (4.1.2)
dill (0.2.7.1)
distributed (1.18.0)
docopt (0.6.2)
docrepr (0.1.0)
docutils (0.13.1)
ecos (2.0.4)
edward (1.3.3)
emcee (2.2.1)
entrypoints (0.2.3)
enum34 (1.1.6)
fast-histogram (0.2.1)
fastcache (1.0.2)
fasteners (0.14.1)
fastparquet (0.1.1)
feather-format (0.3.1)
Flask (0.12.2)
Flask-Cors (3.0.3)
formlayout (1.1.0)
fuzzywuzzy (0.15.0)
geopy (1.11.0)
gitdb2 (2.0.0)
GitPython (2.1.3)
gmpy2 (2.0.8)
greenlet (0.4.12)
guidata (1.7.6)
guiqwt (3.0.3)
h5py (2.7.0)
HeapDict (1.0.0)
holoviews (1.8.1)
html5lib (1.0b10)
humanfriendly (4.4)
husl (4.0.3)
idlex (1.13)
idna (2.5)
imageio (2.2.0)
imagesize (0.7.1)
ipykernel (4.6.1)
ipyleaflet (0.3.0)
ipympl (0.0.5)
ipyparallel (6.0.2)
ipython (6.1.0)
ipython-genutils (0.2.0)
ipython-sql (0.3.8)
ipywidgets (7.0.0b2)
isort (4.2.15)
itsdangerous (0.24)
jedi (0.10.2)
Jinja2 (2.9.6)
jmespath (0.9.3)
joblib (0.11)
jsonschema (2.6.0)
julia (0.1.2)
jupyter (1.0.0)
jupyter-client (5.1.0)
jupyter-console (5.1.0)
jupyter-core (4.3.0)
jupyter-sphinx (0.1.1)
jupyterlab (0.26.4)
jupyterlab-launcher (0.3.1)
Keras (2.0.6)
keras-vis (0.4.1)
Lasagne (0.2.dev1)
lazy-object-proxy (1.3.1)
llvmlite (0.19.0)
lmfit (0.9.7)
locket (0.2.0)
lxml (3.8.0)
Markdown (2.6.8)
MarkupSafe (1.0)
matplotlib (2.0.2)
mccabe (0.6.1)
metakernel (0.20.4)
mistune (0.7.4)
mizani (0.3.2)
mkl-service (1.1.2)
monotonic (1.3)
mpl-scatter-density (0.2)
mpld3 (0.3)
mpldatacursor (0.6.2)
mpmath (0.19)
msgpack-python (0.4.8)
multipledispatch (0.4.9)
multiprocess (0.70.5)
mypy (0.521)
mysql-connector-python (2.0.4)
nbconvert (5.2.1)
nbconvert-reportlab (0.1)
nbdime (0.3.0)
nbformat (4.3.0)
netCDF4 (1.2.9)
networkx (1.11)
nltk (3.2.4)
notebook (5.0.0)
numba (0.34.0)
numdifftools (0.9.20)
numexpr (2.6.2)
numpy (1.13.1+mkl)
numpydoc (0.7.0)
oct2py (4.0.6)
octave-kernel (0.26.2)
odo (0.5.0)
olefile (0.44)
palettable (3.0.0)
pandas (0.20.3)
pandas-datareader (0.5.0)
pandocfilters (1.4.1)
param (1.5.1)
partd (0.3.8)
patsy (0.4.1)
pdfrw (0.3)
pep8 (1.7.0)
pexpect (4.2.1)
pg8000 (1.10.5)
pickleshare (0.7.4)
Pillow (4.2.1)
pip (9.0.1)
pkginfo (1.4.1)
plotnine (0.2.1)
ply (3.10)
prettytable (0.7.2)
prompt-toolkit (1.0.15)
protobuf (3.3.0)
psutil (5.2.2)
ptpython (0.41)
ptyprocess (0.5.2)
PuLP (1.6.5)
Pweave (0.30.0a1)
py (1.4.34)
pyarrow (0.4.1)
PyAudio (0.2.11)
pybars3 (0.9.3)
pybind11 (2.1.1)
pycodestyle (2.3.1)
pycparser (2.17)
pyflakes (1.5.0)
pyflux (0.4.15)
pygame (1.9.3)
Pygments (2.2.0)
pylint (1.7.2)
pymc (2.3.6)
pymc3 (3.1)
PyMeta3 (0.5.1)
pymongo (3.4.0)
pyodbc (4.0.17)
PyOpenGL (3.1.1)
pypandoc (1.3.2)
pyparsing (2.2.0)
PyQt5 (5.9)
pyqtgraph (0.10.0)
pyreadline (2.0)
pyserial (3.4)
pystache (0.5.4)
pytest (3.1.2)
python-dateutil (2.6.1)
python-hdf4 (0.9)
python-Levenshtein (0.12.0)
python-snappy (0.5.1)
pythonnet (2.4.0.dev0)
PythonQwt (0.5.5)
pytz (2017.2)
PyWavelets (0.5.2)
pywin32 (221)
pywinpty (0.1.3)
pywinusb (0.4.2)
PyYAML (3.12)
pyzmq (16.0.2)
QtAwesome (0.4.4)
qtconsole (4.3.0)
QtPy (1.2.1)
redis (2.10.5)
reportlab (3.4.0)
requests (2.18.2)
requests-file (1.4.2)
requests-ftp (0.3.1)
requests-toolbelt (0.8.0)
rope (0.10.5)
rpy2 (2.8.6)
Rx (1.5.9)
s3fs (0.1.2)
s3transfer (0.1.10)
scikit-fuzzy (0.3.dev0)
scikit-image (0.13.0)
scikit-learn (0.18.2)
scikit-neuralnetwork (0.7)
scikit-optimize (0.3)
scilab2py (0.6)
scipy (0.19.1)
scs (1.2.6)
seaborn (0.8.0)
setuptools (36.2.4)
simplegeneric (0.8.1)
simplejson (3.11.1)
sip (4.19.3)
six (1.10.0)
smmap2 (2.0.3)
snakeviz (0.4.1)
snowballstemmer (1.2.1)
sortedcontainers (1.5.7)
sounddevice (0.3.8)
Sphinx (1.6.3)
sphinx-rtd-theme (0.2.4)
sphinxcontrib-websupport (1.0.1)
spyder (3.2.1.dev0)
spyder-notebook (0.1.0.dev0)
spyder-reports (0.1.0.dev0)
spyder-terminal (0.2.0)
SQLAlchemy (1.1.12)
sqlite-bro (0.8.11)
sqlparse (0.2.3)
statsmodels (0.8.0)
supersmoother (0.4)
sympy (1.1.1)
tables (3.4.2)
tblib (1.3.2)
tensorflow (1.2.1)
testpath (0.3.1)
Theano (0.9.0)
thriftpy (0.3.9)
toolz (0.8.2)
tornado (4.5.1)
tqdm (4.15.0)
traitlets (4.3.2)
traittypes (0.0.6)
twine (1.9.1)
twitter (1.17.1)
typed-ast (1.0.4)
urllib3 (1.21.1)
vega (0.5.0)
ViTables (3.0.0)
wcwidth (0.1.7)
webencodings (0.5.1)
Werkzeug (0.12.2)
wheel (0.29.0)
widgetsnbextension (3.0.0b2)
win-unicode-console (0.5)
winpython (1.9.20170715)
wordcloud (1.3.1)
wrapt (1.10.10)
xarray (0.9.6)
xlrd (1.0.0)
XlsxWriter (0.9.8)
xlwings (0.11.4)
zarr (2.1.4)
zict (0.1.2)

Move terminal webview to be inside a QFrame so that dock panes in Spyder are uniform

Make 0.2 to depend on Spyder 3.2

This is so we can use check_compatibility, added in spyder-ide/spyder#4644.

Please see what I did in spyder-ide/spyder-notebook@6513d24 to make the corresponding changes.