goerz / jupytext.vim Goto Github PK
View Code? Open in Web Editor NEWVim plugin for editing Jupyter ipynb files via jupytext
License: MIT License
Vim plugin for editing Jupyter ipynb files via jupytext
License: MIT License
Describe the bug
Getting this error on opening file:
Error detected while processing BufReadCmd Autocommands for "*.ipynb"..function <SNR>41_read_from_ipynb:
line 24:
jupytext --to=md --output='C:/Users/vaqxai/Downloads/lab10/lab.md' 'C:/Users/vaqxai/Downloads/lab10/lab.ipynb': 1
Describe the bug
When trying to make a new copy of the notebook I'm working on with :saveas, the buffer name changes, the message "two.ipynb saved via jupytext" appears, but no new file is created, while the previous one is overwritten. Steps to reproduce:
Diagnostics
vim --version
:centi@centi-Latitude-3540:~$ nvim --version
NVIM v0.10.0-dev
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Run "nvim -V1 -v" for more info
uname -a
):centi@centi-Latitude-3540:~$ uname -a
Linux centi-Latitude-3540 6.5.0-18-generic #18~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 7 11:40:03 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
python --VV
:centi@centi-Latitude-3540:~$ python -VV
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
let g:jupytext_print_debug_msgs = 1
in your ~/.vimrc
. What is the output of :messages
when reproducing the problem?// :ed one.ipynb
DBG: filename: /home/centi/one.ipynb
DBG: filename exists: 0
DBG: jupytext_file: /home/centi/one.py
DBG: jupytext_file exists: 0
DBG: autocmd jupytext_ipynb BufUnload <buffer> call s:cleanup("/home/centi/one.py", 1)
DBG: autocmd jupytext_ipynb BufWriteCmd,FileWriteCmd <buffer> call s:write_to_ipynb()
DBG: filetype: python
// :w
DBG: overwriting /home/centi/one.py
DBG: Updating notebook from /home/centi/one.py
DBG: cmd: jupytext --from=py --to=ipynb --update '/home/centi/one.py'
DBG: [jupytext] Reading /home/centi/one.py in format py^@[jupytext] Writing /home/centi/one.ipynb
// :saveas two.ipynb
DBG: overwriting /home/centi/one.py
DBG: Updating notebook from /home/centi/one.py
DBG: cmd: jupytext --from=py --to=ipynb --update '/home/centi/one.py'
DBG: [jupytext] Reading /home/centi/one.py in format py^@[jupytext] Writing /home/centi/one.ipynb (destination file updated)
Does converting the notebook to/from ipynb with jupytext
on the command line work? Not tested, conversion works correctly in vim.
Does it work when you set g:jupytext_command
in ~/.vimrc
to be the exact some jupytext
that you used manually, with the exact same version of Python? Not tested, conversion works correctly in vim.
Describe the bug
It looks nice when open the file with neovim. However, when starting edit the buffer, the file format changed from python to json. All syntax highlight lost. Edits are saved back to the notebook.
I set the format to python, but the problem happens with md as well.
Diagnostics
vim --version
: NVIM 0.9.4
Operating system information (e.g. uname -a
): Darwin MBP
python --VV
: unknown option --VV
Are you using Anaconda?
NO. miniconda
Put let g:jupytext_print_debug_msgs = 1
in your ~/.vimrc
. What is the output of :messages
when reproducing the problem?
NO output
Does converting the notebook to/from ipynb with jupytext
on the command line work?
YES
Does it work when you set g:jupytext_command
in ~/.vimrc
to be the exact some jupytext
that you used manually, with the exact same version of Python?
YES
When closing .ipynb files on Win 10, the plugin fails to delete the generated .md files. You can see the redacted log below:
DBG: filename: C:\path\to\file.ipynb
DBG: filename exists: 1
DBG: jupytext_file: C:\path\to\file.md
DBG: jupytext_file exists: 0
DBG: Generate file C:\path\to\file.md
DBG: cmd: jupytext --to=md --output="C:\path\to\file.md" "C:\path\to\file.ipynb"
DBG: [jupytext] Reading C:\path\to\file.ipynb in format ipynb^@[jupytext] Writing 'C:\path\to\file.md'
DBG: read C:\path\to\file.md
DBG: autocmd jupytext_ipynb BufUnload <buffer> call s:cleanup("C:\path\to\file.md", 1)
DBG: autocmd jupytext_ipynb BufWriteCmd,FileWriteCmd <buffer> call s:write_to_ipynb()
DBG: filetype: markdown
DBG: a:jupytext_file:C:pathtofile.md
DBG: deleting C:pathtofile.md
This happens because of the double quotes in s:cleanup("C:\path\to\file.md", 1)
: backslash is treated as an escape symbol. If there is no particular reason to use double quotes instead of single ones, changing 2 lines of code will solve the problem (at least it did for me).
Probably not the best place to ask this question, but I'm quite new to Jupyter Notebook, so I'll give it a try. I do all my notebook editing in vim through this plugin, but any changes made are not reflected in Jupyter Notebook until a manual reload is done in JN; I can verify that both the .ipynb and .py files have changed on disk. Is there any way to automatically trigger a reload of the .ipynb file within JN?
I've tried something like this, but as far as I can tell it applies to submodules that the current file relies on, and not the current file itself.
When you set
let g:jupytext_filetype_map = {'md:myst': 'markdown'} let g:jupytext_extension_map = {'md:myst': 'md'} let g:jupytext_fmt = 'md:myst'
the new format isn't taken into account and opening a file yields an error about unrecognized format despite jupytext --to md:myst
working
A clear and concise description of what the bug is.
Diagnostics
vim --version
: NVIM v0.4.4
Operating system information (e.g. uname -a
): SMP Debian 5.14.6-3 (2021-09-28) x86_64 GNU/Linux
python --VV
: Python 3.9.7 (default, Sep 24 2021, 09:43:00)
Are you using Anaconda? No
Does converting the notebook to/from ipynb with jupytext
on the command line work? Yes
Does it work when you set g:jupytext_command
in ~/.vimrc
to be the exact some jupytext
that you used manually, with the exact same version of Python? Yes
Trying to load a blank ipynb notebook throws an error since it is not of a valid JSON format. Trying to load a template on 'BufNewFile' gets overwritten by jupytext.
" Create .ipynb with skeleton
autocmd BufNewFile *.ipynb 0r ~/.vim/skeletons/skeleton.ipynb
One alternate way that I tried to explore is to disable the auto conversion of ipynb to allow vim templates to load, saving and reloading with jupytext. Disabling auto conversion can be done by unsetting let g:jupytext_enable = 1
but being someone relatively new to vim, I am not sure on how to manually call the jupytext extension to do its thing.
Any help would be appreciated. Thanks in advance!
A notebook with spaces in the filename results in an empty buffer
With an empty vimrc, on either Ubuntu or Windows, jupytext.vim quickly displays this error message:
Error detected while processing function <SNR>11_read_from_ipynb:
line 40:
E121: Undefined variable: g:jupytext_filetype_map
E116: Invalid arguments for function get(g:jupytext_filetype_map, g:jupytext_fmt, s:jupytext_filetype_map[g:jupytext_fmt])
E15: Invalid expression: get(g:jupytext_filetype_map, g:jupytext_fmt, s:jupytext_filetype_map[g:jupytext_fmt])
line 42:
E121: Undefined variable: l:ft
E116: Invalid arguments for function <SNR>11_debugmsg
line 43:
E121: Undefined variable: l:ft
E15: Invalid expression: "set ft=".l:ft
Hello,
thanks for your plugin, I have been playing with it a bit and it is great.
I have a question regarding the usage though. I mostly code in R and find Rmd (rmarkdown
) format superior to alternatives. However I see the advantage of being able to share ipynb
to other people.
The question I have is then: is it possible for me to edit a Rmd file and have it "transformed" to ipynb
in real time (basically the opposite of what I think this plugin allows to do)
EDIT:
Basically I think what I am asking is is it possible to setup a map from -> to like
{'ipynb' : 'rmarkdown', 'rmarkdown' : 'ipynb', 'md' : 'ipynb'}
Thanks and regards
While opening an .ipynb
file, an error will occur:
Error detected while processing BufReadCmd Autocommands for "*.ipynb"..function <SNR>17_read_from_ipynb:
line 24:
jupytext --to=md --output='/private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Personal/IBDP/BusinessManage
ment/IA/j_p.md' '/private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Personal/IBDP/BusinessManagement/IA/j_
p.ipynb': 1
However, if an .md
file with the same name generated by Jupytext is there, the .ipynb
file can be opened, changes made in the .ipynb
file is synced to the .md
file.
After pressing Enter to proceed, the file will seem to be empty (as conversion failed).
Jupytext itself works well. For-back conversions can be done correctly.
.ipynb
file via Jupyter Lab/NotebookMessages from :messages
after let g:jupytext_print_debug_msgs = 1
:
DBG: filename: /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.ipynb
DBG: filename exists: 1
DBG: jupytext_file: /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md
DBG: jupytext_file exists: 0
DBG: Generate file /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md
DBG: cmd: jupytext --to=md --output='/private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md' '/pr
ivate/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.ipynb'
DBG: pandoc: command not found^@[jupytext] Reading /private/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Documents/Loca
l/j_p.ipynb in format ipynb^@Traceback (most recent call last):^@ File "/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/
Library/bin/jupytext", line 8, in <module>^@ sys.exit(jupytext())^@ ^^^^^^^^^^^@ File "/var/mobile/Containers/Data/Application/6B7E2B9
2-E388-481D-8893-F1E3F0AAE8B8/Library/lib/python3.11/site-packages/jupytext/cli.py", line 488, in jupytext^@ exit_code += jupytext_single_file(nb_f
ile, args, log)^@ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^@ File "/var/mobile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1
E3F0AAE8B8/Library/lib/python3.11/site-packages/jupytext/cli.py", line 864, in jupytext_single_file^@ base_path(nb_dest, dest_fmt)^@ File "/var/mo
bile/Containers/Data/Application/6B7E2B92-E388-481D-8893-F1E3F0AAE8B8/Library/lib/python3.11/site-packages/jupytext/paired_paths.py", line 55, in base
_path^@ raise InconsistentPath(^@jupytext.paired_paths.InconsistentPath: Notebook path ''/private/var/mobile/Containers/Data/Application/6B7E2B92-E
388-481D-8893-F1E3F0AAE8B8/Documents/Local/j_p.md'' was expected to have extension '.md'^@
I love this plugin, using it quite often, however I find it failing for plugins (LSP servers) that need/use files on disk (not vim buffers).
I think when opening an ipynb
file, the name of the vim buffer (foo.ipynb
) is misleading, I am not editing the original .ipynb
file, but the converted plain file .py
.
Is there an option to rename the vim buffer to foo.py
? The automatic write back to the ipynb
is great, as well as deleting the py
file if I opened a .ipynb
.
Hello,
I am using the following options let g:jupytext_fmt = 'R:spin'
as allowed by jupyter --help
As a matter of fact the following works perfectly
jupytext --set-formats R:spin,ipynb,Rmd --sync test.R
[jupytext] Reading test.R
[jupytext] Updating notebook metadata with '{"jupytext": {"formats": "R:spin,ipynb,Rmd"}}'
[jupytext] Updating 'test.ipynb'
[jupytext] Updating 'test.Rmd'
[jupytext] Updating 'test.R'
But when I try to vim test.ipynb
, when I save the file the conversion fails because you jupytext.vim
try to run jupytext --to=R:spin --output='/home/statquant/CodeProjects/jupyter/workflow_4/test.r' '/home/statquant/CodeProjects/jupyter/workflow_4/test.ipynb' if it was .R all would be fine
Describe the bug
I used let g:jupytext_fmt = 'py'
but there is no python syntax highlighting.
Diagnostics
vim --version
: VIM - Vi IMproved 8.1 (2018 May 18
Operating system information (e.g. uname -a
): macOS 10.14.3
python --VV
: Python 2.7.10
Are you using Anaconda? No
Put let g:jupytext_print_debug_msgs = 1
in your ~/.vimrc
. What is the output of :messages
when reproducing the problem?
Messages maintainer: Mike Williams <[email protected]>
DBG: filename: /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.ipynb
DBG: filename exists: 1
DBG: jupytext_file: /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py
DBG: jupytext_file exists: 0
DBG: Generate file /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py
DBG: cmd: !jupytext --to=py --output='/Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py' '/Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.ipynb'
DBG: read /Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py
DBG: autocmd BufUnload <buffer> call s:cleanup("/Users/atcold/Work/GitHub/pytorch-minicourse/03-autograd_tutorial.py", 1)
DBG: filetype: py
Describe the bug
If the ipynb file is overwritten, for example if the user has an autosave autocmd, the update command fails.
jupytext
on the command line work?No if using the --update
argument, yes otherwise.
I'm unsure about the best way to go about this, but possibly, if jupytext fails to update, try again without the update? Potentially with a check for the ipynb file contents now being the same as the buffer?
When creating a notebook with vim
and jupytext.vim
(vim notebook.ipynb
), if not any cell has a title when first writing the file (:w
), then following writings of the file will make titles disappear. It does not happen if a title was present the first time the notebook was written.
What I call a title here is the short description of the cell that can be written next to # %%
in a py:percent
format.
# %% imports <-- "title"
import os, sys
All of my configuration wrt jupytext
is as follows:
Plug 'goerz/jupytext.vim'
let g:jupytext_fmt = 'py:percent'
Reproduce
vi notebook.ipynb
, then type:# %%
import os, sys
:w
write the file# %% imports <-- title added
import os, sys
:w
write the file:e
reload the file, the header disappearedDiagnostics
uname -a
):$ uname -a
Linux IMT-21092020 5.11.0-38-generic #42~20.04.1-Ubuntu SMP Tue Sep 28 20:41:07 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
python --VV
:$ python -V
Python 3.9.7
$ which jupytext
/home/p20aime/miniconda3/bin/jupytext
let g:jupytext_print_debug_msgs = 1
in your ~/.vimrc
. What is the output of :messages
when reproducing the problem?Writing the file the first time (:w
) without header:
DBG: overwriting /tmp/python/nb3.py
DBG: Updating notebook from /tmp/python/nb3.py
DBG: cmd: jupytext --from=py:percent --to=ipynb --update '/tmp/python/nb3.py'
DBG: [jupytext] Reading /tmp/python/nb3.py in format py:percent^@[jupytext] Writing /tmp/python/nb3.ipynb
nb3.ipynb saved via jupytext.
Writing an header and writing the file (:w
):
DBG: overwriting /tmp/python/nb3.py
DBG: Updating notebook from /tmp/python/nb3.py
DBG: cmd: jupytext --from=py:percent --to=ipynb --update '/tmp/python/nb3.py'
DBG: [jupytext] Reading /tmp/python/nb3.py in format py:percent^@[jupytext] Writing /tmp/python/nb3.ipynb (destination f
ile updated)
nb3.ipynb saved via jupytext.
jupytext
on the command line work? Does it work when you set g:jupytext_command
in ~/.vimrc
to be the exact some jupytext
that you used manually, with the exact same version of Python?When trying to replicate steps with command line it does work. That is weird because when doing through vim only then an inspection of the temporary .py
file shows that it indeed contains the headers.
$ vi notebook.ipynb
$ jupytext --to py:percent notebook.ipynb
[jupytext] Reading notebook.ipynb in format ipynb
[jupytext] Writing notebook.py in format py:percent
$ cat notebook.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "00069045",
"metadata": {}, <-- no title
"outputs": [],
"source": [
"import os"
]
}
],
"metadata": {
"jupytext": {
"cell_metadata_filter": "-all",
"main_language": "python",
"notebook_metadata_filter": "-all"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
$ vi notebook.py
# Adding a title to the cell
$ cat notebook.py
# %% imports
import os
$ jupytext --from py:percent --to notebook --update notebook.py
[jupytext] Reading notebook.py in format py:percent
[jupytext] Writing notebook.ipynb (destination file updated)
$ cat notebook.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "00069045",
"metadata": {
"title": "imports" <- title
},
"outputs": [],
"source": [
"import os"
]
}
],
"metadata": {
"jupytext": {
"cell_metadata_filter": "-all",
"main_language": "python",
"notebook_metadata_filter": "-all"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
$ vi notebook.ipynb
# I can see the title OK
This plugin does not play nice with various git-related plugins such as fugitive, gitgutter and signify.
For gitgutter
, bogus signs are shown, based on the output of
git diff --no-color --no-ext-diff -U0 -- <ipynbfile>
Personally, I already use nbdime as a diff tool for notebooks, but its output is (obviously) not valid for jupytext. Probably, the only solution is to create a diff tool based on jupytext that is used in the context of git-gutter
only. This would probably depend on airblade/vim-gitgutter#568 as well.
Alternatively, one would probably want to just disable git-gutter
for notebook files.
For signfiy
, no signs are shown at all (presumably because it can detect that the generated patch is bogus?). In principle, the solution should be the same as for gitgutter
, except that signfiy
appears to be better set up to handle using a custom diff-tool for notebook files. However, that diff-tool would still need to exist.
The fugitive
plugin appears simply inoperative (e.g. the Gdiff
command is "not an editor command"). I'm not sure what it would take to fix this, but it seems like something that should be possible. For now, "inoperative" is a better place to be in than "broken".
Currently vim's :DiffOrig command starts a diff between the current modified buffer (which is a notebook converted to a markdown file using jupytext) and the on disk copy of the original .ipynb notebook. Can we make this plugin first convert the opened .ipynb file to markdown and then do a diff, when :DiffOrig is used?
while editing the md file it work perfect, the ipynb is updated
but i need to close and re open the ipynb file in the jupyter lab page to trigger the update in jupyter lab
after installing fresh jupyter-lab with conda
and after checking installed lab extension both jupyterlab_jupytext jupytext enabled
but it failed to validate
i think this might the issue causing it won't update the ipynb file
jupyter_lsp enabled
- Validating jupyter_lsp...
jupyter_lsp 2.2.5 ok
jupyter_server_terminals enabled
- Validating jupyter_server_terminals...
jupyter_server_terminals 0.5.3 ok
jupyterlab enabled
- Validating jupyterlab...
jupyterlab 4.2.2 ok
jupyterlab_jupytext enabled
- Validating jupyterlab_jupytext...
jupyterlab_jupytext ok
jupytext enabled
- Validating jupytext...
X validation failed
notebook_shim enabled
- Validating notebook_shim...
notebook_shim ok
I am using the py:percent format converting from .ipynb. The syntax highlighting is working well, but I don't get any linting with ALE as I normally do with .py files.
NVIM v0.5.0-nightly-1445-g1ff5b60cb
Build type: Release
LuaJIT 2.0.5
with Python 3.8.0 (default, Nov 8 2019, 09:53:30)
I know that it's possible to set jupytext_fmt
to convert the .ipynb to a given format, either, py
, jl
, md
, etc. But I use both Julia and Python Jupyter Notebooks in my work. Is it possible to detect the notebook language and dynamically convert to .py
or .jl
?
I could convert to .md
, but this would prevent me using https://github.com/jalvesaq/vimcmdline to run code in a repl.
I thought it might be possible to add support for reading metadata:language
in the Notebook JSON, e.g.:
"metadata": {
"kernelspec": {
"display_name": "Julia 1.6.2",
"language": "julia",
"name": "julia-1.6"
Is this a feature that already exists? Or could it be implemented?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.