Auto-start the jupyter notebook server if none exists

[FelixKratz]

Auto-start the jupyter notebook server if none exists

[kiyoon]

Thanks for the cool suggestion!

I think Jupyter Notebook doesn't provide me with the token with the API. I can launch the server but can't know the token. Maybe there's a way if I delve into their implementation..
A little hacky way is I can maybe infer the URL from the output and open it. I'll look into this.
In the meantime, you can use password rather than token. You can even diable the password completely, which will make it easier to connect.

[FelixKratz]

You can supply a custom token for the server to use as an additional argument along with a bunch of other things:

jupyter notebook --NotebookApp.token=1234

The directory can be specified via the argument --NotebookApp.notebook_dir=dir or --NotebookApp.open_browser=False prevents the default browser from auto starting

[kiyoon]

@FelixKratz I originally thought about reading the URL but this looks simpler, although it will add some security risk. I think I will just make the command configurable and give you a default command template. I think the token has to be configurable also, but if it stays in the init.lua it is also a little risky

[FelixKratz]

@FelixKratz I originally thought about reading the URL but this looks simpler, although it will add some security risk. I think I will just make the command configurable and give you a default command template. I think the token has to be configurable also, but if it stays in the init.lua it is also a little risky

If the token is generated somewhat randomly it should be fine, as it is only hosted on the loopback interface anyways.

[kiyoon]

The current :JupyniumStartAndAttachToServer doesn't really start syncing. Obviously because you need some manual input to open the notebook. I can make it till it opens the notebook but not sure if I should force start syncing. If it starts syncing, should I sync from the ipynb or from the ju.py? So many variables..

I think it makes sense to sync from the ju.py because it's the one that is open and you can check before the command. How do you think?

[FelixKratz]

If I have a *.ju.py file open I could imagine that it would be nice to have a command that starts the notebook server, starts jupynium and starts syncing the file to the server, such that it is an all-in-one command. When the notebook server is run from jupynium directly there is no reason to wait with syncing, because no manual input is needed. Maybe a new command? Something like: JupyniumNotebookFromBuffer or something a bit shorter ^^

If a server is already running on 8888 this command should probably fall back to attaching to the server and syncing the new file to a new ipynb instance

[kiyoon]

I wanted to make commands as self-explanatory as possible so users have no confusion. Do you fuzzy-find the commands? I'm using wilder.nvim and I start server by typing :jara<tab> for example. I get that it sometimes gets too long though.
I can leave the current behaviour as is so that you can customise. You can make a custom function that calls both commands (or lua functions).
And maybe I can add the fallback behaviour when server is not available, it will open the server and open the notebook if it exists and start syncing.

[FelixKratz]

I wanted to make commands as self-explanatory as possible so users have no confusion. Do you fuzzy-find the commands? I'm using wilder.nvim and I start server by typing :jara<tab> for example. I get that it sometimes gets too long though.
I can leave the current behaviour as is so that you can customise. You can make a custom function that calls both commands (or lua functions).
And maybe I can add the fallback behaviour when server is not available, it will open the server and open the notebook if it exists and start syncing.

Nice, at some point I will create a keyboard shortcut to execute these commands.

[FelixKratz]

Another idea that I just had was that when the jupyter notebook process is started via jupynium in nvim it could be nice to have a popup window available in nvim that shows the console output of the jupyter notebook process. It is not important, but could be nice to have to check for the status of plugins etc.

[kiyoon]

By popup you mean another split or using something like plenary popup and make it look like Telescope? I've never used the popup feature before but in that case you can only see the output when it's open and you're not changing the buffer right?

[FelixKratz]

I had something like telescope in mind, but this is something I would maybe use only once or twice a month to be honest.

[kiyoon]

This has been almost implemented (except the popup stuff).

You can try the feat/autostart-notebook branch (branch = 'feat/autostart-notebook' on packer) to test it out before it gets released.

[kiyoon]

Now this is done. Thank you for the awesome feedback!

[FelixKratz]

First of all, thank you for adding this awesome feature! It is really nice.

I have noticed a problem however, it seems that the auto sync is not working, a tab appears and then instantly closes again:

bug.mp4

This is my config:

require("jupynium").setup({
  -- Conda users:
  -- python_host = "~/miniconda3/envs/jupynium/bin/python",
  python_host = vim.g.python3_host_prog or "python3",

  default_notebook_URL = "localhost:8888",

  -- Open the Jupynium server if it is not already running
  -- which means that it will open the Selenium browser when you open this file.
  -- Related command :JupyniumStartAndAttachToServer
  auto_start_server = {
    enable = true,
    file_pattern = { "*.ju.*" },
  },

  -- Attach current nvim to the Jupynium server
  -- Without this step, you can't use :JupyniumStartSync
  -- Related command :JupyniumAttachToServer
  auto_attach_to_server = {
    enable = true,
    file_pattern = { "*.ju.*", "*.md" },
  },

  -- Automatically open an Untitled.ipynb file on Notebook
  -- when you open a .ju.py file on nvim.
  -- Related command :JupyniumStartSync
  auto_start_sync = {
    enable = true,
    file_pattern = { "*.ju.*", "*.md" },
  },

  -- Automatically keep filename.ipynb copy of filename.ju.py
  -- by downloading from the Jupyter Notebook server.
  -- WARNING: this will overwrite the file without asking
  -- Related command :JupyniumDownloadIpynb
  auto_download_ipynb = false,

  -- Always scroll to the current cell.
  -- Related command :JupyniumScrollToCell
  autoscroll = {
    enable = true,
    mode = "always", -- "always" or "invisible"
    cell = {
      top_margin_percent = 20,
    },
  },

  scroll = {
    page = { step = 0.5 },
    cell = {
      top_margin_percent = 20,
    },
  },

  use_default_keybindings = true,
  textobjects = {
    use_default_keybindings = true,
  },

  -- Dim all cells except the current one
  -- Related command :JupyniumShortsightedToggle
  shortsighted = false,
})

[kiyoon]

Oh, sorry for the trouble. I tested it on Mac also and it's working okay for me..
So if you don't autostart the server and start syncing, it works?

[FelixKratz]

If I press 'n' it works

[kiyoon]

You mean if you press 'n', and then again run :JupyniumStartSync it works?
I want to know if it's a problem caused by that function, or from the autostarting logic.
I can't reproduce it now. Do you use Mac Homebrew python 3.10 or above? There's a bug that will be fixed from the pip side, which breaks the installation on mac. Maybe that's affecting this too (like maybe some files are not correctly installed)

Maybe can you try building with pip3 install -e . or /usr/bin/pip3 install .?

Or is it possible to run with CLI nvim --listen /tmp/jupy_nvim and jupynium --nvim_listen_addr /tmp/jupy_nvim and paste the output here?

[FelixKratz]

You mean if you press 'n', and then again run :JupyniumStartSync it works? I want to know if it's a problem caused by that function, or from the autostarting logic. I can't reproduce it now. Do you use Mac Homebrew python 3.10 or above? There's a bug that will be fixed from the pip side, which breaks the installation on mac. Maybe that's affecting this too (like maybe some files are not correctly installed)

Maybe can you try building with pip3 install -e . or /usr/bin/pip3 install .?

Or is it possible to run with CLI nvim --listen /tmp/jupy_nvim and jupynium --nvim_listen_addr /tmp/jupy_nvim and paste the output here?

No, if I press 'n' it works without doing anything else. So 'n' does what I thought 'y' would.

[kiyoon]

That's bizarre. It doesn't happen to me. 'n' opens a file and start syncing as well? I need to investigate why that happens

[FelixKratz]

That's bizarre. It doesn't happen to me. 'n' opens a file and start syncing as well? I need to investigate why that happens

Yes, here is a video of that:

bug.mp4

There is another problem, when the .ju.py file does not exist yet and is only opened via nvim test.ju.py it hangs completely

[kiyoon]

That's really funny..

jupynium.nvim/src/jupynium/cmds/jupynium.py

Lines 403 to 410 in 3b86f69

	else:
	sync_input = nvim.eval(
	f"""input("There is no notebook {sync_ipynb_name}. Do you want to create and start syncing? (y/n): ")"""
	)
	if sync_input in ["y", "Y"]:
	nvim.lua.Jupynium_start_sync(
	bufnr, sync_ipynb_name, False, async_=True
	) # bufnr, filename, ask

As you can see it should only do something when the input is y. Also can't really reproduce the hanging issue. What's your python version (I'm assuming Homebrew python 3.10)? I'll test on mac again and will come back to you.

[FelixKratz]

python version is

 # python --version
Python 3.9.7

[kiyoon]

Is that the one that installed Jupynium?

Will this work?

python -m jupynium --version

[FelixKratz]

Is that the one that installed Jupynium?

Will this work?

python -m jupynium --version

 # python -m jupynium --version
Jupynium v0.1.dev20+g3b86f69

[FelixKratz]

Maybe there is something strange going on in my neovim....

[kiyoon]

Version looks fine. Mine is Jupynium v0.1.1.dev41+g3b86f69 (I didn't know the formatting of version is different on different system) but the commit is at least the same

[kiyoon]

I finally reproduced. I'll see and let you know. Sorry for the hassle

[kiyoon]

The issue is when auto_start_sync is enabled it will start syncing on top of the fallback sync behaviour together. That's why n will sync normally as well and y clashes

[FelixKratz]

Ah yes, that makes sense!

[kiyoon]

It should be fixed now by #25