ipython / ipynb Goto Github PK
View Code? Open in Web Editor NEWPackage / Module importer for importing code from Jupyter Notebook files (.ipynb)
Home Page: https://ipynb.readthedocs.io/en/latest/
License: BSD 3-Clause "New" or "Revised" License
Package / Module importer for importing code from Jupyter Notebook files (.ipynb)
Home Page: https://ipynb.readthedocs.io/en/latest/
License: BSD 3-Clause "New" or "Revised" License
Hi,
I found that under certain circumstances __package__
is not defined and hence FSFinder has no package_prefix set. This leads to an error when trying to import a notebook.
Can it be that this is because __package__
is expected to be the same as spec.parent beginning with Python 3.6?
https://docs.python.org/3/reference/import.html#__package__
Anyway, relying on __package__
seems to be a risk.
Best,
Jens Henrik
...python3.6/site-packages/ipynb-0.5.1-py3.6.egg/ipynb/fs/finder.py in find_spec(self=<ipynb.fs.finder.FSFinder object>, fullname='ipynb.fs.full.job_submit', path=['...site-packages/ipynb-0.5.1-py3.6.egg/ipynb/fs/full'], target=None)
41 Claims modules that are under ipynb.fs
42 """
---> 43 if fullname.startswith(self.package_prefix):
fullname.startswith = <built-in method startswith of str object at 0x7f41805a6580>
self.package_prefix = None
44 for path in self._get_paths(fullname):
45 if os.path.exists(path):
TypeError: startswith first arg must be str or a tuple of str, not NoneType
I have strived to import an external ipynb, but I got the following error:
ImportError: Could not import /content/.ipynb for ipynb.fs.full.: incorrect version or language
I was wondering if I could know that you have ever experienced the same issue, please!
NB: I am running the codes on the Google Colab.
Thanks in advance
For some imported notebook, say demo1
import ipynb.fs.defs.demo1
is OK, but import ipynb.fs.defs.demo1 as d1
FAIL
and for other imported notebook, say demo2
import ipynb.fs.defs.demo2 as d2
is OK, but import ipynb.fs.defs.demo2
FAIL
Currently, we don't support IPython Magics at all from noteboks. We initially supported them, but removed them because importing IPython was causing a massive startup time lag (>1s) that seemed pretty bad.
Ideally, I'd like to be able to do the following:
I'd also like to possibly consider implementing the !
action (which runs things in a shell) natively, so we don't have to import all of IPython to get just that behavior. Also investigate why startup takes so long!
Likely on Travis-CI.
Ideally these will not require the ipynb package at all at runtime. We could do some amount of pre-processing to make sure we ship .py files rather than .ipynb files, and dependency on IPython itself will depend on #6.
This totally requires us to have a solid documented understanding of how relative imports work (see #4)
I've something like
import os
os.environ["KERAS_BACKEND"] = "tensorflow"
on top of my code (I know I can do that in some other way…) and ipynb.fs.defs fails to import this into another module. I like to fix this myself but honestly don't have good clue what should be done, excepting it from defs or accepting it? As this case, my favor is to accepting it but I am not sure if that would be what all want.
Hi
I fail to load a book due to an image in a MD cell.
I have a small notebook mylib.ipynb
[Cell 1 - Python]
# <api>
var = 1
[Cell 2 - MD]
![alt text](images/system-map.png "Title")
... the below also does not work...
<!-- <img src="images/system-map.png" width=1000 align="left"/> -->
And a notebook importing:
import ipynb.fs.defs.mylib
When running this, I get
File "<unknown>", line 11
![alt text](images/system-map.png "Title")
^
SyntaxError: invalid syntax
This is OSX. Python 3.6.1
Packages:
appdirs==1.4.3
appnope==0.1.0
attrs==16.3.0
Automat==0.5.0
awscli==1.11.87
bleach==2.0.0
botocore==1.5.50
cffi==1.9.1
click==6.7
colorama==0.3.7
constantly==15.1.0
cryptography==1.7.2
cssselect==1.0.1
cycler==0.10.0
decorator==4.0.11
docutils==0.13.1
entrypoints==0.2.2
et-xmlfile==1.0.1
excel-modelling-helper==0.3.0
Flask==0.12
Flask-SocketIO==2.8.4
google-api-python-client==1.6.2
html5lib==0.999999999
httplib2==0.10.3
idna==2.2
incremental==16.10.1
ipykernel==4.6.0
ipynb==0.4
ipython==5.3.0
ipython-genutils==0.2.0
ipywidgets==6.0.0
itsdangerous==0.24
jdcal==1.3
Jinja2==2.9.6
jmespath==0.9.2
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.0.1
jupyter-cms==0.7.0
jupyter-console==5.1.0
jupyter-core==4.3.0
lxml==3.7.2
MarkupSafe==1.0
matplotlib==2.0.0
mistune==0.7.4
mpmath==0.19
nbconvert==5.1.1
nbformat==4.3.0
networkx==1.11
nose==1.3.7
notebook==5.0.0
numpy==1.12.0
oauth2client==4.1.0
olefile==0.44
onedrivesdk==1.1.8
openpyxl==2.4.2
packaging==16.8
pandas==0.19.2
pandoc==1.0.0b2
pandocfilters==1.4.1
parsel==1.1.0
patsy==0.4.1
pbr==1.10.0
pexpect==4.2.1
pickleshare==0.7.4
Pillow==4.1.0
ply==3.10
prompt-toolkit==1.0.14
ptyprocess==0.5.1
pyasn1==0.2.3
pyasn1-modules==0.0.8
pycparser==2.17
PyDispatcher==2.0.5
pydot-ng==1.0.0
pydotplus==2.0.2
Pygments==2.2.0
pygraphviz==1.3.1
pyOpenSSL==16.2.0
pypandoc==1.3.3
pyparsing==2.2.0
PyQt5==5.7.1
pyspark==2.1.0+hadoop2.7
python-dateutil==2.6.0
python-engineio==1.2.3
python-socketio==1.7.1
pytz==2016.10
PyYAML==3.12
pyzmq==16.0.2
qtconsole==4.3.0
queuelib==1.4.2
requests==2.13.0
rsa==3.4.2
s3transfer==0.1.10
scipy==0.18.1
Scrapy==1.3.2
service-identity==16.0.0
simplegeneric==0.8.1
simplejson==3.10.0
sip==4.19
six==1.10.0
statsmodels==0.8.0
stevedore==1.20.0
sympy==1.0
tabulate==0.7.7
terminado==0.6
testpath==0.3
tornado==4.4.3
traitlets==4.3.2
Twisted==17.1.0
tzlocal==1.4
uritemplate==3.0.0
virtualenv==15.1.0
virtualenv-clone==0.2.6
virtualenvwrapper==4.7.2
w3lib==1.17.0
wcwidth==0.1.7
webencodings==0.5
Werkzeug==0.11.15
Whoosh==2.7.4
widgetsnbextension==2.0.0
xarray==0.8.2
xlrd==1.0.0
xlwt==1.2.0
zope.interface==4.3.3
Many thanks
Create a notebook with a number as the first character example 1-initial-process.ipynb
Create a second notebook and import the first notebook.
import ipynb.fs.full.1-initial-process as initial_proc
You will get a syntax error that highlights the 'dot' before the filename with the text "SyntaxError: invalid syntax"
When the first notebook is renamed to initial-process.ipynb, the import call works.
It's really wierd that when the source notebook has the magic cell funcction %matplotlib inline
statement,
the from ipynb.fs.defs.one_eda import guess_age
throws an error citing the same.
In non-interactive python 2, we can use from __future__ import absolute_import
to force explicit relative import. This is not working when, for example, we write some vanilla python files and relative-importing them from a notebook.
I tried pip install git+https://github.com/ipython/ipynb
but it gives that it requires python 3. Is there any workaround about this?
Let us suppose my main notebook (main.ipynb) is next to a folder otherpath containing another notebook to be imported (to_be_imported.ipynb).
If I do
import ipynb.fs.defs.otherpath.to_be_imported
I get
ModuleNotFoundError: No module named 'ipynb.fs.defs.otherpath'
If I move to_be_imported.ipynb in the same folder of the main file, everything works.
When cells in a notebook contain magic commands, ipynb cannot import the notebook.
For example, a cell in a notebook contains %matplotlib inline
. To import this notebook, ipynb throws an exception, SyntaxError: invalid syntax
.
Similar to .pyc files for .py files.
[ ] Investigate how .py -> .pyc compilation works
[ ] Investigate how much of a perf improvement it really is
I'm guessing we'll have user that want to import ..anipynbmodule
not sure how to do that explicitly.
importnb is a newer and better version of this package, that does the same things - but is more maintained.
I spoke to @tonyfast and there is interest in making this happen. It will help adoption, and keep this project moving forward.
Things to do:
I want to be able to import things straight out of github with the following format:
from ipynb.github.someuser.somerepo.branch_or_tag_or_sha.file_path import some_function
and have that work. Relative imports should also be supported for this so you can put a package of ipynb files on github.
Also investigate how to do this without killing github's servers and have them ban us.
Thanks for this great project! However, when I did pip install ipynb
I got version 0.4, which does not correctly comment out markdown cells when converting into code (fixed in commit b266e35, which lies between 0.4 and 0.5). Currently one must do pip install -e git+https://github.com/ipython/ipynb#egg=ipynb
to get the fixed version.
NB, the PyPI page also has an old project URL http://github.com/yuvipanda/ipynb instead of http://github.com/ipython/ipynb .
Right now, we only include assignments where the LHS is all caps. However, LHS all caps might have dependencies on RHS that's not all caps. These will not be evaluated, causing strange amounts of confusion.
Instead, we should attempt to track the dependencies on the RHS of such assignments, and eval those too (recursively). We should do this in such a way that the rules are very clear to explain to people.
This should still only be done for top level imports though.
When using import ipynb.fs.defs.myipynb
, I get an error:
File "<unknown>", line 81
alpha_dict[key] = alpha_dict[key].ewm(span=20,adjust=True,min_periods=0,ignore_na=False).mean()
^
IndentationError: unexpected indent
The cell is:
for key in alpha_dict:
alpha_dict[key] = alpha_dict[key].ewm(span=20,adjust=True,min_periods=0,ignore_na=False).mean()
I don' t know why would it try to import a for loop like this.
Make sure they're py3 and nbformat 4
I dont' have access to webhooks to set that up, but registered ipynb.readthedocs.org.
In one notebook (en2ru.ipynb) I create function:
def ruchar(): return 'Б'
In another notebook I import first notebook and call ruchar()
from ipynb.fs.full.en2ru import *
ruchar()
and I get 'Р‘'
Here is the error msg
File "", line 13
"""uses Gauss's method for summing integers
http://wmueller.com/precalculus/advanced/hint4_3_6.html"""
^
IndentationError: unexpected indent
Hi, I am posting this issue up here after having received zero help from Stackoverflow. I would greatly appreciate anyone's help on this!
Cryptic TypeError: 'decimal.Decimal' object cannot be interpreted as an integer
Hi and thanks for this great package.
I getting an error while opening a symbolic link notebook to the working directory. Do you have any recommendations how to over come that?
Thanks, HM
Assuming there is a function files I want to import: project/helper_fun.ipynb
My current working ipynb file is: project/branch/work.py
when I write from ipynb.fs.defs.helper_fun import fun1
in mywork.p
, it saysthere is no module called helper_fun
So my question is how to import the ipynb functions that are in a different folder.
Hi,
newbie here - I have tried to use ipynb in google colab with no success... Here is my code:
!pip install ipynb --upgrade
from ipynb.fs.defs.testingnb import *
and in the same folder I have a testingnb.ipynb file. I get the error:
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-7-8088143b2d48> in <module>()
----> 1 import ipynb.fs.defs.testingnb
ModuleNotFoundError: No module named 'ipynb.fs.defs.testingnb'
Am i missing some setup step? thanks!
Luigi
I'm in JupyterLab, with two notebooks open, one of which is importing a function from the other (tools.ipynb
):
import ipynb.fs.defs.tools
However, it doesn't seem to pick up changes from the other one, even when I re-run the cell containing the import. However, if I:
import importlib
importlib.reload(ipynb.fs.defs.tools)
Then it does indeed pick it up changes each time. Unfortunately, this doesn't work for the from X import Y
style, since the reference to the module containing the imported item isn't in scope to pass to reload
.
Is it possible that ipynb could watch timestamps on the imported files and handle this under the hood when the imported file changes? Thanks for a great project!
I was testing the following import:
import ipynb.fs.defs.utils.testfoo
and got the following exception:
... ipynb/fs/defs/__init__.py in <listcomp>(.0)
39 if isinstance(node, ast.Assign):
---> 40 return all([t.id.isupper() for t in node.targets])
AttributeError: 'Subscript' object has no attribute 'id'
So I modified the _filter_ast_node
to include a couple of try/excepts to see where it was failing:
if isinstance(node, ast.Assign):
try:
return all([t.id.isupper() for t in node.targets])
except AttributeError:
print(ast.dump(node))
for t in node.targets:
try:
t.id
except AttributeError:
print(ast.dump(t))
raise
ast.dump(node)
:
Assign(targets=[Subscript(value=Name(id='df', ctx=Load()), slice=Index(value=Str(s='test')), ctx=Store())], value=Str(s='foo'))
ast.dump(t)
:
Subscript(value=Name(id='df', ctx=Load()), slice=Index(value=Str(s='test')), ctx=Store())
Looks like it's occurring on the following line of my notebook:
df['test'] = 'foo'
One possible solution:
def _filter_ast_node(self, node):
for an in ALLOWED_NODES:
if isinstance(node, an):
return True
if isinstance(node, ast.Assign):
return all(self._id_is_upper(t) for t in node.targets)
return False
@staticmethod
def _id_is_upper(t):
try:
return t.id.isupper()
except AttributeError:
return False
Following https://stackoverflow.com/questions/44116194/import-a-function-from-another-ipynb-file
Python/Sagemath9.4/Jupyter error?
Using jupyter with kernel selections:sagemath9.4, python3
I found that the import works if the file pyt.ipynb is written with the Jupyter kernel Python3 selected, but can't be read if it's written from Sagemath9.4 kernel.
Is there a more correct way or is it a bug?
File pyt.ipynb to be imported
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
File for importing
from ipynb.fs.full.pyt import *
testing = factorial(5)
testing
Here is the diff output, comparing the python3 save and the sagemath9.4 save
diff pyt.ipynb pyt-python.ipynb
< "display_name": "SageMath 9.4",
< "language": "sage",
< "name": "sagemath"
'> "display_name": "Python 3",
'> "language": "python",
'> "name": "python3"
Note sure how to debug the following error. Attached is the notebook I tried to import. They are in the same directory
Any ideas?
Thanks
Andy
#from ipynb.fs.defs.week1 import patternToNumber
import ipynb.fs.defs.week1 as wk1
#import ipynb.fs.defs.week1 as week1
AttributeError Traceback (most recent call last)
in ()
----> 1 from ipynb.fs.defs.week1 import patternToNumber
2 #import ipynb.fs.defs.week1 as wk1
3
4 #import ipynb.fs.defs.week1 as week1
5 # import ipynb.fs from .defs.week2 \
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in find_and_load(name, import)
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in find_and_load_unlocked(name, import)
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec)
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap_external.py in exec_module(self, module)
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in get_code(self, fullname)
60 ))
61 return self.source_to_code(
---> 62 self._get_filtered_ast(code_from_ipynb(nb)),
63 self.path
64 )
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in _get_filtered_ast(self, source)
47 """
48 tree = ast.parse(source)
---> 49 tree.body = [n for n in tree.body if self._filter_ast_node(n)]
50 return tree
51
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in (.0)
47 """
48 tree = ast.parse(source)
---> 49 tree.body = [n for n in tree.body if self._filter_ast_node(n)]
50 return tree
51
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in _filter_ast_node(self, node)
38
39 if isinstance(node, ast.Assign):
---> 40 return all([t.id.isupper() for t in node.targets])
41
42 return False
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/init.py in (.0)
38
39 if isinstance(node, ast.Assign):
---> 40 return all([t.id.isupper() for t in node.targets])
41
42 return False
AttributeError: 'Tuple' object has no attribute 'id'
How about
with ipynb.importer(full=True): # or defs=True
import my_notebook
For example, it would allow to specify options like
with ipynb.importer(ignore_errors=True):
import my_notebook
Also, we may add the ability to write
import ipynb.finder
import nb1
import nb2
I can make a PR if it's ok
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.