rasbt/watermark

An IPython magic extension for printing date and time stamps, version numbers, and hardware information

watermark's Issues

Getting Info about the Python environment

As suggested by @bollwyvl it would be a nice feature to also add a flag for additional info about the Python "environment", e.g,.

• python
  • conda
• "native" managers:
  • apt
  • dnf / yum
  • brew
• other vcs
  • hg

I feel like we are slowly running out of 1-letter flags ... Right now, we have

-v, --python          prints Python and IPython version

which prints something like:

CPython 3.4.3
IPython 3.0.0

I would suggest to add an integer argument for verbosity here (default: 1).

for example,

%watermark -v 0
CPython 3.4.3


%watermark -v 1    (default)
CPython 3.4.3
IPython 3.0.0


%watermark -v 2   
CPython 3.4.3
IPython 3.0.0
pip 7.1.0
conda 3.16.0


%watermark -v 3
CPython 3.4.3
IPython 3.0.0
pip 7.1.0
conda 3.16.0
/Users/sebastian/miniconda3/bin/python

But I am open to suggestions!

--iversions returns an error if no package has been imported

Example (watermark v1.8.1):

%load_ext watermark
%watermark --iversions

the output (traceback) is the following:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-f12554b3f891> in <module>
----> 1 get_ipython().run_line_magic('watermark', '--iversions')

~/.pyenv/versions/miniconda3-latest/envs/myenvname/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
   2311                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2312             with self.builtin_trap:
-> 2313                 result = fn(*args, **kwargs)
   2314             return result
   2315

</home/myusername/.pyenv/versions/miniconda3-latest/envs/myenvname/lib/python3.7/site-packages/decorator.py:decorator-gen-126> in watermark(self, line)

~/.pyenv/versions/miniconda3-latest/envs/myenvname/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188
    189         if callable(arg):

~/.pyenv/versions/miniconda3-latest/envs/myenvname/lib/python3.7/site-packages/watermark/watermark.py in watermark(self, line)
    133                 self._get_git_branch(bool(args.machine))
    134             if args.iversions:
--> 135                 self._print_all_import_versions(self.shell.user_ns)
    136             if args.watermark:
    137                 if self.out:

~/.pyenv/versions/miniconda3-latest/envs/myenvname/lib/python3.7/site-packages/watermark/watermark.py in _print_all_import_versions(vars)
    245                             continue
    246
--> 247         longest = max([len(i[0]) for i in to_print]) + 1
    248         for entry in to_print:
    249             print(('%s' % entry[0]).ljust(longest) + entry[1])

ValueError: max() arg is an empty sequence

Remove Travis CI config and replace build status badge with AppVeyor?

Travis CI no longer works for open-source projects, and I see recent PRs in this repo are being tested via AppVeyor, but the main README.md still lists a Travis CI badge that links to an inactive project, and there's still a .travis.yml config that can be removed.

https://www.appveyor.com/docs/status-badges/ has documentation for how to create an AppVeyor badge; I think you can use this URL for the image badge:

• https://ci.appveyor.com/api/projects/status/github/rasbt/watermark?svg=true

Which looks like this: , but I don't know what the URL to your project is, only you do.

Lack of ISO 8601 date if we use options

Continuing the last thought...

Without options we get a summary of information and an ISO 8601 date.

%watermark

2016-01-31T21:17:31

CPython 3.4.4
IPython 4.0.1

compiler   : GCC 4.4.7 20120313 (Red Hat 4.4.7-1)
system     : Linux
release    : 3.16.0-38-generic
machine    : x86_64
processor  : x86_64
CPU cores  : 8
interpreter: 64bit

If we add extra options, we lose the ISO 8601 date-time (and just have the date):

%watermark -d -m -v -p numpy,matplotlib -g

2016-01-31 

CPython 3.4.4
IPython 4.0.1

numpy 1.10.2
matplotlib 1.5.0

compiler   : GCC 4.4.7 20120313 (Red Hat 4.4.7-1)
system     : Linux
release    : 3.16.0-38-generic
machine    : x86_64
processor  : x86_64
CPU cores  : 8
interpreter: 64bit
Git hash   : bd6a7a30199b5d52248b26ad0b6898a63b44a56a

Could we either have a new flag for an ISO 8601 date, perhaps -i which uses the same new strftime('%Y-%m-%dT%H:%M:%S')?

Add LICENSE to package

Could you please add the license to MANIFEST.in so that it will be included in sdists and other packages? This came up during packaging of watermark in conda-forge.

cc/ @proinsias

Not an issue, just saying good job!

That's not an issue, I just wanted to say thanks and bravo 👏 for this neat and useful Jupyter extension! Well done, it's really well realized, quite useful and straightforward to install and use!
Continue the good work! ✨ !!

import error

Hi,

After successful install with pip install watermark I have an import error with %load_ext watermark:
ImportError: No module named watermark

I can get rid of the error after modifying the __init__.py. Changing this:

from watermark.watermark import *

into

from watermark import *

make it work.

Is there a problem with my install or with the __init__.py file ?

Include information about how Python was installed

Python can be installed in an number of ways. Limiting focus just to Linux, for example:

• Official system package
• On Ubuntu, deadsnakes PPA
• pyenv
• Conda (official default and conda-forge)
• python Docker image

Knowing how Python was installed can be useful at times for tools that have more intrusive integration with the interpreter, e.g. profilers like https://pythonspeed.com/fil/ or https://sciagraph.com.

If I submitted a PR to add this, would you be willing to accept it? I'd probably do Linux only, as a first pass, with a limited number of Linux distributions, because I don't know as much about macOS/Windows install mechanisms.

New feature to check for latest packages

It would be nice to have a --check_latest flag that checks if the latest packages are installed. For example

%watermark --packages numpy
numpy: 1.22.1 (version 1.22.4 is available)

This could be build based on the dependency checker package's is_latest_version feature (https://github.com/muellerzr/dependency_checker)

cli version

I think it would be useful to have a CLI version for this for non-Python kernels and for Python kernels that don't support magics.

For example, in an R notebook, you could write:

system("watermark")

and get the same as %watermark in IPython.

The obvious problem being that it will show the versions of the relevant Python packages, like CPython and IPython, by default, but I think that's OK for a first iteration, and we can just recommend including some flags instead.

What do you think?

Python 3.8's new metadata package

It looks like Python 3.8's new metadata package in the standard lib could be useful for getting package versions more reliably if someone is on Python 3.8.

reference: https://docs.python.org/3/library/importlib.metadata.html

Why not use an ISO-8601 date format?

Hi there. Out of curiosity - how come you're not using an ISO-8601 date format (e.g. 2015-01-02 where this is unambiguously the 2nd of Jan 2015 and cannot be the 1st of Feb)? Would you accept a PR to switch to ISO-8601?

Determine what version of jupyter notebook the code is currently being run in

Many times users don't know if they are running in a Jupyter Notebook, Jupyter Lab, Google Colab, or VSCode notebook. There are many other notebook environments. All with slight incompatibilities that make it difficult to debug for notebook extension maintainers.

I wrote a convenience function for my library that does this. Do you think this would be useful for watermark?
https://github.com/paddymul/buckaroo/blob/main/buckaroo/__init__.py#L50-L67

Fails to capture versioning info for missingno package

This plugin fails to capture the version info for the missingno-package.

Otherwise a really useful little plugin :)

setup.py fails unless all install_requires are already installed

Because of how the version is managed (https://github.com/rasbt/watermark/blob/master/setup.py#L12), setup.py will fail if there are any mandatory dependencies that are not installed. This means that it cannot run setup.py to get the install_requires unless all of the install_requires are already installed!

This is discussed as option 6 of https://packaging.python.org/guides/single-sourcing-package-version/.

This doesn't show up in CI because ipython is installed separately before watermark is installed.

See here for an example: https://travis-ci.org/github/rasbt/watermark/jobs/739884501

strfime missing `t` for custom_dates

Hi Sebastian,

thanks for watermark.py! I wanted ISO formatted dates and to me this looks like a typo (missing "t" in strftime):

%watermark -c "%Y-%m-%d" -v -m -p numpy,scipy,matplotlib,seaborn,pydicom -g

.ipython/extensions/watermark.py in watermark(self, line)
    103                 self.out += 'Last updated: '
    104             if args.custom_time:
--> 105                 self.out += '%s ' %strfime(args.custom_time)
    106             if args.date:
    107                 self.out += '%s ' %strftime('%d/%m/%Y')

NameError: name 'strfime' is not defined

Fail to import watermark to jupyter notebook - anaconda (py36 env)

Hi there!

After successfully installing watermark via pip install watermark to both anaconda environments (py27 and py36), I'm struggling with a strange issue.

While I can easily import it using py27 environment (it works perfectly in jupyter notebook python 2), I can't import it using py36 environment (see screenshot). What could be the reason for it? Thanks!

Watermark fails to recognize all project used libraries

I find Colab pre-installed libraries convenient as well as AWS Sagemaker pre-defined kernels but this convenience becomes very annoying when gathering the requirements.txt file as I end up with many libraries I have not actually used in my project. I know I could create a virtual at the very beginning but I am wondering if there is a way to avoid it.

I have recently discovered watermark which partially solves this issue. Nevertheless, for this solution to be a perfect fit it still has two issues that I will exemplify below and that you can easily reproduce in Colab.

!pip install fastai --upgrade
!pip install voila
!jupyter serverextension enable voila --sys-prefix
!pip install watermark
from fastai.vision.all import *
from fastai.vision.widgets import *
%load_ext watermark
%watermark --iversion  

Neither fastai nor voila appear in the output as I am not running import fastai and loading voila as an extension.

%watermark -p fastai

This would return the correct output for e.g. fastai but I would like to be able to generate automatically without having to manually check for the missing packages.

--iversion does not work on dask

dask's version is stored in dask._version, I imagine some other packages have a similar design - is it possible to account for that?

Would love the option to get the GPU and Nvidia revision info

As getting the GPU and Nvidia revisions can be kind of problematic..
Appreciate this great addition to the toolbox..

Python 2.7 and 3.5 EOL

Python 2.7 and 3.5 are both EOL. Can they be removed from the supported Python versions and the build matrix?

Adding python_requires=">=3.6" to the setup will ensure that only the latest version that supports 2.7 or 3.5 will be installed via pip on those platforms. This means that the old versions will continue to be available but new features will only be available for Python 3.6+

https://packaging.python.org/guides/distributing-packages-using-setuptools/#python-requires

Feature to auto-pull info of all imported packages

I think a new flag, e.g.,

--pull_pkginfo

or so could be useful to list all unique packages used in a notebook and the respective version numbers. Right now, we have

-p      --packages

which is useful to display the version numbers of a list of packages, which can be useful to limit the scope of packages that should be displayed. For example.

%watermark -p numpy,scipy
numpy 1.9.2
scipy 0.15.1

The --pull_pkginfo would preferably be run at the end of an IPython notebook so that a user don't have to worry about specifying packages "manually." It should scan an collect unique imports in the notebook and list all the packages, e.g.,

%watermark --pull_pkginfo
    numpy 1.9.2
    scipy 0.15.1
    ...

Maybe excluding packages from the standard lib since we already have a -v flag to show the Python version number.

v1.7 tag missing

Is it possible to blur background?

Is it possible to blur background like on the foto for videos with empty places in background?

https://i.stack.imgur.com/cZXPp.png

support -iv edge cases?

I am wondering if there is an efficient way to support fetching the versions of Python packages that don't follow the __version__ convention, gurobi's Python API for example?

In [1]: %load_ext watermark                                                                                                      

In [2]: %watermark -w                                                                                                            
watermark 2.0.2

In [3]: import gurobi                                                                                                            

In [4]: import gurobipy                                                                                                          

In [5]: %watermark -iv                                                                                                           


In [6]: gurobi.gurobi.version()                                                                                                  
Out[6]: (9, 0, 0)

In [7]: gurobipy.gurobi.version()                                                                                                
Out[7]: (9, 0, 0)

In [8]: %watermark                                                                                                               
2019-12-25T17:55:30-05:00

CPython 3.7.3
IPython 7.10.2

compiler   : Clang 9.0.0 (tags/RELEASE_900/final)
system     : Darwin
release    : 19.2.0
machine    : x86_64
processor  : i386
CPU cores  : 4
interpreter: 64bit

Or would this have to be (1) a brute-force search through the entire directory; or (2) a manual print() for the version.

information on current notebook filename?

Does watermark include an option to include the notebook filename in the watermark magic output? I could not find such an option in the documentation.

If this is not currently possible, it would be an extremely useful feature to add.

By default a notebook printed out using the _File _PrintPreview option does not print the filename in the header or footer. I was looking at printouts from a couple weeks ago and watermark provides details about the hardware/software environment and date/time printed, but no an answer to the all important question of the name of the notebook, to know what file to return to.

From this stackoverflow discussion there seem to be ways to have the filename printed in a cell but none of the proposed solutions are compact or elegant. So it would be great if watermark could do the work behind the scenes.

Thanks for a great tool that I use daily.

**update: ** the jupyter notebook contributors appear to be discussing a similar issue jupyter/notebook#1000

New release to include LICENSE

When possible, please create a new release to include #21, the fix for #20.
Thanks!

version unknown

Hi! Thanks for developing watermark, it is indeed fundamental to make my workflows reproducible! However, it seems like it cannot read the version of some packages - see attachment. Is there a way to read the version of these packages? Where does watermark get the version from? Thanks!

Error in the date with the options -u -n -t -z

Hello Sebastian,

I noticed that with %watermark -u -n -t -z the date is incorrect, the day takes the same value as the minute. Here is, for comparison, the outputs I have with different options:

---

• Output of %watermark

01/10/2014 14:17:34

CPython 2.7.3
IPython 2.2.0

compiler : GCC 4.2.1 (Apple Inc. build 5666) (dot 3)
system : Darwin
release : 12.5.0
machine : x86_64
processor : i386
CPU cores : 2
interpreter: 64bit

---

• Output of %watermark -d -t

01/10/2014 14:17:34

---

• Output of %watermark -u -n -t -z

Last updated: Wed Oct 17 2014 14:17:34 CEST

Timezone name has no trailing space

Hi

First of all, thank you for this package:

Second, I believe the Timezone name option is missing a trailing space.

%watermark -z -i


E. Australia Standard Time2019-08-14T15:37:07+10:00

This running in an environment as follows:

2019-08-14T15:32:43+10:00

CPython 3.7.1
IPython 7.2.0

compiler   : MSC v.1915 64 bit (AMD64)
system     : Windows
release    : 10
machine    : AMD64
processor  : Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
CPU cores  : 8
interpreter: 64bit

Looking at the code, I find

            if args.timezone:
                self.out += strftime('%Z')

where most other constructed strings have a trailing newline or space.

Happy to supply more information if needed

Don

Adopt NEP 29?

The scientific Python community (including IPython) have adopted the deprecation policy in NEP 29: https://numpy.org/neps/nep-0029-deprecation_policy.html. I think that this project should also do so.

Using the python_requires setting for setuptools will ensure that you still get a compatible version from pip when using on older version of Python. However, new features and bug fixes will only be available for newer versions of Python.

This is an extension of #61.

Cannot retrieve the version information of OpenCV

I can import cv2 package in my jupyter notebook, but watermark cannot find the proper information for this package:

DistributionNotFound: The 'cv2' distribution was not found and is required by the application

Error UsageError on author argument

When executing watermark on RASBT-STAT453 notebook with...
%watermark -a 'Sebastian Raschka' -v -p torch

Get error message...
UsageError: unrecognized arguments: Raschka
and no other watermark info.

Simple work-around is to insert underscore between first/last names.

Using VS Code 1.60.2 notebook with Python 3.8 & Torch 7.26 under Win10. Installed watermark-2.2.0 with conda from conda-forge.

Improve continuous integration tests by checking HTML output

Resource: http://stackoverflow.com/questions/37689308/how-to-validate-installation-of-a-jupyter-nbextension

`-d` doesn't print date, only when combined with `-u`

There is currently an issue that -d alone doesn't print the date. It only works with -u. In my opinion -d should work as a standalone argument.

Supporting nested library imports via `-iv`

Currently, nested imports do not seem to work:

1)

In [1]: from sklearn.model_selection import train_test_split

In [2]: %load_ext watermark

In [3]: %watermark -iv

2)

and only the current scenario is supported (among other variants):

In [4]: import sklearn.model_selection

In [5]: %watermark -iv
sklearn   0.20.2

It would be nice if we could also get 1) to work.

Newline is not taking into account

Generally, programmer may use more than one library packages to be listed.

%watermark -a "author" -d -t -v -m -p numpy,pandas,scipy,sklearn,statsmodels,matplotlib,seaborn,bokeh,xgboost,`\n
h2o,pymc3,lifelines,theano,altair

Any scope of including this functionality?

Watermark fails to identify (some) packages when imported as 'from X import Y'

See the jupyter notebook outputs below:

Watermark doesn't catch classes/functions imported using `from`

Hello,
first, thank you for this package.
I just noticed this behaviour when importing a subclass from Biopython:

from Bio import SeqIO

the watermark magic doesn't show the info of the imported package (Bio). I'm aware that I can solve this doing

import Bio

but i was wondering if it could be implemented given that for many libraries It is customary to use:

from X import Y

Cheers!

Missing space before timezone

From the example:

In [7]:
%watermark -u -n -t -z

Out[7]:
Last updated: Fri Nov 20 2020 15:27:00UTC

Should be a space before "UTC".

Remove extra whitespace between version numbers via --iversions

Remove the whitespaces in --iversion's output so that it's consistent with other output. E.g.,

%watermark -iv -v -w 
dask        0.15.1
CPython 3.6.1
IPython 6.1.0
watermark 1.5.0

should be

%watermark -iv -v -w 
dask 0.15.1
CPython 3.6.1
IPython 6.1.0
watermark 1.5.0

Alternatively, everything could be aligned, e.g., say version numbers starting in the 20th column (I don't any python package is more than 18 characters long ;)

%install_ext is deprecated...

IPython deprecated the %install_ext magic (ipython/ipython#8634). The replacement story is using a package to distribute the extension and then use %load_ext package.extension to load the extension.

Ah, and shamless plug: if you don't want to work with your own package, there is also now https://github.com/ipython-contrib/IPython-extensions watermark would be a great addition there!

Use of pkg_resources for package versions

Is there a reason why _get_packages doesn't use pkg_resources to get the package version numbers, like pip does?

The version attributes inside the imported modules are not guaranteed to exist or to match the version that pip (or conda) understand. pkg_resources is the way to do this. Otherwise you risk one or more of the following recommended approaches not working: https://packaging.python.org/guides/single-sourcing-package-version/.

Allow storing watermark info in metadata?

Watermark looks great for reproducibility.

It would be nice to have an option to (also) store this data in the notebook metadata:

{
"metadata": {
  "watermark": {
    "date": "2015-17-06T15:04:35",
    "CPython": "3.4.3",
    "IPython": "3.1.0",
    "compiler": "GCC 4.2.1 (Apple Inc. build 5577)",
    "system" : "Darwin",
    "release" : "14.3.0",
    "machine": "x86_64",
    "processor" : "i386",
    "CPU cores": "4",
    "interpreter": "64bit"
  }
}
}
}

Maybe some more hierarchy in there as well...

Since the kernel doesn't have any idea what's going on w/r/t notebooks, it would probably have to be done with a display.Javascript:

if as_metadata:
    display(
        Javascript("""IPython.notebook.metadata.watermark = {}""")
            .format(json.dumps(the_data)
    )

Happy to help with a PR, if you would think there is a place for this!

Is it possible to add the git branch to the report?

Currently we can add the git hash, is it possible to show the git branch as well? This is more interpretable than the hash.

git rev-parse --abbrev-ref HEAD seems to do it at the command line as noted (with other options) here: https://stackoverflow.com/questions/6245570/how-to-get-the-current-branch-name-in-git

Sidenote - I've introduced watermark to many colleagues, I tend to make sure it is adopted by projects in each organisation I work with, many thanks for supporting it!

Wrong package versions within virtualenv

I'm not sure how widespread this issue is, or if it's something particular to my setup, but when I use watermark to report package information (using -p) within a jupyter notebook that is running in a virtualenv, version information about system level packages are reported rather than packages installed within my environment. If a package is installed in my virtualenv and not installed at the system level, then I end up getting DistributionNotFound exceptions when I try to report the version number using watermark.

The problem stems from the call to pkg_resources.get_distribution. If I use get_distribution directly from within my notebook to report the version of e.g. numpy it shows my system level numpy info (v 1.11.0) rather than info about numpy installed within my virtualenv (v 1.11.1). For example:

pkg_resources.get_distribution('numpy')
numpy 1.11.0 (/usr/local/lib/python2.7/site-packages)

Similarly

%watermark -p numpy
numpy 1.11.0

But when I check the version of what is imported:

import numpy as np
np.__version__
'1.11.1'

If I use pkg_resources from a python console or script running within a virtualenv, it reports the proper information about packages. But if I use it from an ipython console or jupyter notebook it reports system level information.

If this is a widespread issue, should watermark find another more robust solution to getting package information other than using pkg_resources?