e3v3a/pip-date

A simple CLI tool to show the installation/modification times of all your pip packages

pip-date - Date your pip packages!

A simple Python3 CLI tool to show the installation or modification times of all your pip packages.

STATUS:	Version	Date	Maintained?
Working	1.0.5	2022-01-23	YES

---

Example Output:

Using pip-search <RegEx>:

---

Q: What does pip-date do?

The primary use is for finding the time when a certain pip package was last modified or installed. It is basically using one or more of: atime, ctime and mtime from the file status (stat) info. This is essentially equivalent to using the *nix stat command, but is handled differently on Windows. (See below for further details.)

Using this information, it can show you and highlight packages that may have been corrupted or outdated. I also has some functionality of checking packages for outdated and deprecated installation methods.

But it can do more. Some features are:

• Highlight packages with inconsistent file modification times (mTime).
• Highlight package versions which are not conforming to the PEP-0440 standard.
• Highlight packages installed with an unusual package distribution priority given by: [chk, src, bin, egg, dev]
• Highlight setuptools dependency packages for easy review
• Show package installation type: with pip/wheel as wheel, and source as sdist (FIX!)
• Show package installation location: usr for --user and sys for global installations.
• Show correct file modification time, depending on OS/FS architecture (mtime vs ctime)

Q: What does it not do?

• Does not install packages
• Does not show dependencies
• Does not (yet) show packages in a virtualenv or pipenv envrionment (ToDo)
• Does not check package consistency
• Does not show the very first time you installed a package, if it has been updated since.
  (Althought there are left-over artifacts that may show otherwise, we don't look for these.)

Q: Why is this needed?

It probably isn't, BUT...

Because python packages often rely on a large number of sub-dependencies, it is very easy to accidentally overwrite some required dependency of one package with a different version needed by another package. You will never know about it, until it breaks something. One common scenario causing package corruption is that you have installed some package XXX using pip, but then get an OS update and install the update using you OS packagemanagement system, like apt-get install XXX, which would probably overwrite the globally installed pip package. This is especially true for beginners of python, who has not yet learned how to use a virtual environment, and installing evything in either the global system (default) or user (--user) environments. This may also occur when installing packages from sources, or when you have to run some other non-pip installers like setup.py, make install or like, and you don't really know what it is going to do.

Q: What else is included?

• A script called pip-describe, that will do what pip doesn't, which is to show the full-text long_description for any PyPI package (including those not already installed).

• A script called pipbyday, that will print a simple table with:
  mTime/aTime + package-name + package-version, sorted by time.

• A script called pyfileinfo, that will show detailed file and date information for a given file using python's os.stat info.

• A script called pyOSinfo, that will print a number of os, system and platform variables, as seen by your Python interpreter.

• NEW A script called (hold your breath!) pip-search, that will download and search all of the PyPi package database for packages matching your search criteria. However, for colored and sexy package searches, I recommend using pip_search. Also, for cool powershell wrapper of pip-search, read this.

Q: Will I continue to support this tool?

Sure, if it is broken, but I will not spend any more time for new features. So if you would like to add something just send me a PR, or at the very least, a detailed code snippet of what I need to implement.

---

Dependencies

• requests - used by pip-describe to get PyPI info
• lxml - used by pip-search to parse html from PyPI

and what you already have:

• Python3
• pip

Installation

There is nothing to install really. Just download the pip-date.py file and make sure to place it in your PATH.

For pip installation:

pip install pip-date

For single file installation:

cd /usr/bin/
wget https://github.com/E3V3A/pip-date/raw/master/pip-date
chmod 755 pip-date

For developer installation:

git clone https://github.com/E3V3A/pip-date.git
cd pip-date
pip install pip-date --user

How to Run

pip-date      # When it's in your PATH
./pip-date    # When it's not in your PATH

---

References:

Time Stamps

It's quite amusing to see how different OS's and File System's (FS) are handling file time stamps. In the Linux world the available time stamps are called atime, ctime and mtime, where they are generally available through the stat command. However, Windows systems doesn't have this commmand because NTFS is using a different way to keep track of time-stamps, so older python versions would (and Windows OS) would report the wrong values for these. But have since been fixed.

Then we use: os.path.getctime(pkg_loc) to get the file time stamp.

For all the gory details, see: here, here and here.

---

Glossary:

• bdist - "Built Distribution":
  A Distribution format containing files and metadata that only need to be moved to the correct location on the target system, to be installed. Wheel is such a format, whereas distutil’s Source Distribution is not, in that it requires a build step before it can be installed. (A "Binary Distribution" is also a bdist, but with additional compiled extensions.)

• sdist - "Source Distribution":
  A distribution format (usually generated using python setup.py sdist) that provides metadata and the essential source files needed for installing by a tool like pip, or for generating a Built Distribution.

• egg - [deprecated]:
  The older Built Distribution format introduced by setuptools, which is being replaced by wheel.

• wheel - "":
  A Built Distribution format introduced by PEP-0427, which is intended to replace the "egg" format. A wheel (bdist_wheel) is a ZIP-format archive with a specially formatted file name and using the .whl extension. Normally, you need one wheel file for each operating system and architechture. And that list can get long for big projects, like numpy.

---

Recommeded or Similar Tools:

• pip-check - Check you pip package update status with nice ANSI colored CLI
• pip-chill - Lists only the dependencies (or not) of installed packages
• pip_search - Victor's amazing pip search replacement
• venvlink - Using virtual environments outside of the project folder, like a boss!

---

Bugs and Warnings

None

ToDo / Help Needed

See issues marked ToDo.

Contribution

Feel free to post issues and PR's related to this tool.
Feel free to fork, break, fix and contribute. Enjoy!

Additional Badges

---

License

A license to 💖!

_{I use GPLv3 because sharing code modifications is more beneficial for the world.}

pip-date's Issues

Linux Mint install results in many "Skipping Bad Path"

Running pip-date in Vanilla Linux Mint 19.1 results in most packages not being reported due to:
Skipping Bad Path of:.

The path in question are:
/usr/lib/python3/dist-package/*

Question is if also:

• /usr/lib/python3.6/site-packages/ is should be added as ok.
• ./.local/lib/python3.6/site-packages/

More testing needed.

Fix POSIX terminal check

Fix POSIX terminal check in pip-date.

New Windows Terminal, WSL or pwsh now has VT enabled by default and most can handle ANSI Coloring escape sequences. The current check seem to break this in is_posix().

Add more information from README to PyPi page

See https://pypi.org/project/pip-date/ - it is too short, would be nice to have some basic introduction there, like in README, and a couple screenshots.

Fix README badges

Fix the bottom page README badges.

Column alignment problem

Alignment problem

ZODB                   2019-06-21  05:36:16                          5.5.1                              sdist   sys
zodbpickle             2019-06-20  19:02:04                          1.0.3                              sdist   sys
zope.cachedescriptors   2019-06-21  06:10:13   2019-06-13  01:54:39   4.3.1                              sdist   sys
zope.component         2019-06-21  06:10:13   2019-06-13  01:54:39   4.5                                sdist   sys
zope.configuration     2019-06-21  06:10:13   2019-06-13  01:54:39   4.3.1                              sdist   sys

more pronounced

sphinx-issues          2019-12-15  21:35:58                          1.2.0                              sdist   sys
sphinx-rtd-theme       2019-06-21  05:44:54                          0.4.1                              sdist   sys
sphinx-testing         2019-06-21  05:48:18                          1.0.1                              sdist   sys
sphinxcontrib-actdiag   2019-06-21  06:18:52   2019-06-13  01:54:05   0.8.5                              sdist   sys
sphinxcontrib-apidoc   2019-06-21  06:18:52   2019-06-13  01:54:05   0.3.0                              sdist   sys
sphinxcontrib-asyncio   2019-06-21  06:18:52   2019-06-13  01:54:05   0.2.0                              sdist   sys
sphinxcontrib-autoprogram   2019-06-21  06:18:52   2019-06-13  01:54:05   0.1.5                              sdist   sys
sphinxcontrib-blockdiag   2019-06-21  06:18:52   2019-06-13  01:54:05   1.5.5                              sdist   sys
sphinxcontrib-documentedlist   2019-06-21  06:18:52   2019-06-13  01:54:05   0.6                                sdist   sys
sphinxcontrib-github-alt   2019-12-15  21:35:58                          1.1                                sdist   sys
sphinxcontrib-httpdomain   2019-06-21  06:18:52   2019-06-13  01:54:05   1.7.0                              sdist   sys
sphinxcontrib-issuetracker   2019-06-21  06:18:52   2019-06-13  01:54:05   0.11                               sdist   sys
sphinxcontrib-newsfeed   2019-06-21  06:18:52   2019-06-13  01:54:05   0.1.4                              sdist   sys
sphinxcontrib-pecanwsme   2019-06-21  06:18:52   2019-06-13  01:54:05   0.10.0                             sdist   sys
sphinxcontrib-programoutput   2019-06-21  06:18:52   2019-06-13  01:54:05   0.14                               sdist   sys
sphinxcontrib-seqdiag   2019-06-21  06:18:52   2019-06-13  01:54:05   0.8.5                              sdist   sys
sphinxcontrib-trio     2019-06-21  05:48:11                          1.0.2                              sdist   sys
sphinxcontrib-websupport   2019-06-21  06:18:52   2019-06-13  01:54:05   1.1.0                              sdist   sys
spotipy                2019-06-20  19:04:08                          2.4.4                              sdist   sys
spyder                 2019-06-09  17:57:47                          3.3.4                              sdist   sys

It would be beneficial to determine the console width and use all the space available.

Also grouping packages with common prefixes could avoid repetition of long package name prefixes.

Uset HOME directory to Load/Save tmp_piplist_*.txt

pip-search is saving the tmp_piplist_<date>.txt file in the current working directory.
Make sure to save somewhere in $HOME.

Add "summary" option to pip-describe

An improvement would be to add a few new options to pip-describe:

• "summary"
• (last) "version"
• (pypi) "upload_time"

As taken from JSON at:
https://pypi.org/pypi/pip-date/json

Replace extraordinary utf-16 code for "block" with a Consolas OEM supported one

The currently used escape code for the Unidcode "5/8th box" characted belong to higher Unicode and are unlikely part of most standard windows Console fonts (as included in Console), and this not displaying correctly, Change to use the block found in the (must have) IBM_OEM font.

Need a better way to test if output terminal is not color capable

We were doing a too simple check to see if the output terminal was able to produce the ANSI color codes as used. However, for Windows based python installations, this seem rather daunting.

This did not work well and will be commented out.

if sys.platform == "win32" and "xterm" not in os.getenv("TERM"):    # [xterm, xterm-color, xterm-256color]
    return text

pip-date script fails under Windows

I tested the latest release available on PyPI.
The pipbyday script is working fine.

Debug log:

C:\>py -3 "c:\python3\scripts\pip-date"

Traceback (most recent call last):
  File "c:\python3\scripts\pip-date", line 194, in <module>
    pkg_mtime = yellow(pkg_mtime)
  File "c:\python3\scripts\pip-date", line 50, in yellow
    def yellow(text): return color(text, 33)            #
  File "c:\python3\scripts\pip-date", line 40, in color
    if sys.platform == "win32" and "xterm" not in os.getenv("TERM"):    # [xterm, xterm-color, xterm-256color]
TypeError: argument of type 'NoneType' is not iterable

pip-date improvements ToDo list

Some internal comments suggests some improvements to the pip-date script.
It is probably better to add that info here.

ToDo:

• better RegEx for "pip"
• add flake8 QA ignore comments
• fix rounding of floats in [a/c/m]Time
• Add CLI options:
• '-d' : Enable extra debug info
• '-e' : Show env column to display virtualenv name
• '-f' : Force to use opposite (to detected) ctime method for FS
• '-n' : Disable colors
• '-h' : THIS help/usage message
• '-v' : THIS program version
• '-t ' : To highlight packages installed <days> ago

not working with pyenv

Hi,

It doesn't seem to work in a pyenv/virtualenv environment

Environment

• pip version: pip 19.2.1 from /home/xxxxxx/.local/share/virtualenvs/bo-JcXeAW25/lib/python3.6/site-packages/pip (python 3.6)
• Python version: Python 3.6.8
• OS: Ubuntu 19.04
• pyenv : pyenv 1.2.12-2-geb68ec94

Description

$ pip install pip-date
$ pip-date


Using mTime for Linux FS

Traceback (most recent call last):
  File "/home/bdupuis/.local/share/virtualenvs/bo-JcXeAW25/bin/pip-date", line 264, in <module>
    site_loc = site.getsitepackages()           # [...]
AttributeError: module 'site' has no attribute 'getsitepackages'

The package installs its executables incorrectly

Environment
Please provide some information about your computer environment:

• OS:

  $ uname -a
  Linux NTT-44Q83J3 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  $ lsb_release -a
  No LSB modules are available.
  Distributor ID: Ubuntu
  Description:    Ubuntu 22.04.3 LTS
  Release:        22.04
  Codename:       jammy
  

• Python version:

  $ python -VV && python -c "import os; print('\n'.join([os.name, os.sys.platform]));"
  Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
  posix
  linux
  

• pip version:

  $ pip -V
  pip 23.3.1 from /home/vbrozik/dev/asa-cp-conv/.venv/lib/python3.10/site-packages/pip (python 3.10)
  

• Terminal/Shell: (powershell, pwsh, Windows Terminal, WSL etc.)
  Windows Terminal, bash inside WSL2

Description

After the package is installed its executables are not available to be executed directly from CLI. The usual location $VIRTUAL_ENV/bin/ does not contain executable wrappers used to start the package's scripts.

Expected behavior

After the installation the directory $VIRTUAL_ENV/bin/ should contain a wrapper for every program (like pip-date) with the executable right set and without any suffixes.

Actual Behaviour:

The directory $VIRTUAL_ENV/bin/ directly contains the package's Python scripts like pip-date.py without the executable access right set and with the .py suffix instead of usual wrappers used to start the program the normal way.

How to Reproduce

1. Create a venv, activate it and update its default packages.
2. Install pip-date using pip.
3. Check that pip-date cannot be executed.
4. Check that the bin directory inside the venv does not contain wrappers for the package's scripts.

Output

~$ mkdir -p tmp/test_pip-date
~$ cd tmp/test_pip-date/
~/tmp/test_pip-date$ python3 -m venv venv
~/tmp/test_pip-date$ . venv/bin/activate
(venv) ~/tmp/test_pip-date$ pip install -U pip setuptools
Requirement already satisfied: pip in ./venv/lib/python3.10/site-packages (22.0.2)
...
Successfully installed pip-23.3.1 setuptools-69.0.2
(venv) ~/tmp/test_pip-date$ pip list
Package    Version
---------- -------
pip        23.3.1
setuptools 69.0.2

(venv) ~/tmp/test_pip-date$ pip install pip-date
Collecting pip-date
  Using cached pip_date-1.0.5-py3-none-any.whl (32 kB)
Collecting requests (from pip-date)
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting lxml (from pip-date)
  Using cached lxml-4.9.3-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting charset-normalizer<4,>=2 (from requests->pip-date)
  Using cached charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests->pip-date)
  Using cached idna-3.6-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests->pip-date)
  Using cached urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests->pip-date)
  Using cached certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)
Using cached lxml-4.9.3-cp310-cp310-manylinux_2_28_x86_64.whl (7.9 MB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached certifi-2023.11.17-py3-none-any.whl (162 kB)
Using cached charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
Using cached idna-3.6-py3-none-any.whl (61 kB)
Using cached urllib3-2.1.0-py3-none-any.whl (104 kB)
Installing collected packages: urllib3, lxml, idna, charset-normalizer, certifi, requests, pip-date
Successfully installed certifi-2023.11.17 charset-normalizer-3.3.2 idna-3.6 lxml-4.9.3 pip-date-1.0.5 requests-2.31.0 urllib3-2.1.0

(venv) ~/tmp/test_pip-date$ echo $PATH
/home/vbrozik/tmp/test_pip-date/venv/bin:/home/vbrozik/.local/bin:/usr/local/sbin:...

(venv) ~/tmp/test_pip-date$ which pip-date
(venv) ~/tmp/test_pip-date$ echo $?
1
(venv) ~/tmp/test_pip-date$ pip-date
Command 'pip-date' not found, did you mean:
  command 'sip-date' from deb sofia-sip-bin (1.12.11+20110422.1-2.1+deb10u3ubuntu0.22.04.2)
Try: sudo apt install <deb name>

(venv) ~/tmp/test_pip-date$ ls -l /home/vbrozik/tmp/test_pip-date/venv/bin
total 100
-rw-r--r-- 1 vbrozik vbrozik  2003 Dec 12 10:02 activate
-rw-r--r-- 1 vbrozik vbrozik   929 Dec 12 10:02 activate.csh
-rw-r--r-- 1 vbrozik vbrozik  2209 Dec 12 10:02 activate.fish
-rw-r--r-- 1 vbrozik vbrozik  9033 Dec 12 10:02 Activate.ps1
-rwxr-xr-x 1 vbrozik vbrozik   265 Dec 12 10:03 normalizer
-rwxr-xr-x 1 vbrozik vbrozik   253 Dec 12 10:03 pip
-rwxr-xr-x 1 vbrozik vbrozik   253 Dec 12 10:03 pip3
-rwxr-xr-x 1 vbrozik vbrozik   253 Dec 12 10:03 pip3.10
-rwxr-xr-x 1 vbrozik vbrozik   253 Dec 12 10:03 pip3.11
-rw-r--r-- 1 vbrozik vbrozik  2362 Dec 12 10:03 pipbyday.py
-rw-r--r-- 1 vbrozik vbrozik 20808 Dec 12 10:03 pip-date.py
-rw-r--r-- 1 vbrozik vbrozik  2029 Dec 12 10:03 pip-describe.py
-rw-r--r-- 1 vbrozik vbrozik  6622 Dec 12 10:03 pip-search.py
drwxr-xr-x 2 vbrozik vbrozik  4096 Dec 12 10:03 __pycache__
-rw-r--r-- 1 vbrozik vbrozik  4866 Dec 12 10:03 pyfileinfo.py
-rw-r--r-- 1 vbrozik vbrozik  3432 Dec 12 10:03 pyOSinfo.py
lrwxrwxrwx 1 vbrozik vbrozik     7 Dec 12 10:02 python -> python3
lrwxrwxrwx 1 vbrozik vbrozik    16 Dec 12 10:02 python3 -> /usr/bin/python3
lrwxrwxrwx 1 vbrozik vbrozik     7 Dec 12 10:02 python3.10 -> python3

pyfileinfo crash on binary files

# pyfileinfo kali-linux-2018.4-i386.iso

Traceback (most recent call last):
  File "/usr/bin/pyfileinfo", line 93, in <module>
    for lines in fhand:
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 4: invalid start byte

test with:

# echo -e "\xaa jojo \xf20\xf30\xf4" > del.me
# pyfileinfo del.me

ToDo / Help Needed

ToDo / Help Needed

• improve the time stamp (TS) heuristic to use more reliable files to search for
• improve the package type heuristic to show how a package was installed (wheel or sdist)
• add an Environment column, to show in what virtual environment a package was installed in.
• improve highlighting of special package dependencies, such as these.
  (e.g. Current implementation would for 'pip' also highlight any package with "pip" in it.)
• improve the color terminal check in #3
• Testing on MacOS
• Add screenshots and text in PyPi (#9)
• Add Python 3.8 compatibility in setup.py
• Merge PR #11
• Fix installation paths for windows pythons (#12) if possible

Add sys.implementation to pyOSinfo

Add sys.implementation array to pyOSinfo...

# python3 -c "import os, sys; print(sys.implementation);"

namespace(_multiarch='x86_64-cygwin', cache_tag='cpython-38', hexversion=50856116, name='cpython', version=sys.version_info(major=3, minor=8, micro=0, releaselevel='beta', serial=4))

Add new "pip-search" command

As it is, pip search <dog> just sucks.

Because it doesn't:

• sort the result list
• column align the "version" string
• show last upload date
• show last (repo) commit time, if available.
• truncate "summary" to fit screen

We add a new pip-search command as an artificial quasi-wrapper to "pip search".

❗ NO ETA... as I'm tired of 2020

"pip install pip-date" does not install Windows compatible binary

Describe the bug
Running pip install pip-date on Windows installs Linux binary instead

To Reproduce
Open Windows command prompt, run pip install pip-date then try to run pip-date.

Expected behavior
The pip-date command to execute the binary in /Python37/Scripts

Screenshots

In install successfully, then when I try to run:

pip-date is not recognized as a command. But the file is here:

But is not an .exe or an Windows compatible binary.

Desktop (please complete the following information):

• OS: Windows 10 Education 1903
• Browser: Firefox Quantum 69.0.2 (64bit)

Add *.py extension to all scripts to simplify Windows compatibility

Windows still doesn't like files without extensions, and hacking around it with (windows python) shebangs is tricky and not easy for most. Adding wrappers to each script is also too much work.

I will add the extensions to all files.

remove full path prefix to pip-describe exec in help func

Currently pip-descibe show:
/usr/local/bin/pip-describe

Remove path prefix.

pip-search without argument give error

Add error handling for no argument given to pip-search

Make sure to remove ALL scripts when uninstalling

Apparently from comments in issue #2, when uninstalling it leaves scripts behind.
However, apparently, this is a known pip/setuptools bug!

Remove also scripts when uninstalling.