suizokukan/katal

select a bunch of files, create a catalogue of the selected files without duplicates and tag them

katal's Issues

'doubloon' > 'duplicate'

store db, log... in a katalsys folder

'sieve' > 'filter'

-si / --sourceinformations : show_infos_about_source_path()

on windows systems, fullnames > 255 characters crash the program

improve test_is_ntfs_prefix_mandatory()

///////////////////////////////////////////////////////////////////////////////

def test_is_ntfs_prefix_mandatory(_path):
"""
test_is_ntfs_prefix_mandatory
________________________________________________________________________

    Return True if the _path is a path in a systemfile requiring the NTFS
    prefix for long filenames.
    ________________________________________________________________________

    PARAMETER
        _path : (str)the path to be tested

    RETURNED VALUE
        a boolean
"""
longpath = os.path.join(_path, "a"*250, "b"*250)
res = False

try:
    os.mkdir(longpath)
except IOError:
    res = True

if res is False:
    os.rmdir(longpath)

return res

rename old log files so that they can be easily deleted

NAME > oldlogfile_NAME

-1 for missing config file, -2 for ill-formed config file

    o  sys.exit(-1) is called if the config file is ill-formed or missing.

improve keywords used in cfg file to rename the files

INTTIMESTAMP_SIZE__DATABASE_INDEX.SOURCE_EXTENSION2 juste non, c'est imbitable. Je suis sur Xubuntu et les variables dans Thunar (exemples au pif mais plus claires que les tiennes) c'est :

%f : Le chemin vers le premier fichier sélectionné
%F : Les chemins vers tous les fichiers sélectionnés
%d : Répertoire contenant le fichier qui est passé dans %f
%D : Répertoires contenant les fichiers qui sont passés dans %F
%n : Le premier nom de fichier sélectionné (sans le chemin)
%N : Les noms des fichiers sélectionnés (sans les chemins)

set apart global constant variables and global variables

tests/data1 : c.5 / C.5 : problem with OSX !

--addsizeastag

in action__target_kill(), return if no database

--select/--add : error if a file already exist

avoid name collision !

remove > katal_trash directory

new option : remove unexistent files from the database

after dropping some files out of the target dir, let's clean the database by removing non existent files

files names in db/target path have to be unique

• in database (key ?)
• in target path (check when adding a file)

remove verbosity from config file

--whatabout

if the argument is a file INSIDE the target directory, let's display some informations about it.
if the argument is a path OUTSIDE the target directory, let's walk inside and display all the required informations.

with the filters > with the filter(s)

add an 'original informations' field to the database

(date) path + name + extension

reorder by datetime

fix args' epilog

pip

regex in filters : case sensitive ?

tasks

improved messages in fill_select__checks()

duplicate : test all bytes if hash are identical

cf Zyphos : http://linuxfr.org/news/katal-catalogue-de-fichiers

--findtag

use namedtuple to read/write in the database

no more 0.00Go -> 123Kb

normpath() : .. ->

do not completely overwrite log file

messages.log > messages.log.201506279032

in default cfg file, set the target name to DATE__xxx

--rmnotag

--rmnotag : remove from the database and from the target path all files without tag.

sort the extensions

see --infos options

show how many extensions have been found

cut log file into several log files if the size bounds up over an upper limit

in fill_select(), add a percentage of the work done

naming : constant variables

add CONSTANT__ at the beginning of all constant variables' names

config file stored in *home/.katal

action__add() : improve messages

In action__add(), add a message before writing to the db ("all files have been copied, updating the database...") and after ("database updated").

--mode={copy, move, mirror}

instead of --move and --mirroronly

fill_README

Fill README.md from the xxx_ constants defined in katal.py

size : 1Mb, 1Mo ~ 1000000

feature request (maybe doc update?)

I have a feature request (or a doc update request, if it's already possible):

I need an option to create the files database without copying/moving any (selected) files from srcdir to targetdir.

I'll explain myself:

• I (some of my users) have huge collections of files (more than 100TB, more than billions directories and files).
  One of our workflow take hdf5 files (mostly stars simulations) to generate a movie file, then a contact sheet and a thumbnail.
  This highly automated workflow, fail from time to time for reasons. And we have... difficulties to find which movie/contact or thumbnail have failed to generate (due to the very large amount of dir/files).
• Other example:
  Some users deals with genetic data, one sequenced genome has to be stored in various formats. Finding duplicates, failed transferts, failed conversions... is a task by itself.

As a backend task, I may have to search duplicates (but with another filename), purge old backups, etc.

So, having a regulary updated database at the root of a subdir hierarchy, which could be questioned in various way, could be very usefull.
Katal can be that tool (and easely be integrated in our workflows, as an error detector, or final report complement...) if it doesn't have to copy the whole file/dir hierarchies.

Bests

Katal.py's header

move the following lines :
# see https://pypi.python.org/pypi?%3Aaction=list_classifiers
[...]

see https://pypi.python.org/pypi?%3Aaction=list_classifiers

at the beginning of the ******* lines .

date : cdate ? mdate ?

addtag %%s;%%d

--findtag + --exportto