camdencheek/fre

Command line frecency tracking

FREcency tracking (fre)

fre is a CLI tool for tracking your most-used directories and files. Though inspired by tools like autojump or the z plugin for zsh, it takes a slightly different approach to tracking and providing usage data. The primary difference is fre does not support jumping. Instead, it just keeps track of and provides sorting methods for directories, which can then be filtered by another application like fzf, which does a much better job of filtering than something I can write. Additionally, it uses an algorithm in which the weights of each directory decay exponentially, so more recently used directories are ranked more highly in a smooth manner.

Usage

fre is primarily designed to interface with fzf. For general usage, a user will create a shell hook that adds a directory every time the current directory is changed. This will start to build your profile of most-used directories. Then, fre can be used as a source for fzf. I personally use the fzf-provided control-T bindings, modified to use fre as input. Some examples are below.

Basic usage

# Print directories, sorted by frecency, then pipe to fzf
fre --sorted | fzf --no-sort

# Print directories and their associated frecency, sorted by frecency
fre --stat

# Log a visit to a directory
fre --add /home/user/new_dir

# Decrease weight of a directory by 10 visits
fre --decrease 10 /home/user/too_high_dir

# Print directories and the time since they were last visited in hours
fre --stat --sort_method recent

# Print directories and the number of times they've been visited
fre --stat --sort_method frequent

# Purge directories that no longer exist
fre --sorted | while read dir ; do if [ ! -d "$dir" ] ; then fre --delete "$dir";  fi ; done

Installation

From source: git clone https://github.com/camdencheek/fre.git && cargo install --path ./fre

From crate: cargo install fre

Arch linux: yay -S fre

macOS: brew install camdencheek/brew/fre

For integration with fzf CTRL-T, define the following environment variables

export FZF_CTRL_T_COMMAND='command fre --sorted'
export FZF_CTRL_T_OPTS='--tiebreak=index'

To preferentially use results from fre, but fall back to other results, we can use cat to combine results before sending them to fzf. My favorite alternate source is fd (link), but the more common find can also be used. The following options first use fre results, then use all the subdirectories of the current directory, then use every subdirectory in your home directory. This is what I personally use.

export FZF_CTRL_T_COMMAND='command cat <(fre --sorted) <(fd -t d) <(fd -t d . ~)'
export FZF_CTRL_T_OPTS='--tiebreak=index'

Shell integration

Don't see your shell here? feel free to open a PR to add it!

zsh

(credit to autojump)

fre_chpwd() {
  fre --add "$(pwd)"
}
typeset -gaU chpwd_functions
chpwd_functions+=fre_chpwd

bash

(credit to autojump)

In your ~/.profile:

PROMPT_COMMAND="${PROMPT_COMMAND:+$(echo "${PROMPT_COMMAND}" | awk '{gsub(/; *$/,"")}2') ; }"'fre --add "$(pwd)"'

Comparison to existing solutions

The three projects I'm familiar with that are closest in function to this are autojump, the z shell plugin, and the d portion (and maybe the f in the future) of fasd.

The primary difference from the rest of these is its reliance on a tool like fzf to provide any solid directory jumping functionality. This was an intentional choice, sticking to the Unix philosophy of "do one thing, and do it well".

The other major change from these pieces of software is the algorithm used to rank directories. autojump uses the following formula:

def add_path(data, path, weight=10):
    # ...
    data[path] = sqrt((data.get(path, 0) ** 2) + (weight ** 2))
    # ...

Looking at it closely, it seems to just be calculating the hypotenuse of a triangle where one side is the length of the previous weight and the other is the length of the weight being added. This does not take into account time passed since access at all, which is not ideal since I would rather not have directories from years ago ranked highly.

fasd and z both use the same frecency function that looks something like this:

function frecent(rank, time) {
    dx = t-time
    if( dx < 3600 ) return rank*4
    if( dx < 86400 ) return rank*2
    if( dx < 604800 ) return rank/2
    return rank/4
}

This works fine until you re-visit an old directory. Then, suddenly, dx is small again and all the old visits are re-weighted to rank*4, causing it to jump up in the sorted output. This is not really ideal. I want to be able to re-visit an old directory once without messing up my directory ranking.

fre uses a frecency algorithm where the weight of a directory visit decays over time. Given a list of visit times (bold x), the frecency of the directory would look something like this (using lambda as the half life and "now" as the current time at calculation):

With a little bit of mathemagics, we don't actually have to store the vector of access times. We can compress everything down into one number as long as we're okay not being able to dynamically change the half life.

This algorithm provides a much more intuitive implementation of frecency that tends to come up with results that more closely match those we would naturally expect.

Support

I use this regularly on MacOS and Linux. I wrote it to be usable on Windows as well, but I don't run any tests for it. Caveat emptor.

Stability

I've been using this for over a year with no changes now, and it does everything I need it to do. I'm happy to add features or accept changes if this is not the case for you.

About the algorithm

The algorithm used combines the concepts of frequency and recency into a single, sortable statistic called "frecency". To my knowledge, this term was first coined by Mozilla to describe their URL suggestions algorithm. In fact, Mozilla already came up with nearly this exact algorithm and considered using it to replace Firefox's frecency algorithm. The algorithm is also very similar to the cache replacement problem, and a more formal treatment of the math behind it can be found in this IEEE article (sorry for the paywall).

fre's Issues

Use XDG state home rather than the data home.

Currently, fre stores its default store (fre.json) in ${XDG_DATA_HOME}/fre (usually resolving to ~/.local/share/fre/fre.json).

However, the XDG Base Directory Specification suggests:

The $XDG_STATE_HOME contains state data that should persist between (application) restarts, but that is not important or portable enough to the user that it should be stored in $XDG_DATA_HOME. It may contain:

• actions history (logs, history, recently used files, …)
• current state of the application that can be reused on a restart (view, layout, open files, undo history, …)

Would it make sense to use ${XDG_STATE_HOME}/fre instead?

The reason I'm considering this question is that currently the --store option to fre specifies a path to an alternate store.
My initial assumption when trying this was that just a name was required, and the storage location would be along-side the default store (e.g. --store=foo would resolve to ~/.local/share/fre/foo.json, rather than a file called foo in the current directory), so another possible improvement might be that if the value provided to --store is not an absolute path, it is assumed that the storage location should be in the same directory as the default store (wherever that is most appropriate) with --store=fre being synonymous with the default store.

I'm happy to make a PR if either of these (standard location or "named" stores) would be useful.

Remove deleted files / directories

How can I easily remove deleted files / directories from the output?

Ideally I could run a command to remove all of these, or only entries that match a glob or regex.

Publishing 0.2.0

cargo install still rocking that 0.1.0. Just opening this in case you forgot to publish it manually; If there's an actual reason to withhold master from crates.io I then apologize for inconvenience and this should be closed.

fre --stat: allow to customize the number of digits shown

I use fre to sort my file bookmarks. It is a really useful tool for me.
When I have not visited some files for a longer time period the shown score is 0.000
It would be nice to allow showing more than 3 digits. That way 0.000 would be e.g. shown as 0.00004
This could be done by e.g adding the option --stat-digits 5 (defaulting to the current value 3)

Investigate moving to stable

When the project started, Rust 2018 was nightly only. Now, there shouldn't be any nightly-only features and the project should probably be switched over to stable.

Plans on generalizing to arbitrary input

Hey :)
I really like the idea of fre and this frecency algorithm.

I would like to use the same algorithm for other stuff as well, not just directory and file tracking. In particular, at the moment I'm using raw fzf for my command run script and would like to include your frecency algorithm there as well.

Are there any plans to generalize fre to arbitrary input? Or do you know of a tool that already does this?

Thank you very much :)

Cargo.lock out of date

Cargo.lock in out of date in the 0.4.0 release. This is preventing me from updating the package I maintain for NixOS.

There's already a crate named FE

This crate cannot be cargo installed because it conflicts with an existing crate.

fre results not preferred with FZF

Apologies for posting here, as I'm quite certain this is more of an fzf question, but your default fre+fzf setup seems to be exactly what I'm looking for, so thought you might have some insight.

One bit I'm not clear on, is why in this scenario:

/Users/timkelty/Dev/cloud-extension-yii2 doesn't bubble to the top once I start searching yii2.
Before any query, the results are listed just as fre has them, but once I type a query, this folder gets unexpectedly pushed pretty far down, even though it should have preferential weight from fre.

fre for files : how?

Hi,

The intro in the docs says "for tracking your most-used directories and files" - everything else in the docs refers to directories. How to use it for files?

What I'd like to achieve is to replace fasd, so when I type less ,2022, I get a completion list of all files and directories I've recently accessed that have 2022 somewhere in the name or path, frecency sorted.

Typo?

📝 Hey, did you mean "frequency" instead of frecency?

fre appends $PWD to arguments on --add

Problem: fre saves input as absolute path, which conflicts with using it for non-path things like commands.

#!/bin/dash

set -o errexit
set -o nounset

repls() {
    echo "rlwrap sbcl"
    echo "rlwrap dash"
    echo "bash"
    echo "clisp"
    echo "guile"
    echo "iex"
    echo "python"
}

# Default store location is ~/.local/share/fre/
store_name=$(echo "cmd:$0" | md5sum | awk '{print $1}')

fzf_result=$({
    fre --sorted --store_name "$store_name"
    repls
} | awk '!x[$0]++' | fzf -m --reverse --height 15 --tiebreak=index)

if test -n "$fzf_result"; then
    fre --add "$fzf_result" --store_name "$store_name" && echo "$fzf_result"
fi

Here, $fzf_result would be something like rlwrap sbcl, but fre's suggestion on subsequent reruns would include $PWD, like so: /home/username/rlwrap sbcl.

(awk '!x[$0]++' trick is to avoid sort | uniq )

Tag 0.2

Hey, I've been using this fo years installed from crates.io. I recently wanted to package it and releaized that version 0.2 isn't tagged. Could you please do so. Thx.