Support XDG Base Directory Specification

[jackaudio]

Support XDG Base Directory Specification

[diovudau]

This will be WIP soon.

After the transitional time is over the default will be XDG conform. However, we are not dealing with generated and temporary data, like with a browser, but with real user save files.

While nsmd itself will follow XDG it is advised that GUIs like Agordejo or Carla will make the (permanent) choice of a user dir convenient. The command line switch for choosing your own session base directory exists already, so a GUI could take care of at least informing the user and offering to use a different base directory.

[theGreatWhiteShark]

What about the legacy GUI? Will it choose the former "$HOME/NSM Sessions" or adhere to the XDG specification?

[diovudau]

After the grace period (of several releases) everything will be XDG.
Legacy GUI supports the basedir commandline parameter as well, so a user could choose to override this by starter script or bash alias.

[diovudau]

I am working on the specifications of the new XDG based system (in a different branch). "Readme first" dev style:

https://github.com/linuxaudio/new-session-manager/blob/xdg_and_lockfiles/docs/src/api/index.adoc#session-root-and-session-directories

This also handles lockfiles, see #31

[diovudau]

Further reading material:
https://wiki.archlinux.org/index.php/XDG_user_directories

[diovudau]

Guys, your worries are not ignored. The gravity of this change was clear from the beginning, hence the plan to announce the changes in a release 3 months before anything happens. Until this change becomes official it will be at least till April. There is plenty of time to try things out and revert. Nothing has been finally decided except "Don't put a directory with white space in the home dir" and "Follow an established standard".

[grammoboy2]

I don't see why the feature request to support XDG, results in the choice to put the sessions folder in a hidden folder. If that is the consequence, I wouldn't stick with XDG policies personally. But I don't think that has to be the consequence.

Isn't what you want a base folder for all Linuxaudio projects (after discussion within the LAD community)?

For example:

XDG_AUDIO_DIR="$HOME/Audio"

Where all applications, like Ardour, Qtractor, NSM puts their session folders as default location.

[diovudau]

I'm going to talk about session management philosophy for a bit, since this might be useful information for more people. I might transfer this to a more permanent place later, like the wiki, as a reference document.

Session management is purely about convenience. The goal is to make everything fast, easy and robust against accidents by user-input. Nothing in NSM, or LADISH or custom shell scripts, can't be substituted by the user taking all these actions manually. Of course you all know this, because the steps to discover Session Management are grounded in the pain to start all programs, load all files, make all jack connections by hand etc.

The fear I read between the lines is that someone might be unable to find the saved data of individual programs and this will hinder the audio production workflow. I do not think this is true, and on the contrary, that you don't want to see these as a normal audio producer.

If anyone wants to argue against using the standard XDG Location ~/.local/share then please do it by providing examples ("user stories") when and why access to the files is needed to make music production better. These are the reasons that have the highest chance of convincing me against XDG. Also anticipate in these arguments that I will likely answer with suggesting to solve a specific usability-problem by providing information and actions in a GUI

We are also talking about actual music producing users here. Not developers, not power users.
Hidden dot-directories are basic Linux knowledge. Someone who needs to access the session files directly is likely to have the skill level to know where the files are. E.g. for a developer this whole ordeal is a trivial matter.

Examples

One aspect of convenience is reduction in distractions by hiding background information, and this is the aspect we are talking about here. The comparison to Ardour projects or LibreOffice documents has been made, so I will use these examples as well. I will also explain the Steam comparison I made previously. These examples are here as background information on which decisions are made, such as XDG dir saving.

NSM Sessions are saved files that only become usable when loaded in their specific programs. These are not standard text files to be processed with unix tools and piping. Most important: Even loading the session cannot be done by hand, but you need to run a specialized program (nsmd and a GUI) to do so. It therefore does not matter where the files actually are on the users disk. They are always loaded through a frontend.
If this is the case then having them easily accessible will only result in negative aspects. For example curious, but technical inexperienced users, will want to move the files around manually, either in the session or individually.

LibreOffice Documents are archived sessions

LibreOffice Documents are an archive format, technically, with metadata, text, embedded images etc. This is the LibreOffice-"Session". You want those things packed together and hidden. You don't want to access the individual data yourself.

Ardour Sessions are whole directories

Ardour save files are whole directories. All the user needs is the one single starter file. Direct access to the files, even recorded .wav is so rarely needed that only experts will want to use that (for example destructive processing in Audacity). And in fact, Ardour neatly hides this fact away by saving and loading directories directly, which tells me they might think that handling the individual files is too distracting for a user. I am sure if they could find a performant way to pack all recorded data and lv2 saved meta data into a single file they would happily do that. I mean actually record into this file-container. Why they don't do it is out of scope for this explanation. I suspect technical reasons. Tangent: It is not planned, but theoretically thinkable that a whole session (NSM or not) could be in such a file container (if technical problems are solvable and there is no performance loss).

Steam

Besides its meta-data and cache (which games are installed, library and thumbnail images etc.) Steam saves all its downloaded games and all its save-data into ~/.local/share/Steam. Below that is a very complex directory structure. The comparison to NSM is the saved games and screenshots the user made through Steams capture functionality. There is no everyday-reason to access those save games by hand. Those files are only useful for the actual games to load (which know where they are) and all other user-generated data is handled by the Steam GUI. This is very similar to our NSM, what I described earlier (including the option to integrate non-steam games into your program launcher (which is Agordejo in my case))

Trade-Off LibreOffice/Ardour vs NSM/Steam

LibreOffice and Ardour leaves the organisation of files to the user. You can save and load from wherever you want. This is the traditional way of handling save-files. It works and I having nothing against that (even though I have seen many people in my life struggling with file organisation, to a point that they don't even know where the files are.)

These kind of programs usually keep a "recent files" list and a default suggestion for saving new files/projects that can either be changed or will adapt dynamically by gathering user save-behaviour heuristics. These lists are not complete, they mostly give convenient access based on access-time. If you reinstall your system or transfer your save-files to a different computer these meta-information are usually lost and you have to manually search and load files again until this ad-hoc database is filled up again to be usable and convenient. Again, nothing wrong with that in principle.

But NSM (and Steam) never did it this way. Here the list of sessions is complete and dynamically created each time the users start NSM. You lose control over where to save your files, but you gain faster access and gain a complete list of your sessions.

This is the trade-off and the difference. In NSM you lose a little control over structuring your data but gain convenience through completeness and overview. The proposed XDG change is a step further in this direction.

If one does not like that philosophy the real proposed change must be that NSM can load and save a session (as a directory) anywhere on your disk and the user organises their file structure themselves for the cost of losing the session- list/overview in the GUI. This will not happen in NSM for various reasons.

Alternative solution, but also a standard

Finally finally, notice that I posted https://wiki.archlinux.org/index.php/XDG_user_directories previously as well. This is also an XDG standards that is responsible for creating the special "Documents" directory in your home dir on most all-inclusive distributions This is not a hidden directory at all. I personally don't use this directory in my distribution and find it annoying if I install e.g. Ubuntu on a Laptop. But it also could be a valid saving location. I lean towards ~/.local/share because that is the more established standard of the two, but I am willing to listen for arguments in favour of one or the other.

[dromer]

Why ~/.local/ ? that would imply user installed applications/libraries.

Wouldn't ~/.config/ make more sense in this regard?

[diovudau]

Why ~/.local/ ? that would imply user installed applications/libraries.

Wouldn't ~/.config/ make more sense in this regard?

From https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
$XDG_DATA_HOME defines the base directory relative to which user specific data files should be stored. If $XDG_DATA_HOME is either not set or empty, a default equal to $HOME/.local/share should be used.
This is similar, but not the same to /usr/share, not /usr/local. It is basically the catch-all location for files that are not executables, libraries or sources. At least I'm reading it that way. In any case, .config is different. That would be local configs of programs, independent of actual projects/save-files, like the window state or a "recently used file list".

[dromer]

Hmmm, I have to somewhat disagree here. I think reality lies somewhere in between ~/.local/share/ and ~/.config/.

A session is a configuration of sorts, no? In ~/.config/ I can find many user-generated configurations for all kinds of tools and many programs allow one to switch configurations/settings/sessions. All of which (including things like cached data) are stored here.

~/.local/share/ however is where installation-data is stored. Like /usr/share/ and /usr/local/share/ except not shared globally on the system, but just in the users environment.

For me semantically a session is like a configuration and thus would be stored in ~/.config/

[edit: now diving a bit deeper into my own ~/.local/share/ and now I feel very conflicted :D maybe this is the best place after all. Oh well :) ]

[grammoboy2]

The only possible disadvantage I see of the current situation, is a space. Which is such a small annoyance, that I wonder which real problem we solve here actually, other then 'I want to follow a certain standard'. Not everything has a standard and that's fine in most cases.

A other argument I read is that people thinks the session folder clutters the home directory. Which could be a small annoyance for some, but I wouldn't call it significant myself. I've the habit to make a ~/linuxaudio folder, where I place my linuxaudio related files. So as mentioned before, a standardised ~/Audio folder to follow the XDG for all linuxaudio applications, I could see as a improvement maybe.

The XDG specification should make the organisation of config files more structured as far as I understand. The NSM sessions folder doesn't store config files of applications though, so I don't think XDG specification even applies to the NSM Sessions folder.

It's hard to find examples for applications that saves new made user data in hidden folders. Ok, Steam is mentioned here, but at least users of the original NSM GUI, do use the terminal regularly, while steam users don't if I understand it correctly. I think keeping the NSM files together in one folder is designed with Ardour sessions mind.

To me a disadvantage of XDG is that files are harder to reach and that it takes more time. If the hidden config file is in /home/user/ you've access to it directly by open a terminal, which opens /home/user by default. Same for the file manager.

This is also the advantage of the current situation of the NSM Sessions folder. You open up a terminal or file manager and you have direct access to it, to backup, copy it while changing the session, git backup etc. In the filemanager you've to explicitly show hidden files and then browse to the right place (.config and .local is always a guess for me).

The current situation is faster and more easy cause you're not dealing with hidden folders and hidden multi folder places. It's also what NSM users are used too and which is the same for non- and new-nsm-users. Common ground. A new place for the sessions folder complicates this unnecessarily.

Given the issue is so small and the proposed solution hardly a improvement and arguably even a disadvantage, you should really ask yourself if this is a wise thing to do. I don't think it is and I really have my doubts whether the XDG specifications made for hidden files/folders does actually apply to the NSM Sessions folder.