session file and config file (redundancy)

[aria2]

session file and config file (redundancy)

[tatsuhiro-t]

As for file size, the latest version performs gzip to session file if you add ".gz" extension after the session filename.
At the moment, aria2 internally does not differentiate options from config file and from command-line, also it sometimes changes options on the course of downloads.

[guedouari]

i thought it should be an easy task, i'll see if i can do something about it probably recheck the config file for each save session (i guess this will drain more processor time)

[tatsuhiro-t]

Ah, that would be easier than I think. It would be good to add an option to toggle this because some people may like original behavior.

[guedouari]

looking at SessionSerializer, it shouldn't be that hard. the way i see it. is checking the config file for each save session operation. that way options passed as cmd arguments will be saved since they are not permanent while general config option are ignored. (this works pretty good manually, and i was able to resume download across windows and linux)
any suggestion on the option is much appreciated (if i make it this far)

[nmaier]

You cannot simply ignore options from the global config, as the global config is anything except "permanent". Users may change it all the time.

What would work, however, is de-duplicating option values within the saved session file. You would first store all the values that are common to all session items (e.g. in a special session item) and then ignore them when storing individual items. You might even be less strict and consider all options for de-duplication which "most" individual items share.
When restoring, you would first read the global options. Then for each individual item you would first set all global options and afterwards all individual item options (Also works with the "most" option).
This shouldn't be too hard to do either.
Remember that this is to reduce redundancy and minimizing the file size. So de-duplication has not to be perfect (but slow), but good enough (and fast enough).

Another simple big win would be to use short-option names instead of the long option names. Basically you would assign all PREF_* a new short-option string which is still unique. When reading back options from a serialized session you would then compare against the short and traditional long option names (for backwards compatibility).

Also, @guedouari or @tatsuhiro-t, can you reopen this. I would be interested in this, too.

[nmaier]

Should be noted that the short-option string idea probably won't have much effect when serializing as .gz, though.

[tatsuhiro-t]

In aria2, option values are overwritten in the following order:

1. default values
2. conf file
3. cmd-line

(See option_processing.cc for the order of option application)

For input file, its accompanied options overwrites the result of 3 (or merged into 3).
I understand that the originator wants to just save options specified by 3 and later stage.
The most options are not specified by command-line and hence input file, saving only those options will save significant amount of space.
To implement this stuff, the obvious choice is introduce parent-child relationship in Option class.
Each download item has its own Option object which fills options specified by cmd-line and input-file.
It has the association to Option object which includes options filled by conf file, which in turn has the association to Option object with default value.
When a certain option value is queried, Option object checks its own option values. If it does not have one, it then asks parent Option object and so on.
On serialization, we just asks Option not to consult parent Objects, so the overhead is almost zero.

[tatsuhiro-t]

I'll create a patch based on my idea.

[guedouari]

since you're familiar with the code. i guess you're the best choice. still i've been trying to find onother way with less code. given the existence of a function that returns global-options (like the rpc one) it's shouldn't take more than a couple lines of code and it will be light weight. the only downside is the config file and the cmd arguments will be treated alike and ignored from the session file

[tatsuhiro-t]

I pushed the change 4070113. Please test it. I decided not to add option to toggle this because the content of conf file seems to be rather stable and size reduction is drastic.

[guedouari]

tested with config file only and with cmd args , perfect!
hope i could be of real help next time :)