Overall download limit does not work as far as I can tell

[aria2]

Overall download limit does not work as far as I can tell

[marcbowes]

http://aria2.sourceforge.net/manual/en/html/aria2c.html#cmdoption--max-overall-download-limit claims:

--max-overall-download-limit=
Set max overall download speed in bytes/sec. 0 means unrestricted. You can append K or M (1K = 1024, 1M = 1024K). To limit the download speed per download, use --max-download-limit option. Default: 0

So following that I would expect that limit >= sampled should always hold true. However, for the lowest speed I ever recorded, the equation 20 * 1024 >= 241659 is clearly a lie.

[marcbowes]

Upgrading to 1.15.2 doesn't fix this either.

aria2_client.call("changeGlobalOption", {"max-overall-download-limit" => "10K"})
 => "OK" 
aria2_client.call("getGlobalStat")
 => {"downloadSpeed"=>"111348", "numActive"=>"1", "numStopped"=>"3", "numWaiting"=>"0", "uploadSpeed"=>"0"} 
111348 / 1024.0
 => 108.73828125 

/cc @timsjoberg

[timsjoberg]

Mines closer to the mark. It seems to sit at roughly 10K and spikes occasionally when limited to 10K

server = XMLRPC::Client.new2("http://#{user}:#{password}@127.0.0.1:6800/rpc")
server.call("aria2.changeGlobalOption", {"max-overall-download-limit" => "10K"})
loop do
  puts server.call("aria2.getGlobalStat")["downloadSpeed"]
  sleep 0.2
end

11673
10591
9776
9776
9098
13115
13115
11781
11176
10637
12249
11699
11699
11268
10870
10221
9861
9573
9573
11882
11261
10954
10663
10388
10092
9854
9624
9624
11331
10864
10626
10401
10182
9947
9750
9750
9564
10702
10503
10503
10316
17913
11167
10976
10761
10761
10586
10273
10111
9957
10962
10788
10624
10624
10468

running gentoo with aria 1.14.1

[marcbowes]

I suspect it's got nothing to do with the version and everything to do with the way aria is compiled. Maybe this is a bug with the ppa-vended binary.

[tatsuhiro-t]

I checked current dev version and 1.15.2 PPA and found that they both exhibited some spikes when max-overall-download-limit was used, but overall they sited around the ratio aria2 was told to limit, just like timsjoberg presented.

For marcbowes's case, it seems to me that aria2 behaved as if max-overall-download-limit change is ignored because its huge difference of expected value and actual one.
What kind of RPC library or tool to communicate with aria2?
How many simultaneous downloads when you corrected these values?
Also was bittorrent involved in them?

I admit that aria2c's speed regulation implementation is not so accurate right now. I think it leaves some improvements. But I think 700KB/s when limited to 200K is not what I expected from the current implementation.

[marcbowes]

What kind of RPC library or tool to communicate with aria2?

I'm using Ruby via the XMLRPC::Client just like @timsjoberg. The only reason there is sugar on the syntax is because of a wrapper which just prepends aria2. to the calls. Looking in the aria logs I do see it's receiving the request successfully and, given the response to getGlobalStat reflects my request, I think we can safely say it's not the RPC tool.

How many simultaneous downloads when you corrected these values?

I can reproduce this with a single metalink being downloaded with split set to 16. My aria2.conf looks like:

dir=/home/marc/archive/
log=/tmp/aria.log
check-integrity=true
max-connection-per-server=16
max-concurrent-downloads=5
min-split-size=1M
follow-torrent=mem
max-overall-upload-limit=30K
follow-metalink=mem
split=16
rpc-listen-all=false
rpc-listen-port=6800
rpc-passwd=aria2
rpc-user=aria2
allow-piece-length-change=true
save-session=/home/marc/.leecher/session.aria2

Also was bittorrent involved in them?

All metalinks using HTTP (not S) with the above settings

I admit that aria2c's speed regulation implementation is not so accurate right now.

That's fine. If spikes are a problem I can always lower the throttling to account for it. At the moment I'm just looking for some mechanism to 'renable browsing' during downloads :-)

[tatsuhiro-t]

Thank you for updates and I could reproduce the issue.
The problem is that the occasional spike is amplified with many connections (16 in this case).
I pushed the fix in master branch (46bdaf0).
The fix only covers HTTP/FTP part, so I have to do the similar fix for BitTorrent.

[marcbowes]

Thanks! You rock!

I commented on your commit to make sure I understood what was going on. In my opinion, it looks like your fix will work. When will this be available via the PPA? I'm eager to upgrade since automated downloads currently brown out my browsing experience :-)

---

On a completely different note (feel free to ignore):

What do you think about having a web-hook as an option to aria's callback system. Currently I configure aria to invoke a shell script which lets my workflow know the download has changed state. It'd be nice if it could just deliver a payload via HTTP to some endpoint -- even if that payload is just a notice saying GID 1 has changed state: the receiver can always poll over the RPC interface to determine what the change is.

Feel free to close.

[tatsuhiro-t]

Usually, PPA is updated on each new release.
I think you can build aria2 fairly easily if you are using Ubuntu or Debian.
There is how to build instructions in README.rst.

Maybe you can use curl in shell script to GET/POST GID to Web server?

[marcbowes]

Yes, doing that in a script is pretty easy. Having it as a built-in would be pretty nice too since it's one less moving part to manage. Closing this out.

Thanks again.

[Shekelme]

The issue is closed, but I see now that --max-overall-download-limit is being ignored. I start my downloads in this way:
aria2c --max-overall-download-limit=150M -i 1998.txt -d /media/storage-21/Don/1998 --continue=true -x5
Ubuntu 20.04.6 LTS, aria2 version 1.35.0