Preventing regular ETA update or replacing it

[HarikalarKutusu]

Preventing regular ETA update or replacing it

[wolph]

Thank you for the compliments and the detailed issue! Very helpful :)

Your ETA issue is because there are probably not enough samples for you case.

By default the progressbar will use an AdaptiveETA:

python-progressbar/progressbar/bar.py

Line 745 in 6b0e48e

widgets.AdaptiveETA(**self.widget_kwargs),

The AdaptiveETA will use 2 seconds of samples:

python-progressbar/progressbar/widgets.py

Line 393 in 6b0e48e

samples=datetime.timedelta(seconds=2),

For regular progressbars that is usually enough without storing too many values in memory, but in your case I think you want to increase that number significantly :)

This can be done using the widget_kwargs to the ProgressBar or by specifying the widgets completely manually. In the case of widget_kwargs:

self._bars.append(
  progressbar.ProgressBar(fd=BarStreamWrapper(0), prefix='Chunks:   ', max_value=num_chunks, poll_interval=5, min_poll_interval=5,
    widget_kwargs=dict(samples=datetime.timedelta(minutes=10)),
)

There might be an alternative as well however. I am currently working on a newer version (only available on the develop branch right now) which features an automatic MultiBar class that re-renders the progressbars continuously regardless of the regular update intervals:
https://github.com/wolph/python-progressbar#multiple-threaded-progressbars

With regard to the poll_interval and min_poll_interval, those are used for visual updates because writing to the terminal is the slowest part of the progressbar.

For time sensitive widgets the INTERVAL constant of the widget itself is what decides the interval:

python-progressbar/progressbar/widgets.py

Line 298 in 6b0e48e

INTERVAL = datetime.timedelta(milliseconds=100)

[HarikalarKutusu]

In fact, I have total_chunk_count, num_chunks, and num_completed_chunks. I can estimate the ETA from these. I also have a datetime object around to measure timing. So:

secs = (datetime.now() - start_time).total_seconds()
avg = secs / num_completed_chunks  # assumes num_completed_chunks > 0 for the sake of the example
eta_secs = avg * (num_chunks - num_completed_chunks)

I need a way to override the ETA with a CustomETA class as far as I understand, like in here. But how can I pass these parameters to the Widget?

[HarikalarKutusu]

Thank you for the detailed help and pointers @wolph ... I was just looking at the AdaptiveETA code... I'll try the widged_kwargs argument right away.

I read about MultiBar but could not find it, this is why :) Eagerly waiting for a new version!

[HarikalarKutusu]

Should it read:

widget_kwargs={'interval': timedelta(minutes=5)}

[wolph]

Oops... I didn't read correctly. It's the samples argument: widget_kwargs=dict(samples=datetime.timedelta(minutes=10)),

[HarikalarKutusu]

Oh. I thought samples was for the number of samples for ETA calc...

Can I change these during runtime (not only during creating the object) - so that I can make it adaptive?

[wolph]

Yes, you can change the variable whenever you wish. It should also be noted that you can use either an integer as a specific number of samples (depending on how often the chunks bar is updated) or a timedelta with a specific time interval.

[HarikalarKutusu]

That clarifies it, thanks...

BTW, it works much better now :)