Progress/ETA status line moves across screen and ends up garbled at the far right

[rndusr]

Progress/ETA status line moves across screen and ends up garbled at the far right

[rndusr]

Wow, thank's for researching this! Great information. I remember fearing that one of the escape codes might not work in every terminal. That seems to be the case here. I'll try to get this solved over the weekend.

[rndusr]

Was the line broken right from the beginning or was it displayed correctly at first? Can you reproduce this or was it a one-time bug? Any ideas how I could reproduce this? I'm staring at the code and don't see what could cause this.

[marcelpaulo]

Was the line broken right from the beginning or was it displayed correctly at first?

It's very fast, but it seems that it's first shown in the middle of the screen

Can you reproduce this or was it a one-time bug?

It always happens. Until today I hadn't created any big torrents, but I had noticed that the progress status line ended up not left aligned. As the torrents were small, usually the status line was shown only once, but it was always displaced along the screen. I apologize for being lazy and not reporting it before. The issue became more visible when I created this big torrent today.

Any ideas how I could reproduce this?

This should created a big enough torrent to force torf to show a couple of progress status lines:

mkdir foo
for i in {1..10}; do fallocate -l 500MiB foo/foo$i; done
torf foo

fallocate comes from package util-linux, which is always pre-installed in Ubuntu. In case you don't have it, you can use dd instead, which is more common, but takes much longer to run:

for in {1..10}; do dd if=/dev/zero of=foo/foo$i bs=500MiB count=1; done

Redirecting torf's output to a file, I can see that each progress line is prefixed by 2 ANSI ESC sequences, ESC[2K ESC[1`. Looking it up here, I could decypher that ESC[2K runs clear entire line, but I couldn't decypher ESC[1`. Certainly they're responsible for displacing the progress lines, and the Python code used to output the progress lines is generating them.

Here's an excerpt, from running torf on the fallocated files:

00000000  4e 61 6d 65 09 66 6f 6f  0a 53 69 7a 65 09 35 32  |Name.foo.Size.52|
00000010  34 32 38 38 30 30 30 30  0a 43 72 65 61 74 69 6f  |42880000.Creatio|
00000020  6e 20 44 61 74 65 09 32  30 31 38 2d 30 35 2d 33  |n Date.2018-05-3|
00000030  31 20 30 30 3a 30 30 3a  30 30 0a 43 72 65 61 74  |1 00:00:00.Creat|
00000040  65 64 20 42 79 09 74 6f  72 66 2f 31 2e 32 0a 50  |ed By.torf/1.2.P|
00000050  72 69 76 61 74 65 09 6e  6f 0a 50 69 65 63 65 20  |rivate.no.Piece |
00000060  53 69 7a 65 09 34 31 39  34 33 30 34 0a 50 69 65  |Size.4194304.Pie|
00000070  63 65 20 43 6f 75 6e 74  09 31 32 35 30 0a 46 69  |ce Count.1250.Fi|
00000080  6c 65 20 43 6f 75 6e 74  09 31 30 0a 46 69 6c 65  |le Count.10.File|
00000090  73 09 66 6f 6f 2f 66 6f  6f 31 09 66 6f 6f 2f 66  |s.foo/foo1.foo/f|
000000a0  6f 6f 31 30 09 66 6f 6f  2f 66 6f 6f 32 09 66 6f  |oo10.foo/foo2.fo|
000000b0  6f 2f 66 6f 6f 33 09 66  6f 6f 2f 66 6f 6f 34 09  |o/foo3.foo/foo4.|
000000c0  66 6f 6f 2f 66 6f 6f 35  09 66 6f 6f 2f 66 6f 6f  |foo/foo5.foo/foo|
000000d0  36 09 66 6f 6f 2f 66 6f  6f 37 09 66 6f 6f 2f 66  |6.foo/foo7.foo/f|
000000e0  6f 6f 38 09 66 6f 6f 2f  66 6f 6f 39 0a 50 61 74  |oo8.foo/foo9.Pat|
000000f0  68 09 66 6f 6f 0a 1b 5b  32 4b 1b 5b 31 60 50 72  |h.foo..[2K.[1`Pr|
00000100  6f 67 72 65 73 73 09 30  2e 30 38 20 25 1b 5b 32  |ogress.0.08 %.[2|
00000110  4b 1b 5b 31 60 50 72 6f  67 72 65 73 73 09 33 2e  |K.[1`Progress.3.|
00000120  36 30 20 25 20 20 7c 20  20 45 54 41 3a 20 31 34  |60 %  |  ETA: 14|
00000130  3a 35 38 3a 34 30 20 20  7c 20 20 30 3a 30 30 3a  |:58:40  |  0:00:|
00000140  31 34 20 6c 65 66 74 20  20 7c 20 20 33 34 38 2e  |14 left  |  348.|
00000150  38 32 20 4d 42 2f 73 1b  5b 32 4b 1b 5b 31 60 50  |82 MB/s.[2K.[1`P|
00000160  72 6f 67 72 65 73 73 09  37 2e 32 38 20 25 20 20  |rogress.7.28 %  |
00000170  7c 20 20 45 54 41 3a 20  31 34 3a 35 38 3a 34 31  ||  ETA: 14:58:41|
00000180  20 20 7c 20 20 30 3a 30  30 3a 31 34 20 6c 65 66  |  |  0:00:14 lef|
00000190  74 20 20 7c 20 20 33 35  36 2e 33 39 20 4d 42 2f  |t  |  356.39 MB/|
000001a0  73 1b 5b 32 4b 1b 5b 31  60 50 72 6f 67 72 65 73  |s.[2K.[1`Progres|
000001b0  73 09 31 30 2e 39 36 20  25 20 20 7c 20 20 45 54  |s.10.96 %  |  ET|
000001c0  41 3a 20 31 34 3a 35 38  3a 34 31 20 20 7c 20 20  |A: 14:58:41  |  |
000001d0  30 3a 30 30 3a 31 34 20  6c 65 66 74 20 20 7c 20  |0:00:14 left  | |
000001e0  20 33 35 39 2e 31 39 20  4d 42 2f 73 1b 5b 32 4b  | 359.19 MB/s.[2K|
000001f0  1b 5b 31 60 50 72 6f 67  72 65 73 73 09 31 34 2e  |.[1`Progress.14.|
00000200  36 34 20 25 20 20 7c 20  20 45 54 41 3a 20 31 34  |64 %  |  ETA: 14|
00000210  3a 35 38 3a 34 31 20 20  7c 20 20 30 3a 30 30 3a  |:58:41  |  0:00:|
00000220  31 33 20 6c 65 66 74 20  20 7c 20 20 33 35 39 2e  |13 left  |  359.|
00000230  38 37 20 4d 42 2f 73 1b  5b 32 4b 1b 5b 31 60 50  |87 MB/s.[2K.[1`P|
00000240  72 6f 67 72 65 73 73 09  31 38 2e 32 34 20 25 20  |rogress.18.24 % |
00000250  20 7c 20 20 45 54 41 3a  20 31 34 3a 35 38 3a 34  | |  ETA: 14:58:4|
00000260  31 20 20 7c 20 20 30 3a  30 30 3a 31 33 20 6c 65  |1  |  0:00:13 le|
00000270  66 74 20 20 7c 20 20 33  36 31 2e 34 36 20 4d 42  |ft  |  361.46 MB|

[marcelpaulo]

Ignoring the second escape sequence, the effect of repeatedly printing a text followed by ESC[2K is:

1. The text is output
2. The text is cleared, the cursor remaining in the same position
3. Repeat

The result is: it appears as though the text were "displacing" along the screen, as I observed.

[marcelpaulo]

I get the impression the issue lies with _util.clear_line(), which is called before outputting each progress line.

[marcelpaulo]

Yes, _util.clear_line() causes the issue:

def clear_line():
    print('\x1b[2K\x1b[1`', end='', flush=True)

Looking here, I realized that this ANSI ESC sequence moves the cursor to the beginning of the current line:

ESC[0E

Actually, it means move to the beginning of 0 lines down, which will have the expected effect. So, changing _util.clear_line() to:

def clear_line():
    print('\x1b[2K\x1b[0E', end='', flush=True)

will solve the 1st problem.

I don't know how to solve the 2nd problem: when piping the output of torf, the progress line is also being output with the ANSI ESC sequences. I guess either no progress line should be shown, or perhaps one on each line, with no ANSI ESC sequences.

EDIT: As it's such a simple fix, let me try my hand again at a pull request.

[marcelpaulo]

I wonder if there isn't a more high-level Python way of doing this, without resorting to ANSI ESC sequences.

Anyway, I'll be sending you a PR in a moment (I'm kind of slow with these things !) for this 1st problem.

[marcelpaulo]

Hey, d14cd96 now finishes off this issue, fixing the 2nd problem. Thanks !

[rndusr]

Sure thing!