Python: line continuation situations sometimes fold incorrectly

[scintillaorg]

Python: line continuation situations sometimes fold incorrectly

[nyamatongwe]

It would be unusual to write this code in that form as Python programmers will commonly indent continued statements. Lexilla lexers and folders concentrate on languages as they are commonly written.

[Davidy22]

Oh, if we're just covering typical formatting and not all possible formatting then I guess this isn't an issue for either language.

[mpheath]

I see invalid code. Let's look into this.

bat file to create "output.log"

> output.log call :test 2>&1
exit /b

:test
py -0
type test1.py
py test1.py
type test2.py
py test2.py
type test3.py
py test3.py

On Linux, type is like cat and py is like python3, except py is a launcher.

"output.log" in the directory "C:\Test"

C:\Test> py -0 
Installed Pythons found by py Launcher for Windows *


 -3.8-64
C:\Test> type test1.py 
param2 = 1

if param2 <= 2:
    print((-20 %


3) / 5)
    else:
        print(1)

C:\Test> py test1.py 
  File "test1.py", line 8
    else:
    ^
SyntaxError: invalid syntax

C:\Test> type test2.py 
param2 = 1

if param2 <= 2:
    print((-20 % 3) / 5)
    else:
        print(1)

C:\Test> py test2.py 
  File "test2.py", line 5
    else:
    ^
SyntaxError: invalid syntax

C:\Test> type test3.py 
param2 = 1

if param2 <= 2:
    print((-20 %


3) / 5)
else:
    print(1)

C:\Test> py test3.py 
0.2

Only test3.py is valid syntax as the else is at the same indent level as the if.

I am curious to know which Python interpreter would allow test1.py:

param2 = 1

if param2 <= 2:
    print((-20 %


3) / 5)
    else:
        print(1)

or test2.py without continuation caused by the parentheses, as I expect the interpreter to read the previous code like:

param2 = 1

if param2 <= 2:
    print((-20 % 3) / 5)
    else:
        print(1)

to run without error.

test3.py runs OK

param2 = 1

if param2 <= 2:
    print((-20 %


3) / 5)
else:
    print(1)

and the code folding appears OK.

I added the param2 = 1 else any test would error.

Look how Github colours the else as it's lexing surprisingly knows whether the else is valid or not. Impressive lexing, or lucky by chance.

The indent before the else cannot be ignored just because the previous line is a continued line. It needs to match the indent before the if.

This is valid Python code:

def test():
    print(1 + 2)
    print(1 +
  2)
    print(1 +
              2)
    print('abc'
'def'
                    'ghi')
    print('\
abcdefghi')
    print('''The rules of indentation
does not apply to continuations as
they are resolved during parsing.
    print() for example, not this textual one, needs to be indented.''')

test()

Indenting requirements does not apply with a continued lines while the above shows indenting of 2, 4 or any other amount of spaces. It is valid as all of the print calls are indented 4 spaces.

@Davidy22, do you still consider the initial code as valid Python code?

[Davidy22]

Oh, whoops, I copied it from a test I had in a file where this was inside a function, nuked the indentation before the first if so the else has an excess indentation. The contents of test3.py were the intention, and they still fold incorrectly: this is what it looks like on its own:

Note how the fold indicator on the side stops on line 17 instead of extending down to just before the else

[mpheath]

OK, please post the full reproducable code next time.

I see the same folding with SciTE.

Line 17 points to the start of the line instead of the end, which is line 20. The lexer would need to account for all continuation types and be able to interpret correctly to the end. The person who would implement this would deserve a rather large treat.