fchauvel/flap

Flattening LaTeX projects

FLaP — Flat LaTeX Projects

FlaP is a simple utility that flattens LaTeX projects. It takes a well-organised LaTeX project—one you so carefully crafted—and merged it into a vulgar single LaTeX file in a 'flat' directory.

• Why? Some publishers require one such flat structure when you submit the sources of your manuscript, and I got tired to flatten the sources by hand.

• Another one? There already a couple of tools that merge latex projects such as latexpand, flatex or flatten. As far as I know, they only merge TeX files, without moving graphics around. I learned recently about texDirflatten, which seems to be an alternative, but I haven't tried it.

Installation

FLaP requires Python 3 (3.6, 3.6, 3.8 and 3.9 are tested). The easiest way to install latest official release is to use pip using:

$ pip install flap

Alternatively, you get the latest development version using:

$ pip install git+https://github.com/fchauvel/flap.git

Should you want to look at the code, you may download the sources distribution and use

$ git clone https://github.com/fchauvel/flap.git 
$ cd flap
$ python setup.py install .`

Usage Example

For the newest (development) version, use:

$ flap project/main.tex output/directory

Prior to v0.4.1 (included), we invoke FLaP using:

$ python -m flap project/main.tex output/directory

See also the online documentation.

Contact Us

If you give FLaP a try, please report any bugs, issues or feature request using the issue tracker.

flap's Issues

`\def`are stripped out

From @gro1729, Issue 22:

In combination with the package "wrapfigure" I had to define some variables:

\newcounter{figure} 
\makeatletter 
\def\ext@figure{} 
\makeatother 
\newcounter{table} 
\makeatletter 
\def\ext@table{} 
\makeatother 

FLaP translates this to :

\newcounter{figure} 
\makeatletter 
\makeatother 
\newcounter{table} 
\makeatletter 
\makeatother

in the file "merged.tex"

Local style files are copied but the inclusion directives are not adjusted

From @gro1729, Issue 22:

My style file "onimage.sty" is copied into the "merged"-Root-Directory, but in "merged.tex" I still find the line \usepackage{input/onimage}

ValueError: Unexpected CONTROL '\MessageBreak' in file svjour3.cls (line 130, column 1).

Below is the whole log:

Traceback (most recent call last):
  File "/Users/shin/miniconda3/envs/latex_env/bin/flap", line 8, in <module>
    sys.exit(main())
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/ui.py", line 102, in main
    Controller(OSFileSystem(), Display(sys.stdout, verbose)).run(tex_file, output)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/ui.py", line 43, in run
    request.execute()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/engine.py", line 79, in execute
    flattened = self._rewrite(self.read_root_tex, str(self.root_tex_file.resource()))
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/engine.py", line 88, in _rewrite
    return parser.rewrite()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 78, in rewrite
    result += self._rewrite_one()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 86, in _rewrite_one
    return self._evaluate_one()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 138, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 164, in evaluate_command
    return macro.invoke(self)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 124, in invoke
    return self._execute(parser, invocation)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 207, in _execute
    self._flap.relocate_dependency(class_name, invocation)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/engine.py", line 109, in relocate_dependency
    self._rewrite(file.content(), file.fullname(), symbol_table)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/engine.py", line 88, in _rewrite
    return parser.rewrite()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 78, in rewrite
    result += self._rewrite_one()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 86, in _rewrite_one
    return self._evaluate_one()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 138, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 164, in evaluate_command
    return macro.invoke(self)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 123, in invoke
    invocation = self._parse(parser)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 129, in _parse
    self._capture_arguments(parser, invocation)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 144, in _capture_arguments
    value = parser._evaluate_until(lambda token: token.has_text(next_token._text))
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 153, in _evaluate_until
    tokens += self._evaluate_one()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 136, in _evaluate_one
    return self._evaluate_group()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 146, in _evaluate_group
    tokens = self._evaluate_until(lambda token: token.ends_a_group)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 153, in _evaluate_until
    tokens += self._evaluate_one()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 138, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 164, in evaluate_command
    return macro.invoke(self)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 123, in invoke
    invocation = self._parse(parser)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 129, in _parse
    self._capture_arguments(parser, invocation)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/macros.py", line 203, in _capture_arguments
    invocation.append_argument("class", parser.capture_group())
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 203, in capture_group
    tokens = self._accept(lambda token: token.begins_a_group)
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 109, in _accept
    self._raise_unexpected_token()
  File "/Users/shin/miniconda3/envs/latex_env/lib/python3.6/site-packages/flap/latex/parser.py", line 118, in _raise_unexpected_token
    raise ValueError(error)

Support for overpic environment

Created after #1 (comment)
Uups, there is another "include"-Environment, I use:

        \begin{overpic}[scale=0.25,unit=1mm,grid,tics=10]{%
           figures/TeXMakerUndStudioUndCo/DieHardcoreVariante/AcrobatReader}
        \end{overpic}

\bibliographystyle{unsrt} raises flap.engine.ResourceNotFound: None

control.tex                       99      1 \bibliographystyle{unsrt}          
Traceback (most recent call last):
  File "/home/mq/miniconda3/bin/flap", line 9, in <module>
    load_entry_point('FLaP', 'console_scripts', 'flap')()
  File "/home/mq/repo/github/flap/flap/ui.py", line 106, in main
    Controller(OSFileSystem(), Display(sys.stdout)).run(arguments)
  File "/home/mq/repo/github/flap/flap/ui.py", line 38, in run
    request.execute()
  File "/home/mq/repo/github/flap/flap/engine.py", line 69, in execute
    flattened = self._rewrite(self.read_root_tex, str(self.root_tex_file.resource()))
  File "/home/mq/repo/github/flap/flap/engine.py", line 76, in _rewrite
    return parser.rewrite()
  File "/home/mq/repo/github/flap/flap/latex/parser.py", line 100, in rewrite
    result += self._rewrite_one()
  File "/home/mq/repo/github/flap/flap/latex/parser.py", line 108, in _rewrite_one
    return self._evaluate_one()
  File "/home/mq/repo/github/flap/flap/latex/parser.py", line 156, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/home/mq/repo/github/flap/flap/latex/parser.py", line 181, in evaluate_command
    return macro.invoke(self)
  File "/home/mq/repo/github/flap/flap/latex/macros.py", line 111, in invoke
    return self._execute(parser, invocation)
  File "/home/mq/repo/github/flap/flap/latex/macros.py", line 357, in _execute
    new_link = self.update_link(parser, link, invocation)
  File "/home/mq/repo/github/flap/flap/latex/macros.py", line 397, in update_link
    return parser._engine.update_link_to_bibliography_style(link, invocation)
  File "/home/mq/repo/github/flap/flap/engine.py", line 111, in update_link_to_bibliography_style
    return self._update_link(path, invocation, [self.root_directory], ["bst"], ResourceNotFound(None))
  File "/home/mq/repo/github/flap/flap/engine.py", line 119, in _update_link
    resource = self._find(path, location, extensions, error)
  File "/home/mq/repo/github/flap/flap/engine.py", line 151, in _find
    raise error
flap.engine.ResourceNotFound: None

Support for Python 3.2

Create after #1 (comment).

I will check for the compatibility with python 3.2 and let you know.

biblatex

In the latest version (0.5.0) \addbibresource{main.bib} is ignored and file not moved.

\endinput

I use \LaTeX\ based slides since several years --and at least several thousands of slides later, I decided -for my department too- "one slide - one file"

Instead of
\begin{comment}
\end{comment}

I use the
\endinput
command

handling of comments in \verb{}

Hello, inside a fragile slide, I use des "\verb"-Environment

           \begin{center}
              \verb|\| \verb|{| \verb|}| \verb|%| \verb|~| \verb|&|
              \verb|#| \verb|$| \verb|^| \verb|_|
           \end{center}

"flap" translates to

           \begin{center}
              \verb|\| \verb|{| \verb|}| \verb|
              \verb|#| \verb|$| \verb|^| \verb|_|
           \end{center}

I think, this can be considered to be a "featured bug"

Invalid substitutions performed within user-defined commands

See initial comments from Issue 15 about how FLaP fails when an input directive is used within a user-defined command.

Unable to specify the name of the merged file

As pointed by @gro1729 (see Issue 28), Version 0.5.0 does not permit the definition of the name of the merged file:

$ flap project/main.tex flatten/my-customised-merged.tex

Relative paths yield invalid file names (Windows)

From @gro1729, Issue 22:

I use relative pathnames like: "../VeryLongPathName/Pyrit".

\includegraphics[width=0.275\textwidth,interpolate]{% 
.._VeryLongPathName_Pyrit}

On Linux FLaP generates a local file ".._VeryLongPathName_Pyrit" in the merged-directory. The filename is valid for Linux but not for Windows. "lualatex" also has trouble with the compilation.

Appropriate re-naming of files

Last issue --

I often have files with the same names in different subdirectories (eg. I include two files with name appendix.tex from two different chapters). The way things are named now, one will overwrite the other when they are moved to the same directory. The obvious way around this is to rename things with their directory structure in the filename, so eg. chapter/figures/figure.pdf goes to chapter-figures-figure.pdf. Would that make sense to do?

Comments replaced by newline cause havoc

Hi,

After applying FLaP to my latex project, I tried compiling the created "merged.tex" file. However, it didn't compile, because it had odd spacing in some of the calls. This is because I have lines like:

\documentclass[ twoside,openright,titlepage,numbers=noenddot,headinclude,%1headlines,% letterpaper a4paper
                 footinclude=true,cleardoublepage=empty,abstractoff, % <--- obsolete, remove (todo)
                BCOR=5mm,paper=a4,fontsize=11pt,%11pt,a4paper,%
                american,%
                %listof = withchapterentry,
                ]{scrreprt}

The commented line gets replaced with a newline, but that won't compile. If however comments are replaced with an empty string, it compiles. I don't know if this would break anything else however.

ValueError: not enough values to unpack (expected 2, got 1)

OS: Win10 Pro 64 bit
FLaP version: 0.5.0
I used a normal TeXlive 2018 installation and then installed flap and click (both via pip).

The Source-Code of main.tex and lit.bib was in a directory C:\temp\1 and below there were the directories C:\temp\1\pics and C:\temp\1\sections witj further source files.

I tried:
C:\temp>flap 1/main.tex 1-flapped/output
and
C:\temp>flap 1/main.tex output1

In both cases I got:

FLaP 0.5.0
Traceback (most recent call last):
File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\Scripts\flap-script.py", line 11, in
load_entry_point('FLaP==0.5.0', 'console_scripts', 'flap')()
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\click\core.py", line 722, in call
return self.main(*args, **kwargs)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\click\core.py", line 697, in main
rv = self.invoke(ctx)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\click\core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\click\core.py", line 535, in invoke
return callback(*args, **kwargs)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\ui.py", line 102, in main
Controller(OSFileSystem(), Display(sys.stdout, verbose)).run(tex_file, output)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\ui.py", line 43, in run
request.execute()
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\engine.py", line 79, in execute
flattened = self._rewrite(self.read_root_tex, str(self.root_tex_file.resource()))
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\engine.py", line 88, in _rewrite
return parser.rewrite()
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\latex\parser.py", line 78, in rewrite
result += self._rewrite_one()
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\latex\parser.py", line 86, in _rewrite_one
return self._evaluate_one()
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\latex\parser.py", line 138, in _evaluate_one
return self.evaluate_command(str(self._next_token))
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\latex\parser.py", line 164, in evaluate_command
return macro.invoke(self)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\latex\macros.py", line 124, in invoke
return self._execute(parser, invocation)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\latex\macros.py", line 288, in _execute
style_file = self._fetch_style_file(parser, invocation)
File "c:\users\admin\appdata\local\programs\python\python36-32\lib\site-packages\flap\latex\macros.py", line 296, in _fetch_style_file
_, value = each.split("=")
ValueError: not enough values to unpack (expected 2, got 1)

Any help is really appreciated.

Best regards, Bernhard

FLaP cannot process symbolic links used for resources files

See #1 (comment)

A matter of taste

Today, I sent the merged presentation to several colleagues. Some of them use Linux, most of them work with Windows. Conversion from SVG to PDF works properly on Linux. The typical Windows-Installation of Inkscape does not work in combination with TeXLive. As a consequence, I changed "engine.py":

   def extensions_by_priority(self):

return ["svg"]

       return ["pdf"]

[Errno 2] No such file or directory ```flap\__main__.py\merged.tex```

Was looking for a LaTeX flattening script in Python. Found yours. Was very content installing it. But FLaPing my project fails with the following error message

$ /c/Program\ Files/Python35/python.exe -m source.tex root.tex
FLaP v0.3.0 -- Flat LaTeX Projects
Error: [Errno 2] No such file or directory: 'C:\\Program Files\\Python35\\lib\\site-packages\\flap\\__main__.py\\merged.tex'

This was performed on Win 7 x64 in a GitBash, but even the regular cmd would fail with the same error message. At first I thought it was due to missing elevated rights on the flap package directory, but after starting GitBash and cmd as admin and testing it there to no avail, I realized that your script is trying to create a file in a folder called __main__.py which obviously cannot work as there is no such folder.

Python 3.2

What do I have to do to use Python 3.2?

My Linux-Box is outdated, I know, but never change a running system. SuSE Linux 11.1 does not support new Releases of Python. Switching to a new SuSE-installation would take to much time in the next month.

Regards from Andreas

number images consecutively

Some journals require you to map the figure filename to the figure number in text. Doing this automatically would be a great feature.

Local packages in subdirectories are not copied

According to @gro1729, the traversal of style files that are located in subdirectories seems broken
(see Issue #21).

wrong files was used in includegraphics if multiple files have the same prefix

I have the following file structure

main.tex
image
|___screenshot.png
|___screenshot_not_the_one_I_want.png

And the content of main.tex is

\documentclass{article}

\begin{document}
        \includegraphics[width=\textwidth]{image/screenshot}
\end{document}

When I run flap main.tex merge, the resulting merged file is using screenshot_not_the_one_I_want.png instead of screenshot.png

merged.tex:

\documentclass{article}

\begin{document}
        \includegraphics[width=\textwidth]{image_screenshot_not_the_one_I_want}
\end{document}

flap is confused by the use of `\includesvg` as a parameter

To avoid compiling svgs during externalization, I have the following directive in my preamble:

\tikzset{external/optimize command away={\includesvg}{1}}

That confuses flap, which says ValueError: Unexpected END_GROUP '}' in file 00-includes/preamble.tex (line 101, column 54).

Backtrace

% flap -v main.tex flapped
FLaP 0.6.0
File                            Line Column LaTeX Command                      
-------------------------------------------------------------------------------
main.tex                          10      1 \input{00-includes/preamble.tex}   
Traceback (most recent call last):
  File "/home/mcantan/.local/bin/flap", line 8, in <module>
    sys.exit(main())
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/ui.py", line 117, in main
    .run(tex_file, output)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/ui.py", line 43, in run
    request.execute()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/engine.py", line 84, in execute
    tokens = self._rewrite(self.read_root_tex,
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/engine.py", line 94, in _rewrite
    return parser.rewrite()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 90, in rewrite
    result += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 98, in _rewrite_one
    return self._rewrite_command()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 114, in _rewrite_command
    return macro.rewrite(self)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 54, in rewrite
    return self._execute(parser, invocation)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/inlining.py", line 37, in _execute
    return parser._spawn(parser._create.as_tokens(content, link), dict()).rewrite()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 90, in rewrite
    result += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 96, in _rewrite_one
    return self._rewrite_group()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 105, in _rewrite_group
    tokens += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 96, in _rewrite_one
    return self._rewrite_group()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 105, in _rewrite_group
    tokens += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 96, in _rewrite_one
    return self._rewrite_group()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 105, in _rewrite_group
    tokens += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 98, in _rewrite_one
    return self._rewrite_command()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 114, in _rewrite_command
    return macro.rewrite(self)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 53, in rewrite
    invocation = self._parse(parser)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 64, in _parse
    self._capture_arguments(parser, invocation)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 127, in _capture_arguments
    invocation.append_argument("link", parser.capture_group())
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 237, in capture_group
    tokens = self._accept(lambda token: token.begins_a_group)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 135, in _accept
    self._raise_unexpected_token()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 144, in _raise_unexpected_token
    raise ValueError(error)
ValueError: Unexpected END_GROUP '}' in file 00-includes/preamble.tex (line 101, column 54).

I'll comment that line out and report the next error :)

Conditional `\makeindex` makes compilation fail with `ValueError: not enough values to unpack (expected 2, got 1)`

Simple reproducer:

\ifdefined\tikzexternalrealjob{} % This could be any conditional
\else
\makeindex  
\fi

Backtrace

FLaP 0.6.0
Traceback (most recent call last):
  File "/home/mcantan/.local/bin/flap", line 8, in <module>
    sys.exit(main())
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/ui.py", line 117, in main
    .run(tex_file, output)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/ui.py", line 43, in run
    request.execute()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/engine.py", line 84, in execute
    tokens = self._rewrite(self.read_root_tex,
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/engine.py", line 94, in _rewrite
    return parser.rewrite()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 90, in rewrite
    result += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 98, in _rewrite_one
    return self._rewrite_command()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 114, in _rewrite_command
    return macro.rewrite(self)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 54, in rewrite
    return self._execute(parser, invocation)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/bibliography.py", line 62, in _execute
    style_file = self._fetch_style_file(parser, invocation)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/bibliography.py", line 70, in _fetch_style_file
    _, value = each.split("=")
ValueError: not enough values to unpack (expected 2, got 1)

The error is similar to #35 but the traceback is different

Support for KOMA-Scripts

From @gro1729, Issue 22:

I use one of the KOMA-Script packages, where you can define external declarations of style information in files with extension ".lco". I do that in the file "input/sections.lco"

 \documentclass[ * input/sections*, a4paper, foldmarks=true, fontsize=12pt, version=last ]{scrlttr2}

The declaration file is not copied into the "merged"-Folder and not replaced by "sections.lco" inside "merged.tex"

handle bibtex

latexpand has a subcommand to copy the contents of a .bbl and replace the corresponding bibtex command in your .texfile:

latexpand --expand-bbl merged.bbl merged.tex

If you point me in the right direction I am happy to help with this.

Long Filenames in "includegraphics"

Hello, this is a great tool!

Unfortunately, I am not an expert in regular expressions. I have problems with the includegraphics-command. Usually, I prefer to write options in the first line, followed by a comment sign. In the second line follows the name of the image

\includegraphics[interpolate,width=\columnwidth,height=1.\textheight,keepaspectratio]{%
figures/DirectoryOne/ImageTwo}

Additionally, I use "includesvg". The Macro converts SVG-files on the fly into pdf-files. It would be great, if this "include"-state could also be recognized.

Thanks in advance and regards from Andreas

Unable to process due to 'invalid syntax'

When trying to run flap on a fully uptodate debian jessie, it consistently fails with this error report -

python -m flap /home/boudiccas/research/herbal/000herbal.tex output
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/flap/__main__.py", line 19, in <module>
    from flap.ui import main
  File "/usr/local/lib/python2.7/dist-packages/flap/ui.py", line 73
    print(message, file=self._output)
                       ^
SyntaxError: invalid syntax

It makes no difference if I try with python3 or sudo python3 or sudo python it just fails with above error report.

Support for Package `adjustbox`

From @gro1729, Issue 22:

Besides "includegraphics" for instance I use \adjincludegraphics[valign=t,width=.9\textwidth,interpolate]{% ../VeryLongPathName/Pyrit} The referenced image-file is not converted. "merged.tex" contains the same text as the original file.

wrong \input resolution order

15:12 $ flap tex/main.tex compressed -v
FLaP 0.5.0
File                            Line Column LaTeX Command                      
-------------------------------------------------------------------------------
main.tex                           6      1 \input{misc/header.tex}            
main.tex                          13      1 \input{content/abstract.tex}       
main.tex                          14      1 \input{misc/frontmatter.tex}       
main.tex                          15      1 \input{content/introduction.tex}   
main.tex                          16      1 \input{content/sota.tex}           
main.tex                          17      1 \input{content/methods.tex}        
content/methods.tex               17      5 \includegraphics[width=\columnwi...
content/methods.tex               73      5 \includegraphics[width=\columnwi...
main.tex                          18      1 \input{content/results.tex}        
content/results.tex               12      1 \input{"../tables/grid.tex"}       
Traceback (most recent call last):
  File "/home/mq/.miniconda/bin/flap", line 11, in <module>
    sys.exit(main())
  File "/home/mq/.miniconda/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/ui.py", line 102, in main
    Controller(OSFileSystem(), Display(sys.stdout, verbose)).run(tex_file, output)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/ui.py", line 43, in run
    request.execute()
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/engine.py", line 79, in execute
    flattened = self._rewrite(self.read_root_tex, str(self.root_tex_file.resource()))
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/engine.py", line 88, in _rewrite
    return parser.rewrite()
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 78, in rewrite
    result += self._rewrite_one()
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 86, in _rewrite_one
    return self._evaluate_one()
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 138, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 164, in evaluate_command
    return macro.invoke(self)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/macros.py", line 124, in invoke
    return self._execute(parser, invocation)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/macros.py", line 317, in _execute
    return parser._spawn(parser._create.as_tokens(content, link), dict()).rewrite()
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 78, in rewrite
    result += self._rewrite_one()
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 86, in _rewrite_one
    return self._evaluate_one()
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 138, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/parser.py", line 164, in evaluate_command
    return macro.invoke(self)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/macros.py", line 124, in invoke
    return self._execute(parser, invocation)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/latex/macros.py", line 314, in _execute
    content = self._flap.content_of(link, invocation)
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/engine.py", line 118, in content_of
    file = self._find(location, [self.root_directory], ["tex"], TexFileNotFound(None))
  File "/home/mq/.miniconda/lib/python3.6/site-packages/flap/engine.py", line 175, in _find
    raise error
flap.engine.TexFileNotFound: None

The file structure is:

tables
figures
tex
|___main.tex
|___content
    |___results.tex

results.tex has the \input{../tables/grid.tex} command.

It compiles just fine, but I think flap resolves the inner most input first, hence the file cannot be found.

Support for the SVG package (\includesvg)

As pointed by @gro1729 (see Issue 28), the SVG package is not support in Version 0.5.0, especially the \includesvgcommand.

subfiles

Hi,

Sorry I'm adding a few issues that I found when using this recently all at the same time.

It doesn't seem like FLaP can deal with subfiles? It seems to work fine for my \include statements, but not the subfiles. Is this easy to add?

Escaped % signs are eliminated

Hi,

I like the script, but it messes up if there is an % in the file. It will then drop everything beyond the %, although it is part of the document, which is kind of bad. Any way to fix that?

Example from a table:
Original:
1 & 20 \degree C & nno & 0 & 0 % & 0 kWh \
After flattening:
1 & 20 \degree C & nein & 0 & 0 \

Best

The "verbose"-Option does not work any longer

Custom command causes `AttributeError: 'NoneType' object has no attribute 'has_text'`

Hello, I use the following snippet as part of my manuscript:

% Overleaf has an outdated inkscape version, try to work around it:
% Inspired from https://tex.stackexchange.com/a/650621
\def\overleafStripPrefix#1>{}
\def\isoverleaf{FF\fi
        \def\predicate{output}%
        \edef\predicate{\expandafter\overleafStripPrefix\meaning\predicate}%
        \edef\job{\jobname}%
        \ifx\job\predicate
}

Including just that in a tex file is enough to make flap abort.

Backtrace

$ flap main.tex flaped/
FLaP 0.6.0
Traceback (most recent call last):
  File "/home/mcantan/.local/bin/flap", line 8, in <module>
    sys.exit(main())
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mcantan/.local/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/ui.py", line 117, in main
    .run(tex_file, output)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/ui.py", line 43, in run
    request.execute()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/engine.py", line 84, in execute
    tokens = self._rewrite(self.read_root_tex,
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/engine.py", line 94, in _rewrite
    return parser.rewrite()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 90, in rewrite
    result += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 98, in _rewrite_one
    return self._rewrite_command()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 114, in _rewrite_command
    return macro.rewrite(self)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/core.py", line 32, in rewrite
    self._execute(parser, invocation)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/core.py", line 43, in _execute
    self._rewritten_body = parser._spawn(body, dict()).rewrite()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 90, in rewrite
    result += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 96, in _rewrite_one
    return self._rewrite_group()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 105, in _rewrite_group
    tokens += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 96, in _rewrite_one
    return self._rewrite_group()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 105, in _rewrite_group
    tokens += self._rewrite_one()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 98, in _rewrite_one
    return self._rewrite_command()
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 114, in _rewrite_command
    return macro.rewrite(self)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 112, in rewrite
    invocation = self._parse(parser)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 64, in _parse
    self._capture_arguments(parser, invocation)
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 80, in _capture_arguments
    value = parser._evaluate_until(
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/parser.py", line 186, in _evaluate_until
    while not is_excluded(self._next_token):
  File "/home/mcantan/.local/lib/python3.10/site-packages/flap/latex/macros/commons.py", line 81, in <lambda>
    lambda token: token.has_text(next_token._text))
AttributeError: 'NoneType' object has no attribute 'has_text'

I tried to remove that snippet, but I'm encountering another issue. I'll isolate a reproducer and open a new issue.

Image inclusions within a style file are not adjusted

From @gro1729, Issue 22:

In one of my style files I set \setkomavar{fromlogo}{\includegraphics[width=3cm]{images/Logo}} In the resulting "merged.tex" I find the same text. The "Logo.png"-File ist not copied.

Maximum recursion depth exceeded

I am using flap to flatten a fairly standard journal file. When I run the command flap source.tex out.tex I get the following error:

 File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 153, in _evaluate_until
    tokens += self._evaluate_one()
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 138, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 164, in evaluate_command
    return macro.invoke(self)
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/macros.py", line 124, in invoke
    return self._execute(parser, invocation)
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/macros.py", line 150, in _execute
    return parser._spawn(self._body, invocation.arguments)._evaluate_group()
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 146, in _evaluate_group
    tokens = self._evaluate_until(lambda token: token.ends_a_group)
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 153, in _evaluate_until
    tokens += self._evaluate_one()
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 138, in _evaluate_one
    return self.evaluate_command(str(self._next_token))
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 161, in evaluate_command
    if command not in self._definitions:
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 43, in __contains__
    return key in self._definitions or (self._parent and key in self._parent)
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 43, in __contains__
    return key in self._definitions or (self._parent and key in self._parent)
  File "/home/shaunak/anaconda/envs/Python3x/lib/python3.6/site-packages/flap/latex/parser.py", line 43, in __contains__
    return key in self._definitions or (self._parent and key in self._parent)
  [Previous line repeated 77 more times]
RecursionError: maximum recursion depth exceeded

My python version is:

Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux

I have attached the directory structure of the project to this issue. The main file is called 0_Trento_2016.tex. There are image files in the Figures folder.

TREE.txt

\begin{frame}[fragile]

Very often I need "fragile" slides.

According to my "philosophy" : "One slide, one file"

"fragile"-slides do not have to have an identation. "flap" ignores my indentation-guidelines. This is a very serious problem.

Font files are missing

From @gro1729, Issue 22:

I have to use Fonts of my company. Thus I have to compile with xetex or luatex. A Font-Declaration is defined by:

 \setmainfont[ 
Path = ../Fonts/Fontin/ttf/, 
Extension = .ttf, 
Ligatures = TeX, 
BoldFont = Fontin-Bold, 
ItalicFont = Fontin-Italic, 
SmallCapsFont = Fontin-SmallCaps ]
{Fontin-Regular} 

None of the Fonts is copied into the "merged"-Directory