davidfarmer/latextolatex

Utilities for converting LaTeX markup to a standard form

latextolatex's Issues

Huge output on terminal when pretty-printing via a script

ltol.py runs fine on the sample article and produces a pretty version, but always produces near the end:

/////////////problem with nested environment

But if I run it in my script, I get a huge dump of pseudo-XML that starts and ends as follows. Not a huge problem, and maybe I just need to pipe stderr or something to /dev/null, but thought I'd post here.

found {'task': 11, 'figure': 176, 'p': 1188, 'li': 376, 'ul': 40, 'ol': 56}
found {'task': 11, 'figure': 176, 'p': 1188, 'li': 376, 'ul': 40, 'ol': 56, 'sidebyside': 111}
problem with nested environment: pretext >
ACOMMBc64cb7170dd0c1e9d12db4044d9b0e7d8385ad51ENDZACOMMB39fe7f36433391195ec07e10e87ef7ea92d5faecENDZACOMMB25ee7dbf65546c6f893abc2f5d659250b25adea8ENDZACOMMB819664fe948bff5c40e5ad413ad35554d505aedfENDZACOMMB8e1876df8c25a98a670798d684ce91e8f864b238ENDZ

<docinfo>
  ACOMMB403ce0705ca903d374b80e5243a921995a0f81cbENDZ
  <brandlogo url="https://pretextbook.org" source="images/book-cover-google-art-project.jpg" />ACOMMBc0bee3797a45c30324c4764ad0a092091dd36fbfENDZAlatex-preambleB2f2f5b2b10e5531c3a8a8e84fd89a3bef0fd5ab2ENDZACOMMB15c48420aa57bed678996614f037833b354a43a5ENDZAmacrosBf0752bf87cff27230fd7059f2a750ede6dfcaf12ENDZACOMMB045122ce249271bc9bfe574329ad69bdc933d1a7ENDZACOMMB2de75fb9c837adeea04a83e7cdc1b16c6b417600ENDZACOMMBa48285586cd8b5f2050335f24b2243a0025fb229ENDZACOMMBf6784d5c694bed72412074369dad1ccbe8ef9e9cENDZACOMMBca18cc0a9f8eea594eeeea2abedc8e3f53254b5cENDZACOMMB5458808cda8949a8280d069c68add6441bddcf9dENDZACOMMB73401307dfbf47f9e8656c7ee35cf369138b7284ENDZACOMMB361198506b37a591e48dae8a20c261514ba2c8b2ENDZ
  <cross-references text="global" />ACOMMB82958dfaeacbf7627cdb23bf414e041156af65d2ENDZAlatex-image-preambleB6fd3dd9ff8b38680f2036239b0cc5cc060f1e3c1ENDZACOMMBa8db330b7d41c596007c3ef335dc57cb5c878f8dENDZ
  <initialism>SA</initialism>
  ACOMMBd27fecb97d3161dbab9e8c0f5cd59b9b57edc9bcENDZACOMMBa433ba29ec414b8c8cd191d40ca1b90f1d454b8dENDZACOMMBbde6d24dd0a7ab8535ea36cc95e4401e37b5dad3ENDZACOMMB433219e03ca21bfc73e656bd59aaa65d0df83d61ENDZACOMMB57e9cabbc235ff597a8d7507e5862e6cec44610fENDZACOMMB711f499f2b95158b56923f5c24f1df22adc70f91ENDZACOMMBad63964a9e255f219c9f162ece3a480d72d5abe6ENDZACOMMB754eac4118c7168a8d013242fcad1d9dc2bdbdb1ENDZACOMMB8f26105a6808453a2f24185de3a0587e1b8d7a64ENDZACOMMB57e9cabbc235ff597a8d7507e5862e6cec44610fENDZACOMMB7e5010ee870dcc9dfc5e1d667482ab3e9e027114ENDZACOMMB37275a332827bba518ae15c7ae4509fe09d56f6fENDZACOMMB40e3eee15c545b2777caf320f6b400700bdd8019ENDZ
  <document-id>pretext-SA</document-id>ACOMMB05a1004c509c8b03d865c3c6c9054c29dcc81f6dENDZ


[LOTS OF SIMILAR DELETED]

    <colophon>
      <p1134>
        This article was authored in, and produced with, <pretext />.  It is typeset with the Latin Modern font.
      </p1134>
    </colophon>
  </backmatter>
</article>

/////////////problem with nested environment

Subsequent rounds of permid

A requested issue to suggest making it automatic to add new permids when some permids have already been used. (Rather than editing the code to set a new start id reported from the output of a previous run.)

Nested transclusion

Keep track of what has been transcluded, so that a circular self-reference isn't expanded forever.

Upgrade to Python 3

Python 3 fails due to a print statement that needs parentheses. It might be as easy as chasing down all the print statements to allow Python 3 to succeed. I'm aware of tools that do this automatically ("2to3" or similar), but I have no experience with them.

Pretty-printing a PreTeXt "cd" element

Similar to the c element,

<cd>foo</cd>

is verbatim and all whitespace is meant to be significant. It is rendered as a display. So the pretty-printed version introduces extra whitespace.

It can be structured with the line element. So the following is possible:

<cd>
  <line>foo</line>
  <line>bar</line>
</cd>

and might ohnly contain a single line element. So cd is much like me and cd\line is like md\mrow except we can't mess with whitespace that might be significant in context. Search sample article for kng2 (case-insensitive) for an example of the first form.

I did not see anything similar with pre but it is in the same spirit.

No Intro's in Explorations

The most recent version seems to have missed including introductions into the explorations when there is content before the first task.

Pretty-printing a PreTeXt "c" element

A "c" element is inline verbatim, and there is no way to manipulate whitespace in any way. So

<c>foo</c>

pretty-printed as

<c>
  foo
</c>

will introduce (unwanted) newlines and whitespace that will be reproduced by the output format's verbatim mechanisms. In particular, a long c should not be line-broken. Search on "impinge" in the PreTeXt sample article for a long example.

I built HTML output for the sample article. Then pretty-printed the sample article and built HTML output. Then did a git diff with --word-diff which will not show all the double spaces at sentence-end which become single spaces.

XML comments

Judson's AATA had:

<p>Let <m>X</m> be a finite <m>G</m>-set and <m>X_G</m><!--NOTATION TABLE\label{noteXG}--> be the set of fixed points in <m>X</m>;

When pretty-printed, the space after the XML comment was removed, leaving no space between the m and the word "be".

PreTeXt does nothing with XML comments, the xsltproc processor just drops them. Unclear to me what the best rearrangement is. Putting mid-paragraph comments on their own line may give rise to some of the "extra" spaces we are seeing around idx and notation.

For this isolated case, I'm just moving this comment out of the paragraph.

Empty XML elements

I read an "expert" who said empty elements should be written as <foo />. Then another "expert" told me the space looked really odd and it should be <foo/>. Could/Should pretty-printing correct my initial bad habit that others have picked up?

pg to mbx: opertations inside variables

The original PG source will often include things like [$a*$b] inside
the PGML block. It would be a huge time saver if these things could
become , where $name is something auto-
generated, not already used in the problem. And then
$name=$a*$b; would get added to the end of the pg-code element.
$name would also have to become one of the vars at the top of the
setup element, with a static ready to be entered.

From MultiplyTrailingZero10.mbx
We will ignore those trailing zeroes, and do < m>< var name="$a" />\cdot< var name="$b" />=[$a*$b].

            Next, we will add back < m>[$a1Exp+$b1Exp]</m> trailing zeroes,

Error message clarity

My mistake, mixed a file for input and a directory for output, since I was cribbing from my website build script and not paying close attention to variables in the script:

$ ~/mathbook/website/repos/LaTeXtoLaTeX/ltol.py xml_pp ~/mathbook/mathbook/examples/sample-article/sample-article.xml .
/home/rob/mathbook/mathbook/examples/sample-article/sample-article.xml
.
component.current_permid 123
Not proper input.  Does target directory exist?

But "target" makes me think of output and that is the only directory I specified. But it also says "input". So briefly confused.

Maybe say "file + file or directory + directory only, you've got them all mixed up"?

Thin-space comma gets detached from its escape character

Source (Judson's private solutions):

Noting that <m>\omega^3 = 1</m> and <m> (1 - \omega)^3 = -3(\omega - \omega^2) = -3 \sqrt{3} \, </m>,  we can further conclude that

appears to become, via ltol.py xml_pp,

Noting that <m>\omega^3 = 1</m> and
<m> (1 - \omega)^3 = -3(\omega - \omega^2) = -3 \sqrt{3} \</m>,, we can further conclude that

Then the conversion to LaTeX produces \\),, which does not compile because the final \) is not found, due to the \\ being interpreted first.

In the middle of convoluted rearrangements of AATA source, but I'm pretty sure I've got this described accurately.

(Yes, the source is bad, but maybe swapping the trailing punctuation out is a bit over-aggressive?)

Suggestions for pretty-printing PreTeXt "cd"

Follow-on to #16.

OK, cd is structured with cline (not line). The cline part is verbatim and cannot be massaged. Suggestions for pretty-printing, which are suggestive of how the output will render.

For single-shot cd

blah blah
<cd>verbatim<cd>
foo bar

For cd/cline, mimick md/mrow,

blah blah
<cd>
  <cline>verbatim</cline>
  <cline>verbatim</cline>
<cd>
foo bar

indent code inside latex-image

image\latex-image will have (text) lines of LaTeX code. Do you want to indent it to line up with surrounding indentation?

PreTeXt will remove this indentation when creating output.

Line-break after sentence, before parenthetical remark

Should pretty-printing make a line-break inside "period whitespace open-paren"?

There are 15 of these in pretty-printed AATA where a sentence ends and a parenthetical remark begins. There are 5 more that are answers to multi-part questions, so like

(a) 42.  (b) Don't panic.  (c) Bring a towel.

One "caution" begins with a square bracket, [.

Empty/Error'd Conclusions

The script still adds empty conclusion tags.
Actually, several of them are worse than empty, they've got an incorrectly placed < /p > in them.

Pretty-printing index entries and notation

The PreTeXt idx and notation elements are meant to be placed anywhere in a paragraph p. They have an obvious purpose, but never render anything the reader sees where they are located. So invisible, in a sense.

Whitespace, before and after, could be significant, such as a space between words. So I cannot see a 100% correct strategy for sanitizing its surroundings.

When pretty-printed on a line of its own, especially at the start or end of a p introduces whitespace before and after. One of these will be stripped (before at p start, after at p end). The other is more problematic.

I feel like these need to keep the location an author has assigned, but despite a whole lot of thought, can't be certain there isn't a way out.

Respect indentation for multi-line verbatim text

PreTeXt supports multi-line "verbatim" input, with the intent to respect indentation. For code that needs to actually needs to run (e.g. Python in a program for Runestaone use, Sage in a sage for the Sage cell). But also for purely visual uses, like a pre.

Pretty-printing seems to be removing this. Reviewing sample article output from

ltol.py xml_pp mathbook/examples/sample-article/ .

• c leads to some long lines, but they are partly testing artifacts, all OK
• cd, w/ and w/out cline structure, look great. Indented consistent with surroundings.
• pre and pre/cline entirely flush left, both elements and multi-line bare text
• program/input entirely flush left (both elements, and code), indentation clobbered
• sage/input entirely flush left (both elements, and code), indentation clobbered

PreTeXt has a template to strip "common" leading whitespace from multi-line bare text. For pretty-printing purposes, I'd think you want to mimic most/all of this, and then add in the current indentation as whitespace? At least see documentation of effect:

https://github.com/rbeezer/mathbook/blob/512bf5d956f01262bf245f8cf45d8607e0f3a54e/xsl/pretext-common.xsl#L2660

Or maybe you know what current indentation is and believe source is respecting that for multi-line text and you can just work off the delta between author's indentation and destination/pretty-printed output.

I think AATA Sage code is still a bit of a mess from this. But mostly the code is one-liners so not too much had to be re-indented to run. But for an author with lots of Python/Sage, this would be a problem.

And I forgot about asymptote, sageplot, latex-image which could be entirely similar. Some of that, and some of this is on #23.

Period at end of math may need to remain

AATA source:

<me>\sigma_i( x ) = 
\left\{\begin{array}{ll}
\sigma( x ) &amp; x \in X_i \\
x   &amp; x \notin X_i, 
\end{array}\right.</me>

Result of pretty-printing:

<me>
  \sigma_i( x ) = 
  \left\{\begin{array}{ll}
  \sigma( x ) &amp; x \in X_i \\
  x   &amp; x \notin X_i, 
  \end{array}\right
</me>.

LaTeX fails as the \left is missing a proper \right.

Yes, it would be better with \cases. Only happened twice, and am now building the full text as a PDF via LaTeX.

allpermid.txt in output

Placing permid's onto source creates allpermid.txt in the output directory.

Would it be wise to figure out some way to make it less likely that an author gets it into their repository and mistakenly commits it?

Perhaps use a flag/option that will cause it to be created, so the default is not to?