omikhleia / markdown.sile Goto Github PK
View Code? Open in Web Editor NEWNative Markdown and Djot support for the SILE typesetting system
License: MIT License
Native Markdown and Djot support for the SILE typesetting system
License: MIT License
As a reminder when sile-typesetter/sile#1821 makes it one way or another:
We could support appropriate attributes on Markdown and Djot display math syntax to benefit from it.
E.g. in Djot,
$$`formula`{ .numbered }
$$`formula`{ numbered=true }
$$`formula`{ .numbered counter=mycounter }
(Or similar ideas)
Hi Didier (@Omikhleia),
In the past few days, I've been working on installing both the SILE Typesetter and "markdown.sile" in the Windows Subsystem for Linux (WSL). Through trial and error, I was finally able to get it installed in Ubuntu by working through the following steps. I would be interested to know if I just got it working by accident, or might there be a better way of doing this?
In Section 2.7 of the SILE manual (Installing third-party packages), it mentions checking the version of Lua that SILE uses, so that Luarocks will install the correct version. Since I wanted to install locally, I used:
luarocks --lua-version 5.2 install --dev markdown.sile
However, there were problems downloading the "rockspec" file, so I did it manually using:
wget https://luarocks.org/dev/markdown.sile-dev-1.rockspec
I then tried this command, but it gave an error about Lua header files:
luarocks install --dev --local markdown.sile-dev-1.rockspec
Cloning into 'markdown.sile'...
remote: Enumerating objects: 722, done.
remote: Counting objects: 100% (211/211), done.
remote: Compressing objects: 100% (119/119), done.
remote: Total 722 (delta 106), reused 175 (delta 88), pack-reused 511
Receiving objects: 100% (722/722), 527.47 KiB | 2.14 MiB/s, done.
Resolving deltas: 100% (364/364), done.
Error: Failed finding Lua header files. You may need to install them or configure LUA_INCDIR.
The article Error: Failed finding Lua header files. You may need to install them or configure LUA_INCDIR. - Windows 10 with Lua distribution, mentioned that Luarocks could not find "lua.h", "lualib.h" and other related files.
To find these on my system, I installed locate (sudo apt install locate -y
) and updated the database (sudo updatedb --prunepaths='/mnt'
) and then updated the LUA_INCDIR as follows:
luarocks-5.2 config variables.LUA_INCDIR /usr/include/lua5.1 --local
However, the installation still failed. Running the Luarocks command by itself (luarocks
) to get the configuration information, was showing the same error for the "include" directory, even though the folder existed and "lua.h" was present:
LUA_INCDIR : (not found)
I installed Lua 5.2 (sudo apt install lua5.2
), but this did not create any further "include" folders. The article Lua 5.3 is installed, but I can't locate the correct lua.h, however, gave a pointer in the correct direction.
I installed liblua5.2-dev and updated the "locate" database:
sudo apt install liblua5.2-dev
sudo updatedb --prunepaths='/mnt'
and then checked for further "include" folders:
locate include | grep lua
This gave the following:
/usr/include/lua5.1
/usr/include/lua5.1/lauxlib.h
/usr/include/lua5.1/lua.h
....
/usr/include/lua5.2
/usr/include/lua5.2/lauxlib.h
/usr/include/lua5.2/lua.h
....
Updating the LUA_INCDIR (luarocks config variables.LUA_INCDIR "/usr/include/lua5.2" --local
) gave the following configuration, with the "include" directory finally being recognised:
Lua:
Version : 5.2
Interpreter: /usr/bin/lua5.2 (ok)
LUA_DIR : /usr (ok)
LUA_BINDIR : /usr/bin (ok)
LUA_INCDIR : /usr/include/lua5.2 (ok)
LUA_LIBDIR : /usr/lib/x86_64-linux-gnu (ok)
This allowed the installation to work correctly (luarocks-5.2 install --dev --local markdown.sile-dev-1.rockspec
):
Cloning into 'markdown.sile'...
remote: Enumerating objects: 722, done.
remote: Counting objects: 100% (211/211), done.
remote: Compressing objects: 100% (119/119), done.
remote: Total 722 (delta 106), reused 175 (delta 88), pack-reused 511
Receiving objects: 100% (722/722), 527.47 KiB | 1.70 MiB/s, done.
Resolving deltas: 100% (364/364), done.
Missing dependencies for markdown.sile dev-1:
embedders.sile (not installed)
labelrefs.sile (not installed)
ptable.sile (not installed)
smartquotes.sile (not installed)
textsubsuper.sile (not installed)
markdown.sile dev-1 depends on lua >= 5.1 (5.2-1 provided by VM)
markdown.sile dev-1 depends on embedders.sile (not installed)
Installing https://luarocks.org/dev/embedders.sile-dev-1.rockspec
....
....
markdown.sile dev-1 depends on textsubsuper.sile (not installed)
Installing https://luarocks.org/dev/textsubsuper.sile-dev-1.rockspec
Cloning into 'textsubsuper.sile'...
remote: Enumerating objects: 56, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 56 (delta 23), reused 42 (delta 15), pack-reused 0
Receiving objects: 100% (56/56), 47.15 KiB | 928.00 KiB/s, done.
Resolving deltas: 100% (23/23), done.
textsubsuper.sile dev-1 depends on lua >= 5.1 (5.2-1 provided by VM)
textsubsuper.sile dev-1 is now installed in /home/<user>/.luarocks (license: MIT)
markdown.sile dev-1 is now installed in /home/<user>/.luarocks (license: MIT)
As I said, this was trial-and-error over several days. I wasn't always consistent with the commands and the Lua versions that I used, so I may have been giving myself extra work to get it installed locally.
Some final questions: might there have been some checks that I could have run earlier to find out what was missing? Since I am not familiar with the dependencies between Lua versions, or much about the Lua ecosystem, are there some assumptions that needed to be corrected?
Kind Regards,
Liam
Unicode Line Breaking Algorithm:
... then expanding or compressing interword space according to common typographical practice, only the spaces marked by U+0020 SPACE and U+00A0 NO-BREAK SPACE are subject to compression, and only spaces marked by U+0020 SPACE, U+00A0 NO-BREAK SPACE, and occasionally spaces marked by U+2009 THIN SPACE are subject to expansion. All other space characters normally have fixed width.
The "nbsp" is therefore intended to be stretchable.
On the parsers' side of things.
 
(or \
with appropriate/default extension options) to U+00A0 in the Str AST node. 
to U+00A0 passed to the writer.string() method.\
invokes a dedicated nbsp AST node (which we expanded to U+00A0 but see below).On SILE's side, it doesn't seem the typesetter / shaper / whatever involved1 considers U+00A0 as shrinkable/stretchable for the purpose of justification... Bah! Regardless, we can filter them in our various inputters, and make the necessary adjustment.
In Djot, moreover, since any content can have attributes, we may accept, e.g. \ {.fixed}
for a fixed inter-word space, if need be.
IMHO, the SIL and XML inputter should actually be responsible for that, not the typesetters/shapers, etc. = i.e. it's perhaps best regarded as something that should be done at AST level. But heh, the SIL inputter doesn't even split paragraphs, currently, it's done by the typesetter for a part (see typesetter.parseppattern
setting, &c). Perhaps a debatable separation of concerns issue... ↩
Converting a markdown or djot document and using the resilient.book class with default styling
Workaround:
Tweak the style file of the document:
table:
origin: "resilient.book"
style:
paragraph:
after:
vbreak: false
align: "justify" # was "center"
before:
indent: false
Analysis:
It has to do with the table being 100% of the line, and the "center" (idem for "left", "right") adding lskip/rskip glues around it.
I'm not sure exactly why and if this point towards something deeper.
Currently we "normalize" languages to a 2-letter code.
Whatever happens on the SILE front for sile-typesetter/sile#1641
Let's rely on silex too, and remove the normalizeLang
workarounds we have here and there in the code.
So for instance fr and fr-CH get both correct smartquotes and still load correct hyphenation patterns etc.
As reported privately, e.g. { width="50%lw" }
or { width="2.5cm" }
on an image fail to be parsed.
--> The general issue is that attribute syntax is too restrictive (my bad) - Cf. jgm/lunamark#62
I am not sure I can make sense of Pandoc's Haskell code for these, but at least we could be more tolerant in our vendored copy of lunamark for now, so as to work-around this annoying restriction.
MkDocs' Markdown has "admonitions" as a syntax extension.
While quoting the "title" seems a bit idiosyncratic, we could support something similar:
!!! keyword Inline content for title
4-space indented block contents...
Compare to the "fenced divs" syntax, it allows for something simpler for environments needing two arguments (where Markdown can be used in both). I mean, it's doable with fenced divs, but it looks a bit "clumsy" / less straightforward in my opinion, and also makes the task of the writer afterwards less obvious for identifying and extracting the necessary bits.
::: { .keyword }
block contents...
:::: { .title }
Content for title
::::
:::
I would tend to think it would bring a more natural way to typeset some things, with an immediate sensible use -- as shown in the PoC below.
I wonder what other writers using Markdown would have to say (or how they'd address that type of need with other means).
Currently we only support a "naive" syntax highlighting for Lua.
See sile-typesetter/sile#1484 for a broader discussion.
Warning about duplicate identifiers errors when a header occurs in a block - e.g. a Blockquote, but more realistically in a Div used for styling -- and then internal links do not point to the correct element.
It's a misunderstanding on how the Djot section AST node behaves: We currently take the identifier to use on the rendered header from the previously seen section, ignoring a possible identifier on the header AST node itself.
However:
Note by the way that with our version of djot.lua, other attributes are always on the header (differing from djot.js and the online Djot playground at the current date)
See also jgm/djot#213 for reference.
I could possibly see interesting uses for Pandoc's yaml_metadata_block
extension, would it be supported. We'd first have too define/restrict which bits of such a metadata block to use. It also perhaps needs to distinguish between documents processed from the command-line (where one could want to define, e.g., paper size, main fonts, PDF metadata etc.) from documents \include
'd in other documents (where such choices would likely have to be ignored, but where other settings could be however still needed, e.g. additional fonts and fallbacks)
I experimented with an AST to SIL reconstruction (see also #15) with the purpose of helping writing tests, e.g. "busted" such as those initiated in file inputters/markdown_spec.lua
. It would be nice to have a way (e.g. GitHub action?) to automate the testing:
The current implementation for "highlight" a.k.a. "mark" (that is, =inline=
in Markdown, {=inline=}
in Djot -- additionally also supported as a .mark
pseudo-class) , is just a placeholder (hard-coded to show the text in red, with a big FIXME TODO We'd need real support
in the code.
What I'd want here is:
develop
branch for 0.15 and likewise in sile-x.Currently [some link](url){ #id .xxx key=value }
is supported, but only a subset of the attributes is used (= the id
, if present, and the pseudo-classes for cross-references via empty links, e.g. .page
etc.).
Other attributes ought to be wrapped onto the content and be applied, so that e.g. [some link](url){ .underline custom-style=MyStyle }
would do the expected things...
A workaround is to wrap the link in another span [[some link](url)]{ .underline custom-style=MyStyle }
. It works, but it's ugly :)
The current implementation of fenced divs and fence code blocs in lunamark has issues:
jgm/lunamark#44 jgm/lunamark#46 jgm/lunamark#48 jgm/lunamark#49
"See chapter N, section N.M on page P"
We could use our "labelrefs.sile" package for such things, that would be handy for books/articles needing usual cross-reference links.
However Pandoc never seems to have reached a consensus on how this should be declared in (extended-)Markdown. We could come up with our own syntax extension, but that wouldn't be portable. To be investigated.
Currently the native markdown inputter enables all supported extensions before invoking the lunamark reader/writer.
IIRC, SILE's \include
can accept extra options and pass them to the inputter, so the user could have a way for disabling extensions they do not want for some input file.
Markdown has several ways for specifying an horizontal rule: "A line containing a row of three or more *
, -
, or _
characters (optionally separated by spaces) produces a horizontal rule".
Both Pandoc and Lunamark handle all of these identically - resulting in an undifferentiated horizontal rule. And unless I missed it, Pandoc doesn't have an extension to "style" the rule.
I intend to use Markdown for book-like chapters, and no-one uses a bare "full rule" in real books. But overriding it with a single appropriate "separator" design is not fully satisfying either: books may use different rules for different purposes. E.g. it's common to have a centered dinkus * * *
(or an asterism ⁂
) within a chapter, and a fleuron (or a combination of a fleuron and line-shaped dinkus) at the end of a chapter.
A few variants, on the other hand, might be fairly sufficient for reasonable use, say up to 3 (more tends to be bad practice, as for sectioning commands, were more that 3-4 levels is often frowned upon). In other terms, Markdown offers enough flexibility for different interpretations, and the issue is "just" that the above-mentioned converters/readers cancel this without passing the information to the rendering engine ao as to afford customization points.
It would be quite easy to slightly change Lunamark's parsing logic to return the separator(s), so the user-application could take a decision (e.g. render ***
as an asterism, * * *
as a dinkus, - - -
as an end-of-chapter pendant, while keeping ---
for the original basic rule, etc.)
It would be a departure, though, from what Pandoc's AST supports... But heh! Perhaps we can be better here, for the purpose of print-quality books.
I'm interested in any feedback (or other ways it could be addressed).
Lunamark only implements a subset of extension link_attributes
(and friends).
Leveraging Lunamark for full support would allow more things than currently supported (e.g. links to images, etc.)
If LuaJSON is installed from luarocks as follows, at this date (= installing latest 1.3.4-1 at this point)
luarocks install luajson
Then pandocast fails to process files as LuaJSON fails to be loaded: It is broken due to an improper check on the version of LPeg --> See harningt/luajson#47
Workaround:
Install the dev version.
luarocks install luajson --dev
Both pandocast and markdown support smart single and double quotes, but they replace "
by English quotation marks.
Some languages have different preferences for quotation marks, see e.g. https://en.wikipedia.org/wiki/Quotation_mark#Summary_table
This should perhaps be implemented, i.e. using appropriate single or double quotation marks depending on the (current) language
E.g.
::: { lang=de }
Lorem "dolor"
:::
Should output Lorem „dolor“
rather than Lorem “dolor”
This could really improve typesetting (as those characters are not easily available from the keyboard, and this is the very rationale behind "smart" quotes).
Caveat: Some languages allow more than one way, and country code (as used by SILE, e.g. fr
) are not always sufficient.
Some issues with SILE 0.15:
gcc
etc.)mkdir cosmo; touch cosmo/init.lua
in your working directory, this will make require('cosmo')
happy, and since it's not used in our package, that ought to be fine.Opened further to discussion #34 (reply in thread)
With the resilient.book class, one can do in SIL-language:
\begin{figure}
(some figure with a numbered caption)
\caption{yyy}
\end{figure}
\begin[numbering=false]{figure}
(some figure with an unumbered caption)
\caption{yyy}
\end{figure}
In markdown, I would naively have expected the following to do the same thing:
![some figure with numbered caption](examples/images/someimage.png){ width=5cm }
![some figure with unumbered caption](examples/images/someimage.png){ .unnumbered width=5cm }
Apparently I forgot it and we do not propagate the necessary options ;)
$ sile examples/sile-and-markdown-manual.sil -t
SILE v0.14.6.r8-g771d87f-dirty (Lua 5.4)
<examples/sile-and-markdown-manual.sil> as sil
[1] [2] <examples/sile-and-markdown.md> as markdown
[3] [4] <./packages/markdown/commands.lua:304> as sil
<./packages/markdown/commands.lua:304> as sil
[5] <./packages/markdown/commands.lua:304> as sil
[6] ! Underfull frame: 116.55325020005pt stretchiness required to fill but only 72pt available at:
(...)
[7] <./packages/markdown/commands.lua:304> as sil
[8] <./packages/markdown/commands.lua:304> as sil
[9] <./packages/markdown/commands.lua:304> as sil
! Document has more than one parent node that looks like a master document! at:
./packages/markdown/commands.lua:304: in <snippet>:
[[For instance, this \em{entire} sentence is typeset in a \em{raw block}, in SILE language.]] near examples/sile-and-markdown.md:0:0: in \markdown:internal:rawinline[format="sile"]
examples/sile-and-markdown.md:0:0: in \markdown:internal:rawinline[format="sile"]
examples/sile-and-markdown.md: in \markdown:internal:paragraph
examples/sile-and-markdown.md:0:0: in \markdown:internal:rawblock[format="sile"]
examples/sile-and-markdown.md: in \texlike_stuff
examples/sile-and-markdown.md: in <snippet>:
[[# SILE and Markdown::: {custom-style=raggedleft}"Markdown is intended to be as easy-to-read and e]]
examples/sile-and-markdown.md: in <snippet>:
[[# SILE and Markdown::: {custom-style=raggedleft}"Markdown is intended to be as easy-to-read and e]]
examples/sile-and-markdown-manual.sil:58:1: in \include[src="sile-and-markdown.md"]
examples/sile-and-markdown-manual.sil: in <snippet>:
[[\begin[class=resilient.book]{document}\use[module=packages.autodoc]\use[module=packages.barcodes.e]]
stack traceback:
[C]: in function 'error'
/usr/local/share/sile/core/utilities.lua:39: in function 'core.utilities.error'
/usr/local/share/sile/inputters/sil.lua:211: in function 'inputters.sil.parse'
/usr/local/share/sile/inputters/base.lua:47: in function 'inputters.markdown.process'
/usr/local/share/sile/core/sile.lua:300: in function 'core.sile.processString'
./packages/markdown/commands.lua:304: in field '?'
/usr/local/share/sile/core/sile.lua:391: in function 'core.sile.call'
./packages/markdown/commands.lua:320: in function <./packages/markdown/commands.lua:319>
(...tail calls...)
(...)
error summary:
Processing at: ./packages/markdown/commands.lua:304: in <snippet>:
[[For instance, this \em{entire} sentence is typeset in a \em{raw block}, in SILE language.]] near examples/sile-and-markdown.md:0:0: in \markdown:internal:rawinline[format="sile"]
Using code at: /usr/local/share/sile/inputters/sil.lua:211: Document has more than one parent node that looks like a master document!
In 1.2.0 and earlier, we tried to play fair with the default book class from SILE, and when processing a sectioning command (chapter, section, etc.), we insert the identifier label for cross-referencing just after it.
A comment in the code did note that the solution is hardly satisfying:
Somewhat messy. If done before the sectioning, it could end on the previous page.
Within its content, it breaks as TOC entries want a table content, so we can't use a function above...
We are left with doing it after, but that's not perfect either vs. page breaks and indent/noindent...
In the resilient.book class, I added a marker option to sections and reimplemented that part, but here we work with what we have...
Or to re-explain it here:
\label
--> a \pdf:destination
hbox + an \info
hbox)The latter is visible with the resilient.book class when running with -d resilient.styles
in (future) v2.0.1:
The solution adopted by the resilient classes was to add a marker
option to the sectioning commands (e.g. \chapter[marker=id]{Title}
), which inserts the identifier just before the title, but after the latter was extracted for the ToC, headers, etc.
Had we used this solution here, the problem would have been solved:
... But we can't do that currently with SILE's book class.
Implicit figures have apparently changed in pandoc 3.1 (based on pandoc-types 1.23), generating something such as:
[ Figure
( "" , [] , [] )
(Caption Nothing [ Plain [ Str "text" ] ])
[ Plain
[ Image
( "" , [] , [] ) [ Str "text" ] ( "uri" , "" )
]
]
]
.... Instead of a Para with an Image.
I haven't checked yet -- it might relate (or not) to the implicit_figures
extension.
We'd need to check and possibly update the pandocast inputter to support the change.
Because we can: https://github.com/Omikhleia/piecharts.sile
As mentioned in #56, and more generally, so far the markdown/djot packages have tried to play fair with a SILE setup.
So for ordered and unordered lists, we load the (SILE) lists package.
However, when used with the resilient classes, we rather ought to use the resilient.lists package and benefit from its features, incl. styling...
Some Markdown converters recognize fenced code blocks of a certain type (infostring/class) as graph drawing language and renders them visually.
For the record, my previous experiment with a Pandoc custom Lua writer did that too, but only when certain attributes are set:
All the above examples (= of fenced code blocks) specified the programming language after the fenced
code block marker (e.g. lua ). For the DOT graph language, this converter also supports an extended syntax{.dot width=... height=...}
. When a width and/or a height are specified, the graph is included as an image, instead of the corresponding code.
Things to address
dot
is certainly subpar).Djot (https://djot.net/) is John MacFarlane's experimental "light markup syntax", derived from commonmark, but fixing most of the complex syntax pitfalls, yet with more flexibility on some aspects (e.g. attributes everywhere!)
It has an experimental Lua implementation (https://github.com/jgm/djot.lua).
It sounds pretty cool:
A real quick attempt this week-end shows that its use is quite straightforward (reusing most of the commands for Markdown processing), so I might push it here at some point (after some cleanup and experimentation, and adding to the quick experiment some of the important missing bits, e.g. tables!)
Whether in Djot or Markdown, for key-value attributes used as dimensions, we currently state "that any unit system supported by SILE is accepted"
Hence, e.g. ![](someimage.png){height="4cm"}
An implication, though, is that we need to use width="50%lw"
, or height="50%fh"
for frame-relative sizing... This is not very user-friendly, and probably not what most users would expect.
At our level, we can't know whether those attributes will eventually end up as SILE measurements/lengths. Of course, it could be better if SILE did it natively, but that's a bunch of packages to address (such as image and svg in the core distribution, my own embedders, ptable etc. -- and probably others I overlooked here...)
A proposal would we to:
width
(note the "lw" rather than "fw" -- I'd tend to think this is the most correct default interpretation)height
You can't make an omelette without breaking eggs
After struggling with making silex.sile v0.4 less aggressive and not replace SILE's internals globally... I realize that markdown.sile actually enforces the replacements indirectly...
This is due to the feature check it does, loading the resilient class (which enforces all of silex):
markdown.sile/packages/markdown/commands.lua
Lines 154 to 155 in 1a5411f
So even processing a document using the bare "book" ends up loading the silex replacements at some point (and possibly in the course of a document processing, which then causes havoc)
We'd possibly need to do those checks differently, there's some wrong inversion of logic.
Commonly used in Markdown, though.
Bibliography citations are both supported by lunamark (in the native approach route) and Pandoc (with the Pandoc AST route) -- The latter possibly taking advantages of some "pandoc-filter".
SILE has a basic bibTeX bibliography support, but I haven't investigated yet if it could be used here. See comment below...
As for captioned-environments, definition list should likely rely on class or package provided envs if available, and use the current implementation as fallback only.
Thoughts from sile-typesetter/sile#1866 (comment)
Currently, the SIL case (Some SIL stuff \raw[type=markdown]{Some Markdown}
) won't lead to a single paragraph (as the markdown/djot parsers generates a surrounding paragraph).
But likewise, we cannot have e.g. Some Djot stuff `Some Markdown stuff`{.markdown}
We need to do something different whether we are already in horizontal mode or not, and the underlying parser/inputter would likely need to restrict the possibilities to inline elements only.
jgm/lunamark#69 fixes an issue with references (indirect links and footnotes) in fenced divs.
We should backport it to our custom vendored copy of lunamark
Missed it in the initial implementation - actually I didn't thought at all to the possibility - but Djot obviously allows attaching attributes to the footnote reference. This would be super useful, e.g.
... footnote call[^djot-fun-note]...
... See note [](#id)
{#id mark="†"}
[^djot-fun-note]: Footnote content
The REAME currently states, in the pandocast usage section:
Prerequisites: The LuaJSON module must be installed and available to your SILE environment. This topic is not covered here.
This ought to be a package dependency, without causing an impediment to the user.
From what I recollect, I had issues with non-"sile" scoped dependencies in 0.14.0. To be checked again with 0.14.4 or upper, maybe I was initially wrong too.
Or else just a patch application. :-)
SILEX -u inputters.markdown -u packages.autodoc examples/sile-and-markdown.md
...
! This isn't a SILE document! at examples/sile-and-markdown.md: in <snippet>:
[[# SILE and Markdown::: {custom-style=raggedleft}"Markdown is intended to be as easy-to-read and e]]
...
Relates to sile-typesetter/sile#1637
When the packages are used with the resilient classes, the custom-style
should allow referring to a style name (and not only a SILE command). That would allow better integration with the resilient framework and styling paradigm.
Djot input:
i. ddd
i. ddd
Observed = interpreted as an alpha list:
i. ddd
j. ddd
Workaround to get roman numbers:
i. ddd
ii. ddd
Nevertheless we need to prioritize roman numerals over alpha-numbering when ambiguous: djot.js was already fixed that way.
See jgm/djot.lua#7 (not yet merged at the time of writing) --> Check if it does the trick and consider backporting it into our vendored local copy of djot.lua.
Any good document often needs captioned figures -- and for full support, we have them in the resilient.book
class. And as with captioned tables, our packages could have a default fallback for other classes. So the question now is how to implement this.
On one hand, there's Pandoc's implicit_figures
extension (enabled by defaut, IIRC):
"An image with nonempty alt text, occurring by itself in a paragraph, will be rendered as a figure with a caption. The image’s alt text will be used as the caption."
To investigate: This doesn't seem to be a first-class citizen in the Pandoc JSON AST, so it is probably done by the supporting Pandoc writers (?) - If so, I'm not sure we'd have to tweak lunamark here, for the native markdown route.
On the other, an alternative to supporting the Pandoc way would be to use a surrounding div (:::
) with some "recognized" option.
In French (at least), it's common do have speaker changes in dialogues introduced by an em-dash:
— Lorem ipsum dolor sit amet, consectetur adipiscing elit, dit il.
— C’est sûr !
For comparison, English does not use such dashes normally.
“Lorem ipsum dolor sit amet, consectetur adipiscing elit,” he said.
“Sure!”
(English may used em-dashes, but at the end of a quote. Anyway, that's not the point).
The tricky thing here is that the following space ought to be fixed (= not stretched or shrinked by line justification) so that subsequent dialogue lines start identically.
Of course, I need to typeset such dialogues...
I was first tempted to do it in SILE itself (as we do for other fancy Unicode character and French punctuation, etc.) but it is not completely obvious to do there (I'm pulling my hair out trying, and I don't have much hair left, lol) and it requires awaiting a release...
It would be fairly obvious to do here, intercepting —[ ]+
at the start of a paragraph (where the —
is native or derives from ---
in Markdown or Djot, it ends up as — anyway in the SILE AST).
The workaround is to manually enforce it, working in Djot only (attributes on anything, here a nbsp):
---\ {.fixed}Lorem ....
---\ {.fixed}Sure!
Cumbersome and annoying when copying/pasting text....
I dunno. Try harder to implement it in SILE for any input (incl. SIL), or push my already working attempt here?
A generalization on #13 with elements of analysis and discussion...
blocks elements, except <hr>
native_divs
and markdown_in_html_blocks
, both enabled by defaultwriter.display_html
which should work as in Pandoc with the above extensions disabled. (Should = I didn't test).<table>
for instance), most of which we cannot easily render in a satisfying way (even with the help of a 3rd party HTML parsing library, such as htmlparser
hinted at in #18)Block elements, the special case of <hr>
Inline elements, except br
and wbr
native_spans
enabled by default, for <span>
elementsbracketed_spans
, respecting the structure (i.e. the content is below the Pandoc.Span element)
$ pandoc -t json
<span>content _italic_</span>
{"pandoc-api-version":[1,22,2],"meta":{},"blocks":[{"t":"Para", "c":[{"t":"Span","c":[["",[],[]],[{"t":"Str","c":"content"},{"t":"Space"},{"t":"Emph","c":[{"t":"Str","c":"italic"}]}]]}]}]}
<sup>
etc.), the HTML is spit out, but flattened. I.e. the structure is lost, one gets at the same level the opening tag, the content and the closing tag
$ pandoc -t json -f markdown-native_spans
<span>content _italic_</span>
{"pandoc-api-version":[1,22,2],"meta":{},"blocks":[{"t":"Para","c":[{"t":"RawInline","c":["html","<span>"]},{"t":"Str","c":"content"},{"t":"Space"},{"t":"Emph","c":[{"t":"Str","c":"italic"}]},{"t":"RawInline","c":["html","</span>"]}]}]}
writer.inline_html
which is technically equivalent to Pandoc's markdown-native_spans
Inline elements, the special cas of br
and wbr
Preliminary conclusions
<hr/>
could be done more easily - but is it worth the effort and test anyway, as Markdown supports horizontal rules and we can even achieve nice things with them (#27)<sup>
working without much additional logic. The fact that they allow Markdown content also makes the use of an HTML parsing library very clumsy...
<span>
could be supported by implementing native_spans
in Lunamark - but is it worth the effort?<br>
and <wbr>
would be (decently) easy to support (#13)I added a debug locally to do it but it seems highly useful as most likely some people will want to go from Markdown to SILE on a more permanent basis.
diff --git a/inputters/markdown.lua b/inputters/markdown.lua
index 980e97e..8d15216 100644
--- a/inputters/markdown.lua
+++ b/inputters/markdown.lua
@@ -3,6 +3,7 @@
-- Using the lunamark library.
--
local utils = require("packages.markdown.utils")
+local ast = require("sile.core.ast")
local function simpleCommandWrapper (name)
-- Simple wrapper argound a SILE command
@@ -334,6 +335,7 @@ function inputter.parse (_, doc)
escaped_line_breaks = true,
})
local tree = parse(doc)
+ SU.debug("markdown", "Parsed tree: " .. ast.astToSil(tree))
-- The Markdown parsing returns a string or a SILE AST table.
-- Wrap it in some document structure so we can just process it, and if at
-- root level, load a default support class.
We'd need:
tex_math_dollars
extension in Lunamark (= jgm/lunamark#50) to extract the formula and invoke the writer.inputter
, though I've a few doubts regarding the actual implementation, esp. with AST dumping and round-tripping... (#15)The Lua ecosystem is so non-existent that the very choice of Lua for SILE could be questioned. And IMHO, Rust is not even better -- said with a grain of salt, to those in the SILE community who might argue otherwise, heh! 😸 ↩
Hi Didier (@Omikhleia),
I installed the SILE Typesetter and "markdown.sile" on Ubuntu as noted in the Discussions. I cloned the repo and then tried to run SILE on the "introduction.md" file in the "examples" folder:
sile --use inputters.markdown --use packages.autodoc introduction.md
This gave the following error:
The text of the error is:
SILE v0.14.9 (Lua 5.2)
! Unexpected Lua error
error summary:
Processing at: introduction.md: in <snippet>:
[[# IntroductionThis collection of modules for the [SILE](https://github.com/sile-typesetter/sile) t]]
Using code at: /usr/share/sile/packages/math/texlike.lua:188: attempt to call field 'kpairs' (a nil value)
Run with --traceback for more detailed trace leading up to errors.
Running the --traceback
gives the following extended output:
SILE v0.14.9 (Lua 5.2)
! Unexpected Lua error
stack traceback:
/usr/share/sile/packages/math/texlike.lua:188: in function 'fold_pairs'
/usr/share/sile/packages/math/texlike.lua:256: in function 'compileToMathML_aux'
/usr/share/sile/packages/math/texlike.lua:398: in function 'compileToMathML'
/usr/share/sile/packages/math/texlike.lua:420: in main chunk
[C]: in function 'require'
/usr/share/sile/packages/math/init.lua:10: in function 'init'
/usr/share/lua/5.2/pl/class.lua:38: in function 'call_ctor'
/usr/share/lua/5.2/pl/class.lua:171: in function 'pack'
/usr/share/sile/classes/base.lua:145: in function 'loadPackage'
...
/home/new/.luarocks/share/lua/5.2/sile/classes/markdown.lua:7: in function 'init'
/usr/share/lua/5.2/pl/class.lua:38: in function 'call_ctor'
/usr/share/lua/5.2/pl/class.lua:171: in function 'constructor'
/usr/share/sile/inputters/base.lua:31: in function 'classInit'
/usr/share/sile/inputters/base.lua:40: in function 'requireClass'
/usr/share/sile/inputters/base.lua:48: in function 'process'
/usr/share/sile/core/sile.lua:298: in function 'processString'
/usr/share/sile/core/sile.lua:330: in function </usr/share/sile/core/sile.lua:303>
(...tail calls...)
[C]: in function 'xpcall'
/usr/bin/sile:127: in main chunk
[C]: in ?
error summary:
Processing at: introduction.md: in <snippet>:
[[# IntroductionThis collection of modules for the [SILE](https://github.com/sile-typesetter/sile) t]]
Using code at: /usr/share/sile/packages/math/texlike.lua:188: attempt to call field 'kpairs' (a nil value)
new@DESKTOP-SNF8BOK:~/GitHub/Omikhleia/markdown.sile/examples$
As noted in the Discussion, I installed everything locally, which is why there is a reference to /home/new/.luarocks/share/lua/5.2/sile/classes/markdown.lua:7: in function 'init'
in the above output.
Running cat /home/new/.luarocks/share/lua/5.2/sile/classes/markdown.lua
gives:
local book = require("classes.book")
local class = pl.class(book)
class._name = "markdown"
function class:_init (options)
book._init(self, options)
self:loadPackage("markdown")
return self
end
return class
This looks like your classes/markdown.lua file:
I do not know how classes or packages work in Lua, but I would have expected that the local inputters/markdown.lua would have been part of that trace, since it is specified in the --use
on the command line.
The only reference to "inputters" in the traceback are the following lines:
/usr/share/sile/inputters/base.lua:40: in function 'requireClass'
/usr/share/sile/inputters/base.lua:31: in function 'classInit'
/usr/share/sile/inputters/base.lua:48: in function 'process'
Is this expected? Further, when I do a locate "markdown.lua"
, the first three entries are:
/home/new/.luarocks/share/lua/5.2/sile/classes/markdown.lua
/home/new/.luarocks/share/lua/5.2/sile/inputters/markdown.lua
/home/new/.luarocks/share/lua/5.2/sile/lunamark/reader/markdown.lua
These are the expected "local" entries. Is there possibly an issue with the PATH such that the "classes" is being picked up ahead of the "inputters"? Or have I gone down the wrong rabbit trail completely?
In any case, your assistance would be appreciated!
Kind Regards,
Liam
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.