mmarkdown/mmark

Mmark: a powerful markdown processor in Go geared towards the IETF

mmark's Issues

man output: proper callout support

Currently callouts in manual output are supported, but it lacks the option to specify comments to be detected, this does happen in the other renderers and should be added here as well.

mmark math support

I'm trying to use hugo + mmark + katex. Documentation around the web about math support in mmark is sparse and seems to be partly / mostly outdated.

Using mmark as a markdown processor in hugo, $$ E = mc^2 $$ is rendered to <span class='math'>\( E = mc^2\)</span> which is valid TeX, but doesn't seem to be compliant with MathJax as is written in most docs, but rather seems to be MathJS syntax (?).

Trying to run

$("span[class='math']").replaceWith(
  function(){
    var tex = $(this).text();
    return "<span class=\"inline-equation\">" +
           katex.renderToString(tex) +
           "</span>";
});

over it fails because KaTeX cannot deal with the delimiters \( and \).

Is there more documentation on math support in mmark that I have missed and is it possible to configure how the math is rendered?

Old domain still in use in the GitHub project title

Deprecate xml2 parser and renderer

xml2 output for the IETF has already(?) been deprecated and the RFC editor has moved to some form of 7991 XML as the canonical form.

I'll try to get some authoritative source to say so and then delete this xml2 code in Mmark.

Add 'safeInclude' flag/options

Add a 'safeInclude' flag that only allows relative includes and no .. in the paths. This is a simple check.

A more advanced safe include would be to only allow files below the initial file being work on.

xml/xml2: table support and caption isn't there yet

Need to double check if tables are rendered correctly, esp. when they have a caption.

Also tests.

xml: images as artwork doesn't work.

for rfc7991 output we can't output images as artwork, as a image element is not an (block)level artwork, this means we must inhibit outputting <t>s, but that is hard. Also using the title for the name attribute looks weird.

A figureblock works, but must be present in the markdown code.

support convertedFrom for rfc building

To help us debug your issue please add these details.

Can't build an RFC without warnings:

514.xml(5): Warning: Expected explicit values for year, month, and day, but found None, None and None
3514.xml(3): Warning: Expected a <link> with rel='convertedFrom' providing the datatracker url for the origin draft.

We need a convertedFrom in the toml, maybe 7749 can use it as well.

See examples in the rfc dir

Attribute "start" not supported for list style="numbers"

What went wrong?

mmark logs Attribute "start" not supported for list style="numbers" and makes the 2. element of a list start over at 1..

What version of mmark are you using?

v2.0.46

What command line did you use?

mmark -2

What platform (Linux/Apple/Windows)?

• Linux
• Apple
• Windows

What is the most minimal markdown snippets that shows the problem?

1. Process the internal representation into a field element s as follows:

_some unindented code block here_

2. Return the canonical little-endian encoding of s.

includes

Now that gomarkdown/markdown support an IncludeFunc we can implement all the bits we need in mmark.

subfigures

Subfigures in mmark are sorta supported by prefixing a paragraph with F> . This is a bit annoying. The scholary markdown (discontinued) solution seems better:

Do this? http://scholarlymarkdown.com/Scholarly-Markdown-Guide.html#floating-env

Add indexInclude from 7991

Add indexInclude attribute to the TOML block: https://tools.ietf.org/html/rfc7991#section-2.45.4

Update on gomarkdown Parser (parser.ParserOptions => parser.Options)

To help us debug your issue please add these details.

try to build mmark from source and got following error
# github.com/mmarkdown/mmark github.com\mmarkdown\mmark\mmark.go:90:12: undefined: parser.ParserOptions

What went wrong?

failed to build from source after cloning the repo.
check gomarkdown commit

What version of mmark are you using?

2.0.37

What command line did you use?

go get && go build

What platform (Linux/Apple/Windows)?

• Linux
• Apple
• Windows

What is the most minimal markdown snippets that shows the problem?

table markdown output with span elements

A markdown table:

 \          | \             | \         
------------|---------------|-----------
`break`	    |`default`	    |`func`	    

is rendered:

          |            |
----------|------------|---------
`break`	  | `default`	 | `func`	

It's off because it does not account for the backticks (or any other span markings)

2.0.32 release in https://github.com/mmarkdown/mmark/releases ?

Thanks, using your new release now. Installed directly. Can you release a new version also?

[`link`](http://link.link) fails with Error: Did not expect element t there, at /rfc/middle/t a.xml(3): Error: Invalid document before running preptool

To help us debug your issue please add these details.

What went wrong?

Unable to include a link of a code sample. I commonly do this—and see this—on StackOverflow.

What version of mmark are you using?

2.0.40, and xml2rfc 2.22.0

What command line did you use?

$ mmark a.md > a.xml
$ xml2rfc --v3 a.xml
Parsing file a.xml
a.xml(18): Error: Did not expect element t there, at /rfc/middle/t
a.xml(3): Error: Invalid document before running preptool.
Cannot continue, quitting now

What platform (Linux/Apple/Windows)?

• Linux
• Apple
• Windows

What is the most minimal markdown snippets that shows the problem?

$ cat a.md
%%%
title = "Model designer"
abbrev = "ODESIGN"
area = "Internet"
workgroup = "Offscale.io"
keyword = ["platform-independent"]
#date = 2019-12-07T00:00:00Z

[seriesInfo]
name = "RFC"
value = "ODESIGN0.0.1"
status = "informational"

[[author]]
surname="Marks"
fullname="Samuel Marks"
organization = "Sydney Scientific"
  [author.address]
  email = "[email protected]"
%%%

{mainmatter}

You MAY use [`localStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) and/or similar built-ins.

{backmatter}

AST transformer

Now that there is a -markdown renderer which output markdown again, we can work on the AST and transform it and then output "new" markdown.

I.e. you can write a super tiny go program that rewrites the AST: for instance to transform all emphasis to strong. (inline are easiest).

But you can also have a more sophisticated one that looks at a code block and if the language matches exec-dot executes dot and converts the whole code block into an dot-generated image.

xml2: anchor needs to be on figure

<figure title="Scenic Routing Option Layout ">
 <artwork anchor="fig-scenic-routing-option-layout"> 

the anchor attribute need to be on the <figure> tag

Target missing from outputted reference

What went wrong?

References including target attributes on reference blocks do not have a target attribute in the outputted xml. It seems this issue was also reported in a comment on issue #53 but never fixed.

What version of mmark are you using?

2.0.38

What command line did you use?

mmark ./test.md > /tmp/x.xml

What platform (Linux/Apple/Windows)?

• Linux

What is the most minimal markdown snippets that shows the problem?

%%%

 Title = "Test doc"

 date = 2019-02-19T00:00:00Z

 [seriesInfo] 
 name = "Internet-Draft"
 value = "draft-test-00"
 status = "informational"

 [[author]]
 initials="J."

%%%

{mainmatter}

# Introduction

Test [@!IANA]

{backmatter}

<reference anchor='IANA' target='https://www.iana.org/assignments/media-types/media-types.xhtml'>
    <front>
        <title abbrev='IANA'>IANA Media Types</title>
        <author>
            <organization>IANA</organization>
        </author>
        <date month='February' year='2019'/>
    </front>
</reference>

Reference blocks with multiple authors only show one

On the latest version of mmark, if a <reference> has multiple authors, only the last one is rendered in the XML. This includes mmark/rfc/3514.md.

Add v2 output

It might make sense to add v2 XML output as well to mmark2, as the v3 tool chain for the IETF is still in development. As a new render is relatively easy to add this should be doable. And a lot of stuff could be copied from mmark1 (https://github.com/miekg/mmark)

Add a shortcut for non-breaking spaces

Feature request

Description

The use of non-breaking spaces is useful for better typography and indispensable in some languages, furthermore when writing long texts (documentation, books, or just blog posts).

However, inserting   in all files is not ideal as it is tiresome and can bring some brittleness (it only works as long as the target rendering is HTML).

Could we think of a universal shortcut that would be rendered appropriately?

Some use cases

• This can be useful to make sure that references stay together: See Fig. 3.
• This should be used when writing something with a unit: Processing time: 25 ms.
• In some languages countries, typographic rules make heavy use of the non-breaking space. This is for example the case in France, where :, ;, ? and ! should be preceded with a non-breaking space: Et il déclara : « Que c'est pénible d'écrire correctement ! Faut-il systématiquement saisir six caractères pour une espace ? »

Existing solutions in other software

• AsciiDoctor predefines a {nbsp} variable. It is as verbose as writing   but rules out the brittleness as it can be treated differently depending on the renderer.
• Some other Markdown parsers use an escaped space (\ ) or a tilde (~) as a non-breaking space. For instance: 42\ ms, 1337~ms.

markdown: indent all lists with a space

Ident all lists when generating markdown, so

* item1
* item2

will become

 *  item1

 *  item2

this works for kind of listst

Compilation breaks on: # Foo\n\n ### Bar \n\n

Replication steps (mmark 2.0.40, xml2rfc 2.22.0):

$ mmark index.md > index.xml
$ xml2rfc index.xml -o index.html
Error: Unable to parse the XML document: index.xml
 <string>: Line 24: Opening and ending tag mismatch: middle line 17 and
section
 <string>: Line 24: Opening and ending tag mismatch: middle line 17 and
section
 <string>: Line 28: Opening and ending tag mismatch: rfc line 3 and section
 <string>: Line 30: Extra content at the end of the document

More info:
https://mailarchive.ietf.org/arch/msg/xml2rfc/OPep-8cpM-NZFqQZzX_whWpU9iw

sort bibliography

Bibliography is stored in a map and when we range over it we get the elements in random order; this should be sorted so a single source markdown file always maps to the same output.

Expand Syntax.md to describe all the syntax

Cut and paste from blackfriday and/or gomarkdown and flesh out a full document

Implement example lists

(re-) implement example lists.

Follow the Pandoc implementation, but only support the alfanum implementation (@good) is allowed, but not (@) (as you can't reference those).

Referencing happens with the same syntax (@good)

markdown output: captionFigure as subfigure is not supported

This is not supported as of yet:

!---
![Array versus slice](fig/array-vs-slice.png)
!---
Figure: An array versus a slice.

complete citations

Citations can have a suffix. This is not implemented. In pandoc this is signaled using a comma as the separation character we now use a space. We should use a comma as well and implement the suffix use in citation rendering.

release needed: gomarkdown broke API for parser.

To help us debug your issue please add these details.

The issue #56 was fixed 10 days ago, but this software FTBFS. A stable upstream library or dependency management would be nice, but for now a release would fix this also.

What went wrong?

Current release FTBFS

What version of mmark are you using?

Current Release

What command line did you use?

go build

What platform (Linux/Apple/Windows)?

• Linux
• Apple
• Windows

What is the most minimal markdown snippets that shows the problem?

FTBFS

implement table captions

See gomarkdown/markdown#35

Stuff can be copied from codeblock or quotes, just needs to be done.

links in markdown output

This is not rendered correctly in the markdown output.

[I'm an inline-style link](https://www.google.com)

[I'm an inline-style link with title](https://www.google.com "Google's Homepage")

[I'm a reference-style link][Arbitrary case-insensitive reference text]

[I'm a relative reference to a repository file](../blob/master/LICENSE)

[You can use numbers for reference-style link definitions][1]

Or leave it empty and use the [link text itself].

URLs and URLs in angle brackets will automatically get turned into links. 
http://www.example.com or <http://www.example.com> and sometimes 
example.com (but not on Github, for example).

Some text to show that the reference links can follow later.

[arbitrary case-insensitive reference text]: https://www.mozilla.org
[1]: http://slashdot.org
[link text itself]: http://www.reddit.com

asides not rendered correctly

they have no content

cite is rendered weirdly

When rendering <cite>s they look a bit off, need to double check if they are ok or what is wrong here.

allow toml for references

I might be nice to allow toml syntax for reference, so that we unify the preamble (which is in toml) with the references. Downside is that all references come in XML format, so you need to reformat that. We already support the XML from the references natively; so not sure how much functionally this adds.

Implement Parts

V2 has dropped Parts, it may make sense to bring that back (not that difficult either).

See https://github.com/miekg/mmark/wiki/Syntax#parts for the syntax.

support footnotes in xml/xml2 output

In the manual output footnotes are outputted in a "Notes" section, we can do the same in the xml/xml2 renderers, same code (almost) can even be used.

This would be inserted just before the {backmatter} similar to the place of the bibliography, but on the opposite site.

citing an ID doesn't work

For I-Ds you may want to add a draft sequence number, which can be done as such: [@?I-D.blah#06]

There was some special code in mmark1 that hasn't been ported yet.

manual page: crash if title is not correctly set

If the little Toml header is not correctly set the manual page output will crash on index out of range.

xml: caption need to be rendered after `<figure>`

<name> needs to be directly after <figure>, right now it comes after the artwork. Simple change and we can look how xml2/renderer.go does it.

markdown: def list in quote wrongly rendered

> this is a quote quote with *empp* and more
>
hallo more tetg hallo more tetg hallo more tetg hallo more tetg hallo more tetg hallo more
>     tetg hallo more tetg hallo more tetg
>
>  :  ahllo
>
more1
>
>  :  more
>

is rendered wrong, the > is tripped from the prefix, because we cut the entire prefix which is too much.

implement index in gomarkdown

This should be relatively easy. Support mmark indices: https://mmark.nl/post/syntax/#indices

This needs upstream support in parser/inline.go (add a new file index.go or something) and a bunch of tests to see if we can parse it.

upstream is https://github.com/gomarkdown/markdown

xml: figure out how to do images

We can include svg, but not(?) other figures... need to double check what rfc7991 says about this.

`<aside>` in rfc 7749 output

Figure out what to do with asides in 7749 output.

Spurious [] before cross-references

What went wrong?

With a long-form xref like the XML output is

[]<xref target="RFC2119"></xref>

and the HTML is

[]<a href="#RFC2119"></a>

I expected no [] in the output.

What version of mmark are you using?

2.0.10

What command line did you use?

echo '[](#RFC2119)' >xref.md
~/go/bin/mmark xref.md

What platform (Linux/Apple/Windows)?

• [*] Linux

xml: Allow html/xml fallthrough

Sometimes you need to fallback to html or xml; currently all blocklevel elements except <reference> are removed from the final output. This may be sub optimal.

Missing organization in reference

To help us debug your issue please add these details.

What went wrong?

Reference is missing organization in the author block. I am not familiar with writing programs in Go, However, looking at the previous fix, it seems that the test is written to pass if the organization is missing.

What version of mmark are you using?

2.0.39

What command line did you use?

mmark test.md > test.xml

What platform (Linux/Apple/Windows)?

• Linux
• Apple
• Windows

What is the most minimal markdown snippets that shows the problem?

Using the same snipped as #61.

%%%

 Title = "Test doc"

 date = 2019-02-19T00:00:00Z

 [seriesInfo] 
 name = "Internet-Draft"
 value = "draft-test-00"
 status = "informational"

 [[author]]
 initials="J."

 %%%

{mainmatter}

# Introduction

Test [@!IANA]

{backmatter}

<reference anchor='IANA' target='https://www.iana.org/assignments/media-types/media-types.xhtml'>
    <front>
        <title abbrev='IANA'>IANA Media Types</title>
        <author>
            <organization>IANA</organization>
        </author>
        <date month='February' year='2019'/>
    </front>
</reference>

From the XML, here's just the reference section. Note the organization in the markdown above does not appear in the XML.

<back>
<references><name>Normative References</name>
<reference anchor="IANA" target="https://www.iana.org/assignments/media-types/media-types.xhtml">
  <front>
    <title>IANA Media Types</title>
    <author>
      <organization></organization>
    </author>
    <date year="2019" month="February"></date>
  </front>
</reference>
</references>

wrong?

To help us debug your issue please add these details.

I use go get to install

What went wrong?

-html

What version of mmark are you using?

master

What command line did you use?

bash

What platform (Linux/Apple/Windows)?

linux

• Linux
• Apple
• Windows

What is the most minimal markdown snippets that shows the problem?

mmark -html ~/go/src/github.com/mmarkdown/mmark/rfc/3514.md > x.html

result:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x53d7b3]

goroutine 1 [running]:
github.com/mmarkdown/mmark/render/mhtml.bibliographyItem(0x5cfde0, 0xc000114930, 0xc000114690, 0xc000114901)
/home/king/go/src/github.com/mmarkdown/mmark/render/mhtml/hook.go:102 +0x183
github.com/mmarkdown/mmark/render/mhtml.RenderHook(0x5cfde0, 0xc000114930, 0x5d1d00, 0xc000114690, 0x1, 0x5cfde0, 0xc0000d3718)
/home/king/go/src/github.com/mmarkdown/mmark/render/mhtml/hook.go:37 +0x38c
github.com/gomarkdown/markdown/html.(*Renderer).RenderNode(0xc0000e8b40, 0x5cfde0, 0xc000114930, 0x5d1d00, 0xc000114690, 0x1, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/html/renderer.go:914 +0x16ce
github.com/gomarkdown/markdown.Render.func1(0x5d1d00, 0xc000114690, 0x4ff401, 0xc0000e8b40)
/home/king/go/src/github.com/gomarkdown/markdown/markdown.go:63 +0x61
github.com/gomarkdown/markdown/ast.NodeVisitorFunc.Visit(0xc00000bd20, 0x5d1d00, 0xc000114690, 0x1, 0xc0000d38e8)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:547 +0x43
github.com/gomarkdown/markdown/ast.Walk(0x5d1d00, 0xc000114690, 0x5d0060, 0xc00000bd20, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:519 +0x70
github.com/gomarkdown/markdown/ast.Walk(0x5d1ca0, 0xc0001148c0, 0x5d0060, 0xc00000bd20, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:530 +0x1ba
github.com/gomarkdown/markdown/ast.Walk(0x5d1280, 0xc000114000, 0x5d0060, 0xc00000bd20, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:530 +0x1ba
github.com/gomarkdown/markdown/ast.Walk(0x5d1220, 0xc000064480, 0x5d0060, 0xc00000bd20, 0xc00000bd20)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:530 +0x1ba
github.com/gomarkdown/markdown/ast.WalkFunc(0x5d1220, 0xc000064480, 0xc00000bd20)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:553 +0x4b
github.com/gomarkdown/markdown.Render(0x5d1220, 0xc000064480, 0x5d0580, 0xc0000e8b40, 0x5a9adc, 0x13, 0x5a861f)
/home/king/go/src/github.com/gomarkdown/markdown/markdown.go:62 +0xd8
main.main()
/home/king/go/src/github.com/mmarkdown/mmark/mmark.go:171 +0x67

Missing docName

What went wrong?

xml2rfc --v3 complained with Warning: Expected a 'docName' attribute in the <rfc/> element, but found none., and https://datatracker.ietf.org/submit/ rejected the XML.

Manually adding the docName attribute fixed the issue.

What version of mmark are you using?

2.0.46

What command line did you use?

mmark

What platform (Linux/Apple/Windows)?

• Linux
• Apple
• Windows

What is the most minimal markdown snippets that shows the problem?

%%%

    Title = "The ristretto255 Group"
    abbrev = "ristretto255"
    ipr = "trust200902"
    category = "info"
    area = "Internet"
    workgroup = "Crypto Forum Research Group"

    [seriesInfo]
    status = "informational"
    name = "Internet-Draft"
    value = "draft-hdevalence-cfrg-ristretto-01"
    stream = "IETF"

[...]

Support duplicate footnotes

To help us debug your issue please add these details.

Is it possible to support multiple references to a footnote content? Seems that blackfriday has implemented the feature through russross/blackfriday#366.

Sentence one[^1]. Sentence two[^1]. Sentence three[^1].

[^1]: They should all refer to here.

As of Hugo 0.49, whose blackfriday engine outputs one line of footnote while mmark does 3 lines of them with incremental numbers.