dart-lang/markdown

A Dart markdown library

markdown's Issues

4-space indent of sub-bullets renders incorrectly

Consider this markdown for the shelf readme.

Render issues when put through markdown package: https://api.dartlang.org/apidocs/channels/be/dartdoc-viewer/shelf

Renders fine on github: https://github.com/dart-lang/bleeding_edge/blob/master/dart/pkg/shelf/README.md

Renders fine on pub.dartlang.org - http://pub.dartlang.org/packages/shelf

* [Connect](http://www.senchalabs.org/connect/) for NodeJS.
    * Read [this great write-up](http://howtonode.org/connect-it) to understand
      the overall philosophy of all of these models.
* [Rack](http://rack.github.io/) for Ruby.
* [WSGI](http://legacy.python.org/dev/peps/pep-3333/) for Python.

CC @caffinatedmonkey

Publish 0.7.0 on pub.dartlang.org

Thanks!

Make it easier to see which version dartdoc/pub uses

Someone pointed out that something that works in https://dart-lang.github.io/markdown/ doesn't work in dartdoc. (In this case, it was creating a list without a leading blank line.)

It'd be nice to be able to specify which version of markdown/dartdoc/pub you're interested in.

If that's not possible, we should at least make it clear (1) which version the interactive editor uses and (2) that tools might be using an old version. We could create a doc here and link to it from the live editor with text like "markdown 0.11.0+1 (details)".

Best of all (from my pov) would be for pub and dartdoc to use (or be configurable to use) the latest version of markdown all the time.

analysis errors when compiling the markdown package with ddc

When run on the 0.1.4 version of the dev_compiler, I see these errors:

severe: [AnalyzerMessage] The getter 'children' is not defined for the class 'Node' (package:markdown/src/inline_parser.dart, line 361, col 10)
severe: [AnalyzerMessage] The getter 'attributes' is not defined for the class 'Node' (package:markdown/src/inline_parser.dart, line 354, col 36)
severe: [AnalyzerMessage] The getter 'text' is not defined for the class 'Node' (package:markdown/src/inline_parser.dart, line 136, col 48)
severe: [AnalyzerMessage] The getter 'children' is not defined for the class 'Node' (package:markdown/src/inline_parser.dart, line 355, col 34)
severe: [AnalyzerMessage] The getter 'attributes' is not defined for the class 'Node' (package:markdown/src/inline_parser.dart, line 353, col 34)
severe: [InvalidSuperInvocation] super call must be last in an initializer list (see http://goo.gl/q1T4BB): super(pattern) (package:markdown/src/inline_parser.dart, line 191, col 9)
severe: [InvalidSuperInvocation] super call must be last in an initializer list (see http://goo.gl/q1T4BB): super(pattern) (package:markdown/src/inline_parser.dart, line 230, col 9)

CommonMark compliance

Hi,

Is there a plan for this package to become "CommonMark compliant" or is the intention to follow some specific (or it's own) flavor of Markdown?

I have been using a different Dart package, md_proc, simply because it claims to be CommonMark compliant. Not sure how these two package differs, but if this package also intends to become CommonMark compliant maybe there are some "synergies" to be found between the two:
https://github.com/dikmax/md_proc

Formal specification for extensions beyond "standard" Markdown

This markdown package currently supports one non-standard syntax (the code fence), and there are open issues requesting support for more (see #18, #38, #41, and #31). It's of course up to the maintainers whether to include any more non-standard syntaxes beyond code fences, but the none of these issues has been outright rejected.

When syntax extensions are supported in a Markdown parser, they should be able to be toggled, to reduce surprises (see Pandoc, Redcarpet, and others). This is the primary reason for having a formal specification: this package shouldn't format a user's Markdown strangely because it chose to include one certain non-standard syntax, and to always include it.

I propose a new argument to markdownToHtml() and new Document(), perhaps List<String> extensions. This argument defaults to []. The different values might include:

• "code_fences" (already implemented)
• "inline_html" (#18)
• "tables" (#38)
• "no_intra_word_emphasis" (#41)
• "header_ids" (#31)

I realize that this package already supports user-written, inline syntax extensions per-document, but supporting user-written block syntaxes would be very hard, as they would want access to the different private patterns, etc (for example how ListSyntax uses all the other block syntax patterns to see whats up). So if this opens the door to non-standard block syntaxes like tables, and definition lists, then its a super win.

This new argument also eases some anxiety of adding a new syntax extension: it will always be safely disabled by default, and only enabled when the user chooses to use it.

If y'all like the idea, I'm happy to send a PR adding the argument, and disabling the FencedCodeBlockSyntax by default.

Benchmarking

Need more help to add a block syntax extension

I'm trying to write a BlockSyntax for Description Lists (pandoc description, PHP Markdown Extra description), but I need some more helpers from BlockParser, and ListSyntax:

• BlockParser's _pos is (obviously) private, so the only way to peek at lines is with get current and get next. I need to see the line 2 lines ahead, so I propose a String peek(int linesAhead).
• ListSyntax's parse() has a handy blocksInList object that should be a static final member of ListSyntax.

I intend to do the above.

Raw http links are not converted

Consider this case:

API Docs can be found here: http://www.dartdocs.org/documentation/quiver/latest

The generated markdown does not wrap the http link in an <a> tag.

Code indented 5 spaces is not recognized as code

Real-life example: https://github.com/dart-lang/sdk/blob/master/sdk/lib/convert/json.dart#L48

Looks like this:

FWIW it appears that commonmark supports an indention of 5 spaces: http://spec.commonmark.org/dingus/ (but I realize that we aren't commonmark compliant)

extension/plugin mechanism

It would be great to have an extension mechanism like the python version has:

http://pythonhosted.org/Markdown/extensions/index.html

I've found the admonition, the attribute lists and the code highlighter the most useful.

Links with multiple underscores are not recognized

Consider this code:

import 'package:markdown/markdown.dart' as md;

main() {
  String input = 'Hello [WORLD_TWO_UNDERSCORES] but this works [HELLO_WORLD]';

  md.Node _linkResolver(String name) {
    return new md.Text('XX $name XX');
  }

  print(md.markdownToHtml(input, linkResolver: _linkResolver));
}

Output:

<p>Hello [WORLD_TWO_UNDERSCORES] but this works XX HELLO_WORLD XX</p>

Expected:

<p>Hello XX WORLD_TWO_UNDERSCORES XX but this works XX HELLO_WORLD XX</p>

FWIW, If I remove new TagSyntax(r'_', tag: 'em'), from the top of inline_parser.dart, it works.

HTML Elements are not consistent

print(markdownToHtml('<div>\n\n**abc**\n\n</div>\n')); does escape the first '<' of the closing DIV and wraps it inside a P element:

<div>
<p><strong>abc</strong></p>
<p>&lt;/div></p>

Removing 1-1 newlines, and print(markdownToHtml('<div>\n**abc**\n</div>\n')); does not convert the content to strong:

<div>
**abc**
</div>

In both cases, the expected output would be something like this:

<div>
<p><strong>abc</strong></p>
</div>

Benchmark broken

When I run dart benchmark/benchmark.dart, it complains that the output isn't expected. When I diff it with benchmark/output.html, I get:

367c367,368
< </code></pre>
\ No newline at end of file

---
> </code></pre>
>

I think someone's editor killed the newline at the end or something (if I open benchmark/output.html in vim, and save and close, a newline is added, and git shows the file has changed).

To prevent annoying bugs like this, I suggest we trimRight() the actual and expected outputs.

InlineParser fails in checked mode

Run tests with dart --checked

One failure

ERROR: Resolver simple resolver
  Test failed: Caught type 'Text' is not a subtype of type 'Element' of 'function result'.
  package:markdown/src/inline_parser.dart 295:26  LinkSyntax.createNode
  package:markdown/src/inline_parser.dart 337:30  LinkSyntax.onMatchEnd
  package:markdown/src/inline_parser.dart 446:26  TagState.close
  package:markdown/src/inline_parser.dart 408:12  TagState.tryMatch
  package:markdown/src/inline_parser.dart 104:31  InlineParser.parse
  package:markdown/src/document.dart 79:76        Document.parseInline
  package:markdown/src/block_parser.dart 531:49   ParagraphSyntax.parse
  package:markdown/src/document.dart 65:37        Document.parseLines
  package:markdown/src/html_renderer.dart 22:37   markdownToHtml
  test/markdown_test.dart 1057:32                 validate.<fn>
  package:unittest/src/test_case.dart 102:27      _run.<fn>
  dart:async/zone.dart 730                        _rootRunUnary
  dart:async/zone.dart 462                        _ZoneDelegate.runUnary
  dart:async/zone.dart 667                        _CustomizedZone.runUnary
  dart:async/future_impl.dart 488                 _Future._propagateToListeners.handleValueCallback
  dart:async/future_impl.dart 571                 _Future._propagateToListeners
  dart:async/future_impl.dart 331                 _Future._completeWithValue
  dart:async/future_impl.dart 393                 _Future._asyncComplete.<fn>
  dart:async/zone.dart 723                        _rootRun
  dart:async/zone.dart 453                        _ZoneDelegate.run
  dart:async/zone.dart 663                        _CustomizedZone.run
  dart:async/zone.dart 574                        _BaseZone.runGuarded
  dart:async/zone.dart 599                        _BaseZone.bindCallback.<fn>
  dart:async/schedule_microtask.dart 23           _asyncRunCallbackLoop
  dart:async/schedule_microtask.dart 32           _asyncRunCallback
  dart:isolate-patch/isolate_patch.dart 119       _RawReceivePortImpl._handleMessage

See https://code.google.com/p/dart/issues/detail?id=17896

CC @caffinatedmonkey

square brackets inside of inline code should be left alone

I want to write a comment that contains an example with a list literal like

const ["foo", "bar"]

which should render as shown by the github markdown parser here. Instead, the square brackets disappear.

Two or more spaces at the end of the line should create line breaks

See http://daringfireball.net/projects/markdown/syntax#p

When you do want to insert a <br /> break tag using Markdown, you end a line with two or more spaces, then type return.

FencedCodeBlocks don't retain language

From Pandoc and GitHub-flavored Markdown, etc, we should retain language when it is specified after backticks, like:

```dart
code
```

should render to <pre class="dart"><code>code</code></pre>.

Simple way of showing markdown output

The package:markdown has already a bin/markdown.dart script which renders html, but it would be even nicer if it came with way to serve a valid html file, something like:

$ pub global activate markdown
$ pub global run markdown:serve myfile.md
See the rendered markdown at:  http://localhost:8080/myfile.html

Provide a way for using package:markdown with custom syntax highlighter for inline/block code quotes

It would be very nice if one could use package:markdown and provide custom syntax highlighter plugins for inline and block code quotes. The pub-dartlang-dart server app contains a simplified syntax highlighter for Dart -- though only with patches to the markdown package.

One way of doing this would be to:
a) let the renderer take care of html rendering and remove the escaping from the parser
b) allow passing custom formatters to the HtmlRenderer

For a) one can see that html escaping happens already when parsing:

class CodeBlockSyntax {
  ...
  Node parse(BlockParser parser) {
    ...
    var escaped = escapeHtml(childLines.join('\n'));
    return new Element('pre', [new Element.text('code', escaped)]);
  }
}

Add support for tables

Currently support for tables is missing.

Running the following code:

import 'package:markdown/markdown.dart';

var table =
r'''
First Header  | Second Header
------------- | -------------
Content Cell  | Content Cell
Content Cell  | Content Cell
''';

main() {
  print(markdownToHtml(table));
}

prints a paragraph tag with the text.

<p>First Header  | Second Header
------------- | -------------
Content Cell  | Content Cell
Content Cell  | Content Cell</p>

It should print something like this:

<table>
<thead>
<tr><th>First Header</th><th>Second Header</th></tr>
</thead>
<tbody>
<tr><td>Content Cell</td><td>Content Cell</td></tr>
<tr><td>Content Cell</td><td>Content Cell</td></tr>
</tbody>
</table>

Links inside `[...]` not rendered correctly.

Markdown: [ [foobar](http://bazbat.com/) ]

Expected: <p><a href='http://bazbat.com/'>foobar</a></p>

Actual: <p>[ [foobar](http://bazbat.com/) ]</p>

Launch a service that returns compiled HTML from Markdown

For use at babelmark2. Can use App Engine's Managed VMs.

Add instructions on how to bump CommonMark version

I'm not sure if there are instructions on how to bump the CommonMark specs, but we should add some to tool/README.md.

Images not rendered correctly

#19 was closed, but why? Image rendering is still totally broken. An inline image without a hyperlink is rendered with a hyperlink, and an inline image with a hyperlink is completely corrupted, producing ill-formed HTML markup, illegal URLs, and orphaned characters.

Example 1 - image without hyperlink:

![a robot](http://www.littlerobotsoundfactory.com/img/8-bit_1.png)

This renders as:

<p>
  <a href="http://www.littlerobotsoundfactory.com/img/8-bit_1.png">
    <img alt="a robot" src="http://www.littlerobotsoundfactory.com/img/8-bit_1.png">
    </img>
  </a>
</p>

(I added newlines and indentation to clarify structure; the actual rendering has no whitespace between tags.) In addition to the superfluous <a> element, note the illegal </img> tag.

The correct rendering is:

<p>
    <img alt="a robot" src="http://www.littlerobotsoundfactory.com/img/8-bit_1.png">
</p>

Example 2 - image with hyperlink:

This image should be hyperlinked to robotstxt.org:

[![a robot](http://www.littlerobotsoundfactory.com/img/8-bit_1.png)](http://www.robotstxt.org/)

But it unfortunately renders like this:

<p>
  [
  <a href="http://www.littlerobotsoundfactory.com/img/8-bit_1.png)](http://www.robotstxt.org/">
    <img alt="a robot" src="http://www.littlerobotsoundfactory.com/img/8-bit_1.png)](http://www.robotstxt.org/">
    </img>
  </a>
</p>

(Again, whitespace added to clarify structure.) This rendering is truly bizarre and broken. First, there is an orphaned [ as the first child of <p>. Then, the href of the anchor is totally wrong: http://www.littlerobotsoundfactory.com/img/8-bit_1.png)](http://www.robotstxt.org/. It combines the two URLs in an unexpected way and includes characters that are not legal in a URL. The src of the image is set to the same string as href, thus also totally wrong. Finally, there's the illegal </img> closing tag again.

Standard

The CommonMark reference implementation does not hyperlink an inline image, nor does Gruber's Markdown. Why then does Dart Markdown choose to do this? Markdown syntax already supports putting an image inside of a link, so it doesn't make sense to have 2 syntaxes for hyperlinking an image and 0 syntaxes for an image without hyperlink.

Please see comparison of the two syntaxes using CommonMark.

Whitespace should be allowed between a link's destination and title.

e.g.

Links [are](http://foo.com
"woo") awesome.

Release 0.6.0

We've love to load it up with our dartdoc logic in the SDK.

CC @caffinatedmonkey

Static warnings in new image code

CC @caffinatedmonkey re https://code.google.com/p/dart/issues/detail?id=17896

publish 0.5.1

Currently we have code in the Dart project depending on a fork by @efortuna

Please update the pubspec file (to remove -dev) and publish.

Then we can depend on the official release, which will be nice.

Get on Travis

use markdown in client side error

when I use markdown in index.html and index.dart,it shows error:
Internal error: 'package:markdown/src/document.dart': error: line 23 pos 39: unexpected token '?' this.extensionSet = extensionSet ?? ExtensionSet.commonMark;

library my_project.web.index;

import 'package:mypaper/main_app.dart';
import 'package:polymer/polymer.dart';
import 'dart:html';
import 'package:markdown/markdown.dart';
import 'package:codemirror/codemirror.dart';
/// [MainApp] used!
main() async {
  await initPolymer();
  Map options = {
    'mode': 'html',
    'theme': 'monokai'
  };

  CodeMirror editor = new CodeMirror.fromElement(
      querySelector('#textContainer'), options: options);
  editor.getDoc().setValue(markdownToHtml("## bhtml"));
/*  String markdown2Html=markdownToHtml(markdown);*/

  CodeMirror editor1 = new CodeMirror.fromElement(
      querySelector('#textContainer2'), options: options);
  editor1.getDoc().setValue("# html");
}
updateViewer(){

}```

Markdown converts [] space () to links

Take a look at

https://github.com/dart-lang/sdk/blob/master/sdk/lib/developer/developer.dart#L40

Markdown converts it to links

https://api.dartlang.org/1.12.1/dart-developer/log.html

Nested bullets aren't rendering correctly

From this file: https://github.com/dart-lang/shelf/blob/master/README.md

At the bottom, on Github, we see this:

When dartdoc, which uses this markdown package, renders that file, it looks like this:

(note: I didnt verify that the bug is definitely with markdown package. dartdoc could be doing something weird here)

Write Documentation

Let's add some documentation to this project!

Cannot escape a link (or other things)

$ dart bin/markdown.dart
Hi [there]\(foo).
<CTRL+D>
<p>Hi [there]\(foo).</p>

Compare to other stuff on babelmark: http://johnmacfarlane.net/babelmark2/?text=Foo+%5Bdart%5D%5C(bar)

Here's the CommonMark entry: http://spec.commonmark.org/0.22/#backslash-escapes

List items with code blocks not starting at beginning of line cause issues.

The following program (with the first 4 spaces removed) :

    import 'package:markdown/markdown.dart';

    String TEXT = r'''
    * before

      ```
    hello() => print('hello');
    ```

    between

       ```
    hello() => print('hello');
    ```

    after
    ''';

    main() {
      var blocks = new Document().parseLines(TEXT.split('\n'));
      print(new HtmlRenderer().render(blocks));
    }

Produces this output (with package:[email protected]):

<ul><li>
<p>before</p>
<pre><code class="language-hello()"></code></pre></li></ul>
<pre><code>
between

</code></pre>
<p>hello() => print('hello');</p>
<pre><code>
after

</code></pre>

So there seems to be an issue with list items and code blocks starting at nonzero column.

The example was taken from the dart-lang/angular2/master/CHANGELOG.md file.

Support comments in markdown files

Github markdown seems to accept comments of the form

<!--  This gets ignored -->

It would be great if package:markdown could have support for that.

No way to hard newline

Similar to #30 , we should render escaped newlines as <br />:

hello \
there

Large Files?

Is there some kind of size limitation that we should be aware of?
Small files (less than 50kb) get converted with no problem, but larger (more than 500kb) ones fail silently (somewhere inside the loop that converts markdown lines to AST nodes).
Any suggestions on what might be the problem / how to debug this?

Inline image without linking?

The following entry links to the image:

![Alt text](/path/to/img.jpg "Optional title")

producing the following output:

<a href="/path/to/img.jpg" title="Optional title"><img alt="Alt text" src="/path/to/img.jpg" title="Optional title"></img></a>

Usually the markdown parsers just produce the img tag, without the link. It would be helpful if we had an option to control this.

Line breaks not parsed correctly

Line breaks (e.g. the two spaces at the end of a line) are not parsed correctly. Line breaks should emit a <br> tag, but InlineParser emits it as a Text node, not an element node. This results in a correct rendering when using the supplied HtmlRenderer, but makes other NodeVistor implementations useless.

For example, consider this node visitor, which should print an element tree:

class AstPrinter implements md.NodeVisitor {
    int indent = '';

    void visitElementAfter(md.Element element) {
        print('${indent}leaving: ${element.tag}');
        indent = indent.substring(0, indent.length-2);
    }

    bool visitElementBefore(md.Element element) {
        indent += '  ';

        if (element.isEmpty) {
            print('${indent}void: ${element.tag}');
            indent = indent.substring(0, indent.length-2);
            return false;
        } else {
            print('${indent}entering: ${element.tag}');
            return true;
        }
    }

    void visitText(md.Text node) {
        indent += '  ';
        List<String> lines = node.text.replaceAll('\r\n', '\n').split('\n');
        for (String line in lines) {
            print('${indent}TEXT> ${line}');
        }
        indent = indent.substring(0, indent.length-2);
    }
}

The driver code looks like this:

List<String> lines = ['*hello* **world**!', '', '> this is a quote', 'good bye!'];
md.Document doc = new md.Document();
AstPrinter printer = new AstPrinter();

for (md.Node node in doc.parseLines(lines)) {
    node.accept(printer);
}

(I've imported markdown as md so it doesn't conflict with core:html.)

The output of this example is:

entering: p
  entering: em
    TEXT> hello
  leaving: em
  TEXT>  
  entering: strong
    TEXT> world
  leaving: strong
  TEXT> !
leaving: p
entering: blockquote
  entering: p
    TEXT> this is a quote
  leaving: p
leaving: blockquote
entering: p
  TEXT> good bye!
leaving: p

(Side note, since there are no unit tests for proper AST construction, this AstPrinter would be a handy class to include in the Markdown library for debugging purposes.)

What tree is printed when a linebreak is parsed? Here's a test:

List<String> lines = ['this is a test  ', 'this is only a test'];

The result:

entering: p
  TEXT> this is a test
  TEXT> <br />
  TEXT> this is only a test
leaving: p

The <br /> is emitted as a text node, not an element! So a NodeVisitor can't see <br /> elements, which makes AST analysis impractical and also makes rendering to other formats (e.g. PDF) buggy: you'll get a literal <br /> in your PDF instead of the line break that you expected.

Note: there are multiple unit tests covering line breaks, but these tests only compare output in HTML form, not in AST form. AFAICT, there are zero tests on the AST.

Update `gh-pages`

Rewrite the page using polymer.dart.

Underscores in the middle of a word should not be parsed as emphasis like_in_this

From inline_parser.dart:

    // TODO(rnystrom): Underscores in the middle of a word should not be
    // parsed as emphasis like_in_this.
    new TagSyntax(r'_', tag: 'em'),

Thanks for taking a look!

Make a presubmit or travis script

That runs the tests and analyzer strong mode.

Bump CommonMark specs to 0.25

I'm not sure if there are instructions on how to bump the CommonMark specs, but we should add some to tool/README.md.

Consistent Output

Currently, the parser places new lines where it shouldn't. Here is an example: https://drone.io/github.com/caffinatedmonkey/dart-markdown/3

Either the output should have no newlines or it should preserve newlines.

dart:io dependency on importing for a client app

When I try to run this package on a fresh Dart script aimed at the browser I immediately get an error mentioning the fail import of "dart:io". This is even before I use the package so it's not related to syntax highlight I don't think.

This bug makes it impossible to use this package on a client app.

Allow Inline HTML

Not sure if this is in scope, but markdown normally allows HTML tags. You can write something like this:

This is **markdown'd**. But you can also <span color="red">do stuff manually</span>.

The span should be copied verbatim to the HTML output. In this implementation, though, all HTML tags are escaped (< becomes &lt).

Launch a website showing rendered markdown

This would be available at dart-lang.github.io/markdown.

Links are rendered incorrectly if they are in a list

This works:

And anything beyond that gets interpreted as [Markdown][markdown]!
[markdown]: http://daringfireball.net/projects/markdown/syntax

but this fails:

    [Markdown][markdown]!
[markdown]: http://daringfireball.net/projects/markdown/syntax

and will be rendered as

[Markdown][markdown]!