laws-africa/bluebell

Bluebell is a generic Akoma Ntoso 3 parser.

bluebell's Issues

Support ordered list (<ol>)

perhaps, to mirror the BULLETS markup, something like

NUMBERS
  1. sdfsdf
  2. dfsfs
  3. sdfsdf

where the space denotes the end of the num portion (although I realise lis don't have nums)

Escaping when unparsing may be incorrect

This is the code that handles escaping both block elements and inlines when unparsing:

  <!-- first text nodes of these elems must be escaped if they have special chars -->
  <xsl:template match="a:*[self::a:p or self::a:listIntroduction or self::a:listWrapUp]/text()[not(preceding-sibling::*)]">
    <xsl:call-template name="escape">
      <xsl:with-param name="text" select="." />
    </xsl:call-template>
  </xsl:template>

However, I don't think this is correct. Surely we should be escaping inlines for everything, not just p, listIntroduction and listWrapUp? What about headings and crossheadings?

Additionally, I don't think it's possible to have a text() tag that does have a preceding sibling! I think this is a hangover from slaw where we only escaped the first paragraphs of some things.

Finalise footnote syntax

eg. {{FN 1}}

QUOTE{quote "} throws an XML error

Support UL

Support attachments of specific types

Nested attachments are now supported.

Implicit hier elements

eg.:

1. The text of the paragraph

should be recognised as a paragraph.

• parser support for number + text
• unparse when number matches what parser would match
• handle escapes in parser
• apply escapes when unparsing

Support debate/hansard

Debate is a top-level element. It has a different structure to the normal hierarchy.

As an editor I'd like to be able to use Indigo's quick-edit functionality

• what rule do we use?
• what can/can't be quick-edited?
• what context is needed, eg for generating element ids?

Don't assume blockLists

rather be explicit.

• support for BLOCKLIST and ITEM
• unparse (XSL)

Validate parsed output

QUOTE must handle quote character correctly

eg.

QUOTE{startQuote "|endQuote >>}

  PARAGRAPH (a)
    Some text

Generate meta elements using passed-in FRBR URI

Bluebell parses consecutive bold tags incorrectly

eg: ****Notice**s** becomes *<b>*Notice</b>s** when it should be <b><b>Notice</b>s</b>

Drop invalid attributes when writing to XML

For block elements that the grammar allows arbitrary attributes, drop attributes that would create invalid AKN.

Support escaping keywords

Retain footnotes when unparsing crossheadings, subheadings and other headings

Be more tolerant of empty elements on a round trip

eg. <heading/> and <subheading/> shouldn't result in errors on a round trip. They don't need to be preserved, necessarily, but we should be able to handle unparsing and re-parsing them.

eg. this doesn't work, but it should SEC 2. -

ensure statement has at least a mainBody with an empty p-tag in it

Use Statement.required_children and override (part of) DocumentRoot.to_dict() to append mainBody with children = [empty_p()] if main_body has no text?

update eId rewriter in indigo-akn to match bluebell's eid rewriter

There are subtle differences, such as bluebell being more specific about which elements don't need an eId (particularly inline elements).

We should make the code match as much as possible, so we can maintain the two more easily.

Fully support attributes and classes for hier and block elements

Ensure that the grammar and the XSLT fully support attributes and PARA.foo style class names.

Some of the elements do (eg. tables and table cells), and some of the grammar does when parsing, but not when unparsing.

It should also only allow valid attribute names, and drop invalid ones.

• hierarchical elements
• BULLETS and ITEMS
• ITEM inside ITEMS
• crossheadings

Correctly escape a / at the end of an <i> tag when unparsing and reparsing

eg. <i>foo/</i> should be stable, not <i>foo</i>/ -- may be caused by having another / in the string, like <i>the type/class of/</i>

Nested embeds

eg. http://kenyalaw.org/caselaw/cases/view/196612/ quotes a judgment which quotes an act.

inline escaping still not 100%

add inline markup for editorial notes that follows {{ pattern?

• Escaping [[ is tricky in https://edit.laws.africa/documents/4739/
• {{ follows bluebell convention

Tables support

• basics
• th
• attributes
• linebreaks

Smart sections and subsections

Eg. parse:

1. some text at the start of a paragraph,

  that also wraps

as

PARAGRAPH 1.
  some text at the start of a paragraph,

  that also wraps

• parse
• unparse
• ignore escaped number when parsing
• insert escapes when unparsing

Support multi-line editorial remarks

What happens with indents? I suspect we should enforce consistent indents

eg.

SEC 1
  SUBSEC (a)
    some text
    [[this remark
      spans multiple
lines and
      indents]]

Support type on <remark>

• support attributes on remark

Synonyms

eg. PARA for PARAGRAPH

as an editor I'd like some feedback from the editor, e.g. syntax highlighting

Only allow longTitle in places permitted by the schema

Currently we allow longtitle in any block area, which is not technically correct.

When unparsing a footnote in a paragraph, the footnote disappears

Possibly may also be related to multiple footnotes in one paragraph disappearing when unparsing.

Escape hyphens when parsing and unparsing nums

eg: <num>1-</num> and SEC 2- - and SEC 2- - heading and SEC 2-

Support remaining document structures

• amendmendStructure
• collectionStructure
• portionStructure
• debateStructure

Fully support attributes for inline elements

In particular, ref.

address duplicate ids sanely

ref #21 #22

• always prefer num if there is one
• use human-friendly number if there isn't one (e.g. not 1 if it's the sixth paragraph in a section)
• don't end up with duplicate ids

e.g.

PARA 
    Intro

PARA 1.
    First para

PARA 1A.
    Added in later

PARA
    Unnumbered

PARA 2.
    Second (actually third/fourth/fifth, depending on who's counting) para.

PARA 2.
    Another para with the num 2.

PARA 2.3-4.5.
    Another para with the num 2.

PARA 2.3-4.5.
    Another para with the num 2.

PARA 2.3-4.5_1
    Another para with the num 2.

should have the following as their eIds:

• para_nn-1
• para_1
• para_1A
• para_nn-4 para_nn-2
• para_2
• para_2_1
• para_2.3-4.5
• para_2.3-4.5_1
• para_2.3-4.5_1_1

(Not finalised)

Support nested bullets

The following markup should work in theory but doesn't:

BULLETS
* Level 1

  BULLETS
  * Level 2

Footnotes must be supported in headings, subheadings and tables

• heading
• subheading
• th, td

only escape keywords?

e.g. in https://edit.laws.africa/documents/4742/ instrument\instruments: the \ was deleted (I had to use \\)

Generate eIds attribs for p tags

Support provisos

Complicated legislation like Income Tax Acts can have provisos to provisos, and provisos containing deeply nested elements. Delineating where precisely the proviso ends will help readers as well as future drafters understand the structure.

escapes: doubles and immediately between

separate dot-separated nums with dashes when generating eIds

e.g. paragraph 1.1.1. shouls have an eId ending in para_1-1-1, not para_111. See Annex of https://edit.laws.africa/documents/4608/ for an example