brechtm/citeproc-py

Yet another Python CSL Processor

citeproc-py

citeproc-py is a CSL processor for Python. It aims to implement the CSL 1.0.1 specification. citeproc-py can output styled citations and bibliographies in a number of different output formats. Currently supported are plain text, reStructuredText and HTML. Other formats can be added easily.

citeproc-py uses semantic versioning. Currently, its major version number is still at 0, meaning the API is not yet stable. However, you should not expect to see any major API changes soon.

Call for Maintainers

Unfortunately, the current maintainers lack the spare time to properly support citeproc-py at the moment. Even though there are not many bugs being reported, these are currently not being addressed. If you are an active user of this package, consider volunteering to help with maintainership. Please create a ticket to get the ball rolling. Thanks!

Requirements

citeproc-py supports Python 3.7 and up. It depends on lxml for parsing and navigating the CSL style and locale files.

Installation

The recommended way of installing citeproc-py is by using pip:

pip install citeproc-py

If lxml isn't installed, pip will try to install it for you.

If you insist, you can manually install citeproc-py from distribution packages hosted at PyPI. Please ignore the release archives offered by GitHub.

Getting Started

To get started with citeproc-py, take a look at the examples under examples/. Two examples are provided, one parsing references from a JSON representation of references as supported by citeproc-js, another parsing the references from a BibTeX file. Both show and explain how to cite references and render the bibliography.

CSL Compatibility

Currently, citeproc-py passes almost 60% of the (relevant) tests in the citeproc-test suite. However, it is more than 60% complete, as citeproc-py doesn't take care of double spaces and repeated punctuation marks yet, making a good deal of the tests fail. In addition, the following features have not yet been implemented (there are probably some I forgot though):

• disambiguation/year-suffix
• et-al-subsequent-min/et-al-subsequent-use-first
• collapsing
• punctuation-in-quote
• display

Also, some citeproc-js functionality that is not part of the CSL spec is not (yet) supported:

• raw dates
• static-ordering
• literal names

Running the Tests

First clone the citeproc-test suite so that it sits next to the citeproc-py directory. Now you can run citeproc-test.py (in the tests directory). Run citeproc-test.py --help to see its usage information.

citeproc-py's Issues

Fix issues reported by QuantifiedCode

https://www.quantifiedcode.com/app/project/61fcd880bcd04d478d659f2a8a1034ae

Not all issues should be fixed. Some are not sensible.

Problem parsing non numeric characters in pages

I'm facing a problem with references containing non numeric characters in pages field.

  File "/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py", line 77, in __init__
    self.add(self.create_reference(key, entry))
  File "/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py", line 234, in create_reference
    csl_fields = self._bibtex_to_csl(bibtex_entry)
  File "/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py", line 98, in _bibtex_to_csl
    value = self._bibtex_to_csl_pages(value)
  File "/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py", line 120, in _bibtex_to_csl_pages
    pages = Pages(first=int(decimal))
ValueError: invalid literal for int() with base 10: 'R106'

The article in bib that is causing the problem:

@Article{Anders2010,
abstract = {High-throughput sequencing assays such as RNA-Seq, ChIP-Seq or barcode counting provide quantitative readouts in the form of count data. To infer differential signal in such data correctly and with good statistical power, estimation of data variability throughout the dynamic range and a suitable error model are required. We propose a method based on the negative binomial distribution, with variance and mean linked by local regression and present an implementation, DESeq, as an R/Bioconductor package.},
author = {Anders, S and Huber, W},
doi = {10.1186/gb-2010-11-10-r106},
isbn = {1474-760X (Electronic)$\backslash$r1474-7596 (Linking)},
issn = {1465-6914},
journal = {Genome Biol},
keywords = {Animals,Binomial Distribution,Chromatin Immunoprecipitation/methods,Computational Biology/_methods,Drosophila/genetics,Gene Expression Profiling/_methods,High-Throughput Nucleotide Sequencing/methods,Linear Models,Models, Genetic,Saccharomyces cerevisiae/genetics,Sequence Analysis, RNA/*methods,Stem Cells,Tissue Culture Techniques},
number = {10},
pages = {R106},
pmid = {20979621},
title = {{Differential expression analysis for sequence count data}},
url = {http://www.ncbi.nlm.nih.gov/pubmed/20979621},
volume = {11},
year = {2010}
}

BibTex Example script doesn't work

I am trying to get this up and running, but I can't get the example script to work. I'm assuming it has something to do with my Python 2.7x environment. When I run the example bibtex script, I got this error:

/Users/smargheim/Downloads/citeproc-py-master/citeproc/source/bibtex/bibtex.py:71: UserWarning: Unsupported BibTeX field 'key'
  warn("Unsupported BibTeX field '{}'".format(field))
/Users/smargheim/Downloads/citeproc-py-master/citeproc/source/bibtex/bibtex.py:71: UserWarning: Unsupported BibTeX field 'organization'
  warn("Unsupported BibTeX field '{}'".format(field))
/Users/smargheim/Downloads/citeproc-py-master/citeproc/source/bibtex/bibtex.py:71: UserWarning: Unsupported BibTeX field 'howpublished'
  warn("Unsupported BibTeX field '{}'".format(field))
/Users/smargheim/Downloads/citeproc-py-master/citeproc/source/bibtex/bibtex.py:71: UserWarning: Unsupported BibTeX field 'type'
  warn("Unsupported BibTeX field '{}'".format(field))
/Users/smargheim/Downloads/citeproc-py-master/citeproc/source/bibtex/bibtex.py:71: UserWarning: Unsupported BibTeX field 'institution'
  warn("Unsupported BibTeX field '{}'".format(field))
/Users/smargheim/Downloads/citeproc-py-master/citeproc/source/bibtex/bibtex.py:71: UserWarning: Unsupported BibTeX field 'school'
  warn("Unsupported BibTeX field '{}'".format(field))
/Users/smargheim/Downloads/citeproc-py-master/citeproc/source/bibtex/bibtex.py:71: UserWarning: Unsupported BibTeX field 'crossref'
  warn("Unsupported BibTeX field '{}'".format(field))
/Users/smargheim/Downloads/citeproc-py-master/citeproc/frontend.py:34: UserWarning: XML file didn't pass schema validation:
/Users/smargheim/Downloads/citeproc-py-master/citeproc/data/styles/harvard1.csl:16:0:ERROR:RELAXNGV:RELAXNG_ERR_INVALIDATTR: Invalid attribute license for element rights
<string>:0:0:ERROR:RELAXNGV:RELAXNG_ERR_INTEREXTRA: Extra element rights in interleave
/Users/smargheim/Downloads/citeproc-py-master/citeproc/data/styles/harvard1.csl:16:0:ERROR:RELAXNGV:RELAXNG_ERR_CONTENTVALID: Element info failed to validate content
/Users/smargheim/Downloads/citeproc-py-master/citeproc/data/styles/harvard1.csl:2:0:ERROR:RELAXNGV:RELAXNG_ERR_CONTENTVALID: Element style failed to validate content
  warn("XML file didn't pass schema validation:\n%s" % err)
/Users/smargheim/Downloads/citeproc-py-master/citeproc/frontend.py:34: UserWarning: XML file didn't pass schema validation:
/Users/smargheim/Downloads/citeproc-py-master/citeproc/data/locales/locales-en-US.xml:2:0:ERROR:RELAXNGV:RELAXNG_ERR_NOELEM: Expecting an element terms, got nothing
/Users/smargheim/Downloads/citeproc-py-master/citeproc/data/locales/locales-en-US.xml:3:0:ERROR:RELAXNGV:RELAXNG_ERR_INTERSEQ: Invalid sequence in interleave
/Users/smargheim/Downloads/citeproc-py-master/citeproc/data/locales/locales-en-US.xml:3:0:ERROR:RELAXNGV:RELAXNG_ERR_CONTENTVALID: Element locale failed to validate content
  warn("XML file didn't pass schema validation:\n%s" % err)
Citations
---------
(Lipcoll et al. 1977)
Traceback (most recent call last):
  File "/Users/smargheim/Downloads/citeproc-py-master/bibtex.py", line 65, in <module>
    print(bibliography.cite(citation2, warn))
  File "/Users/smargheim/Downloads/citeproc-py-master/citeproc/py2compat.py", line 26, in print
    std_print(*objects, **kwargs)
  File "/Users/smargheim/Downloads/citeproc-py-master/citeproc/py2compat.py", line 25, in <genexpr>
    for obj in objects)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 11: ordinal not in range(128)
[Finished in 0.3s with exit code 1]
[shell_cmd: python -u "/Users/smargheim/Downloads/citeproc-py-master/bibtex.py"]
[dir: /Users/smargheim/Downloads/citeproc-py-master]
[path: /usr/bin:/bin:/usr/sbin:/sbin]

Examples

Hi,

thanks for this library. I've tried to find examples to see if this library fits my needs. How about creating cites and the bibliography? I'm currently trying to parse a citeproc-js JSON file from Zotero. If I have a starting point I'm willing to contribute to the examples.

Regards,
Gernot

Provide option to limit output to a given character encoding

Should convert an en dash to a hyphen. See #10.
Accented characters could be converted to the unaccented variant, but this will not be acceptable for some languages.

Picking up unsupported fields

I will admit to being a naif so this may be a silly question, but is there a good reason why citeproc-py's parser won't pick up attributes that aren't on the official list?

Sure, unsupported fields won't have a way of being used properly with a CSL. But if you had other uses for a parsed BibTeX file, or if you needed to fix some keys before using it with CSL (say you've got a file where the key is pubyear instead of year, or someone sent you a Better BibTeX file, which doesn't play nice with citeproc-py at all) it would be nice for it to pick up everything and just store it as unprocessed dict items in the Reference dict.

For my purposes I just built a secondary parser to get all this stuff and stick it in each Reference and it seems to work fine, so it shouldn't get in the way of anything. But I haven't done a lot of testing yet.

Support document type webpage in BibTeX

Webpage as a document type is apparently supported in the CSL code, but not in the BibTeX parser (bibtex.py).
Moreover, please add the field “url“ in bibtex.py.

BibTeX should accept encoding parameter at initialization?

Hi,

I am Hours old to citeproc-py so I may be not understanding things quite right yet. But it appears I've arrived just in time. BibTeXParse just recently began accepting the optional parameter encoding at initialization of the object. However, BibTeX does not accept an encoding variable and at initialization of BibTex it calls BibTeXParse. Which sure makes processing utf-8 files difficult at the moment.

Currently version of bibtex.py

    #bibtex.py line 63 and 64 
    #CURRENT
    def __init__(self, filename):
        bibtex_database = BibTeXParser(filename)

Proposed changed to bibtex.py

    #bibtex.py line 63 and 64 
    #PROPOSED
    def __init__(self, filename, encoding='ascii'):
        bibtex_database = BibTeXParser(filename, encoding)

simple "demo" run of csl_unsorted fails due to absent in repo style files

just wondered if those should be provided along?

$> PYTHONPATH=$PWD bin/csl_unsorted examples/xampl.bib    
Traceback (most recent call last):
  File "bin/csl_unsorted", line 85, in <module>
    main()
  File "bin/csl_unsorted", line 62, in main
    bib_style = CitationStylesStyle(options.style)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/frontend.py", line 60, in __init__
    raise ValueError("'{}' is not a known style".format(style))
ValueError: '/home/yoh/proj/misc/citeproc-py/citeproc/data/styles/harvard1.csl' is not a known style

but either

bibutils macros are not all supported by BibTex parser

bibutils is a commonly used converter, which can help citeproc-py users as it assists converting records to bib for importing into citeproc-py.

I've created an archive of bibutils releases at https://github.com/jayvdb/bibutils-archive , so it is easier to navigate and link to specifics.

bibutils latex macro/encoding mapping is at https://github.com/jayvdb/bibutils-archive/blob/master/lib/latex.c . It is only accessed by the functions latex2char and uni2latex.
It always emits the bib1 member of the struct for each unicode characer.

It would be good to ensure that all latex sequences it emits are accepted by citeproc-py's BibTex parser.

The following is a diff of supported macros after a quick cleanup and comparison, using only the macros matching ^[a-zA-Z]*$ (i.e. quickly excluding any of the more complex macros)

--- bibutils
+++citeproc
 aa
 AA
 ae
 AE
+b
+c
+copyright
+d
+dag
+ddag
+dh
+DH
 dj
 DJ
-emspace
-enspace
+G
+guillemotleft
+guillemotright
+guilsinglleft
+guilsinglright
+H
 i
+k
 l
 L
-ldots
+ng
+NG
 o
 O
 oe
 OE
+P
+pounds
+quotedblbase
+quotesinglbase
+r
+S
 ss
-textacutedbl
+t
+TeX
-textasciiacute
-textasciiacutex
 textasciicircum
-textasciigrave
 textasciitilde
-textbaht
-textbardbl
-textbrokenbar
+textasteriskcentered
+textbackslash
+textbar
+textbraceleft
+textbraceright
 textbullet
-textcelcius
-textcent
-textcircledP
+textcircled
 textcopyright
 textdagger
 textdaggerdbl
-textdegree
-textdiv
-textdong
-textdownarrow
-textestimated
-texteuro
+textdollar
+textellipsis
+textemdash
+textendash
 textexclamdown
-textflorin
-textfractionsolidus
-textfrenchfranc
-textlangle
-textleftarrow
-textlira
-textlnot
-textlquill
-textmho
-textmu
-textnaira
-textnospace
-textnumero
-textohm
-textonehalf
-textonequarter
-textonesuperior
-textopenbullet
+textgreater
+textless
 textordfeminine
 textordmasculine
 textparagraph
 textperiodcentered
-textpertenthousand
-textpm
 textquestiondown
-textrangle
+textquotedbl
+textquotedblleft
+textquotedblright
+textquoteleft
+textquoteright
 textregistered
-textrightarrow
-textrquill
 textsection
-textservicemark
 textsterling
-textsurd
-texttenthousand
-textthreequarters
-textthreesuperior
-texttimes
 texttrademark
-texttwosuperior
-textuparrow
+textunderscore
 textvisiblespace
-textwon
-textyen
+th
+TH
-thinspace
+u
+U
+v

Are there any of these that are not suitable for citeproc-py's BibTeX parser?

If there is general support for keeping in sync with at least a subset of bibutils latex output, we could write a test that parses lib/latex.c to ensure the defined subset is supported.

AttributeError: 'NoneType' object has no attribute 'single'

Happens with

(Pdb) p bibliography.items
[CitationItem(mckinney-proc-scipy-2010)]

where that entry as in #37 bibtex example (but I believe happened before on another one as well). Here is a full traceback:

Traceback (most recent call last):
  File "/home/yoh/proj/duecredit/venv/bin/duecredit", line 9, in <module>
    load_entry_point('duecredit===0.4.3-1-g4709', 'console_scripts', 'duecredit')()
  File "/home/yoh/proj/duecredit/duecredit/cmdline/main.py", line 171, in main
    args.func(args)
  File "/home/yoh/proj/duecredit/duecredit/cmdline/cmd_summary.py", line 56, in run
    out.dump()
  File "/home/yoh/proj/duecredit/duecredit/io.py", line 214, in dump
    self.fd.write(get_text_rendering(citations_fromentrykey[key], style=self.style))
  File "/home/yoh/proj/duecredit/duecredit/io.py", line 226, in get_text_rendering
    return format_bibtex(entry, style=style)
  File "/home/yoh/proj/duecredit/duecredit/io.py", line 273, in format_bibtex
    biblio_out = bibliography.bibliography()
  File "/home/yoh/proj/misc/citeproc-py/citeproc/frontend.py", line 105, in bibliography
    return self.style.render_bibliography(self.items)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/frontend.py", line 75, in render_bibliography
    return self.root.bibliography.render(citation_items)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 256, in render
    return self.layout.render_bibliography(citation_items)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 641, in render_bibliography
    text = self.format(self.wrap(self.render_children(item)))
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 582, in render_children
    text = child.render(item, **kwargs)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 660, in render
    text, language = self.process(*args, **kwargs)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 674, in process
    text = self.get_macro(self.get('macro')).render(item, context)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 600, in render
    sort_options=sort_options)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 582, in render_children
    text = child.render(item, **kwargs)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 1060, in process
    text = name_elem.render(item, role, context=context, **kwargs)
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 1168, in process
    et_al = self.et_al()
  File "/home/yoh/proj/misc/citeproc-py/citeproc/model.py", line 1131, in et_al
    result = self.get_term('et-al').single
AttributeError: 'NoneType' object has no attribute 'single'

BibTeX parser hangs

After converting the this MODS to bib using bibutils tool mods2bib v5.6, the output is

@Misc{pes:5137,
author="Cao, Daomin and Noussair, E. S. and Yan, Shusen",
title="Existence and Nonexistence of Interior-Peaked Solution for a Nonlinear Neumann Problem",
year="2001",
publisher="University of California",
address="Berkeley, United States of America",
volume="200",
number="1",
pages="19--42",
keywords="Ordinary Differential Equations, Difference Equations and Dynamical Systems",
abstract="We show that the critical problem \{-∆$\upsilon$ + $\lambda$$\upsilon$ = $\upsilon${\texttwosuperior}*⁻{\textonesuperior} + $\alpha$$\upsilon$q⁻{\textonesuperior}, $\upsilon$ > 0 in $\Omega$, ∂$\upsilon$∕∂$\nu$ = 0 on ∂$\Omega$, 2 < q < 2* = 2N/(N − 2), has no positive solutions concentrating, as $\lambda$ {\textrightarrow} ∞, at interior points of $\Omega$ if a = 0, but for a class of symmetric domains $\Omega$, the problem with $\alpha$ > 0 has solutions concentrating at interior points of $\Omega$.",
note="C1",
note="Yes",
issn="0030-8730",
doi="10.2140/pjm.2001.200.19",
language="en"
}

This causes the parser to hang:

  File ".../citeproc/source/bibtex/bibtex.py", line 70, in __init__
    bibtex_database = BibTeXParser(filename, encoding)
  File ".../citeproc/source/bibtex/bibparse.py", line 48, in __init__
    self._parse(self.file)
  File ".../citeproc/source/bibtex/bibparse.py", line 54, in _parse
    entry = self._parse_entry(file)
  File ".../citeproc/source/bibtex/bibparse.py", line 102, in _parse_entry
    value = self._parse_value(file)
  File ".../citeproc/source/bibtex/bibparse.py", line 138, in _parse_value
    value = self._parse_string(file, char)
  File ".../citeproc/source/bibtex/bibparse.py", line 157, in _parse_string
    char = file.read(1)
  File "/usr/lib64/python2.7/codecs.py", line 299, in decode
    def decode(self, input, final=False):
KeyboardInterrupt

Analyse other bibtex parsers

In #18 , bibtexparser (https://github.com/sciunto-org/python-bibtexparser) was suggested as an better parser, and several problems with it were highlighted.

As that package is actively maintained, it would be good to raise those problems with python-bibtexparser, as they may be interested in solving them, and solutions from citeproc-py might be able to be integrated into their package.

And maybe there are other bibtex parsers which could be used.

Fails to build with python3

Getting this error on FreeBSD 11.1 using python-3.6:

Assume we are running from a source distribution.
running build
running build_py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 128, in <module>
    'Topic :: Software Development :: Libraries :: Python Modules',
  File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 74, in run
    convert_rnc()
  File "setup.py", line 67, in convert_rnc
    root = rnc2rng.load(rnc)
  File "/usr/local/lib/python3.6/site-packages/rnc2rng/__init__.py", line 4, in load
    return parser.parse(f=f)
  File "/usr/local/lib/python3.6/site-packages/rnc2rng/parser.py", line 559, in parse
    fn, src = f.name, f.read()
  File "/usr/local/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 19994: ordinal not in range(128)

Unsupported BibTeX fields

I'm new to citeproc and trying to build bibliography from a *.bib file. However I'm experiencing that a lot of fields aren't supported. I've tried updating my libs, even installed citeproc-py from github to get some possible fixes.

/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py:90: UserWarning: Unsupported BibTeX field 'primaryclass'
  warn("Unsupported BibTeX field '{}'".format(field))
/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py:90: UserWarning: Unsupported BibTeX field 'arxivid'
  warn("Unsupported BibTeX field '{}'".format(field))
/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py:90: UserWarning: Unsupported BibTeX field 'doi'
  warn("Unsupported BibTeX field '{}'".format(field))
/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py:90: UserWarning: Unsupported BibTeX field 'eprint'
  warn("Unsupported BibTeX field '{}'".format(field))
/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py:90: UserWarning: Unsupported BibTeX field 'pmid'
  warn("Unsupported BibTeX field '{}'".format(field))
/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py:90: UserWarning: Unsupported BibTeX field 'archiveprefix'
  warn("Unsupported BibTeX field '{}'".format(field))
/usr/local/lib/python2.7/dist-packages/citeproc/source/bibtex/bibtex.py:90: UserWarning: Unsupported BibTeX field 'abstract'
  warn("Unsupported BibTeX field '{}'".format(field))

Example bib file I'm trying to read:

@Article{Trapnell2009,
abstract = {MOTIVATION: A new protocol for sequencing the messenger RNA in a cell, known as RNA-Seq, generates millions of short sequence fragments in a single run. These fragments, or 'reads', can be used to measure levels of gene expression and to identify novel splice variants of genes. However, current software for aligning RNA-Seq data to a genome relies on known splice junctions and cannot identify novel ones. TopHat is an efficient read-mapping algorithm designed to align reads from an RNA-Seq experiment to a reference genome without relying on known splice sites.$\backslash$n$\backslash$nRESULTS: We mapped the RNA-Seq reads from a recent mammalian RNA-Seq experiment and recovered more than 72{%} of the splice junctions reported by the annotation-based software from that study, along with nearly 20,000 previously unreported junctions. The TopHat pipeline is much faster than previous systems, mapping nearly 2.2 million reads per CPU hour, which is sufficient to process an entire RNA-Seq experiment in less than a day on a standard desktop computer. We describe several challenges unique to ab initio splice site discovery from RNA-Seq reads that will require further algorithm development.$\backslash$n$\backslash$nAVAILABILITY: TopHat is free, open-source software available from http://tophat.cbcb.umd.edu.$\backslash$n$\backslash$nSUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.},
archivePrefix = {arXiv},
arxivId = {cs/9605103},
author = {Trapnell, Cole and Pachter, Lior and Salzberg, Steven L.},
doi = {10.1093/bioinformatics/btp120},
eprint = {9605103},
isbn = {1367-4811 (Electronic)$\backslash$r1367-4803 (Linking)},
issn = {13674803},
journal = {Bioinformatics},
number = {9},
pages = {1105--1111},
pmid = {19289445},
primaryClass = {cs},
title = {{TopHat: Discovering splice junctions with RNA-Seq}},
volume = {25},
year = {2009}
}

pep257

There are quite a few pep257 violations found by flake8-docstrings, but mostly because they are missing. The upside of that is that the style can be easily changed now. What docstring format style would you prefer? Is pep257 ok, maybe with a few codes disabled if you dont like them?

$ flake8 --select=D --statistics
21      D100  Missing docstring in public module
80      D101  Missing docstring in public class
179     D102  Missing docstring in public method
13      D103  Missing docstring in public function
5       D104  Missing docstring in public package
17      D105  Missing docstring in magic method
1       D200  One-line docstring should fit on one line with quotes (found 3)
3       D204  1 blank line required after class docstring (found 0)
9       D205  1 blank line required between summary line and description (found 0)
6       D209  Multi-line docstring closing quotes should be on a separate line
1       D210  No whitespaces allowed surrounding docstring text
12      D400  First line should end with a period (not 'y')

Strange bug in Python 2.7

The travis build of 5289f6a resulted in Python 2.7 job failing: https://travis-ci.org/brechtm/citeproc-py/builds/100241495 .

This also happened on my sync build of the same commit, and now a PR has the same problem.

The previous build was for a few commits earlier, so it looks like a few commits were pushed at the same time to github as a series.

It looks like ac27de4 is the cause of the build breakage.

pages might be non-numeric

As reported against duecredit (where we use citeproc-py): duecredit/duecredit#125
Citing: The first one refers to this: Leonardo Uieda and Valéria C. F. Barbosa (2012). ”Robust 3D gravity gradient inversion by planting anomalous densities.” GEOPHYSICS, 77(4), G55-G66. https://doi.org/10.1190/geo2011-0388.1, while the latter refers to the following: Leonardo Uieda, Valéria C. F. Barbosa, and Carla Braitenberg (2016). ”Tesseroids: Forward-modeling gravitational fields in spherical coordinates.” GEOPHYSICS, 81(5), F41-F48. https://doi.org/10.1190/geo2015-0204.1

Here is a bibtex record:

$> http --follow GET http://dx.doi.org/10.1190/geo2011-0388.1 'Accept:application/x-bibtex; charset=utf-8'                                             
HTTP/1.1 200 OK
Access-Control-Allow-Headers: X-Requested-With, Accept, Accept-Encoding, Accept-Charset, Accept-Language, Accept-Ranges, Cache-Control
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Link
Connection: close
Content-Length: 408
Content-Type: application/x-bibtex
Date: Thu, 01 Feb 2018 15:13:29 GMT
Link: <http://dx.doi.org/10.1190/geo2011-0388.1>; rel="canonical"
Server: http-kit
X-Rate-Limit-Interval: 1s
X-Rate-Limit-Limit: 50

@article{Uieda_2012,
	doi = {10.1190/geo2011-0388.1},
	url = {https://doi.org/10.1190%2Fgeo2011-0388.1},
	year = 2012,
	month = {jul},
	publisher = {Society of Exploration Geophysicists},
	volume = {77},
	number = {4},
	pages = {G55--G66},
	author = {Leonardo Uieda and Val{\'{e}}ria C. F. Barbosa},
	title = {Robust 3D gravity gradient inversion by planting anomalous densities},
	journal = {{GEOPHYSICS}}
}

Support for languages other than English

As far as I can tell, citeproc-py doesn't support to set the language as metadata, such as -M lang:es. This is a very useful argument in pandoc-citeproc to render biblographies in other languages. Is this correct?

Decide on how to manage the CSL schema

In commit cc9546c, I replaced the citeproc/data/schema directory with a submodule pointing to the upstream CSL schema repository. I forgot that this includes the CSL schema in RelaxNG compact format (rnc), and not in the XML format (rng) supported by lxml/libxml2, so these need conversion first.

Options:

1. Include the CSL schema in the citeproc-py repository as before.
2. Make the setup.py script check that the rng files are present. However, developers not running setup.py will also need the rng files to run citeproc-test.py.

Error when handling page numbers with non-numerical characters

BibTeX class fails to process a bibtex file if the page contains non-numeric characters, such as in the following example:

@article{DESTRIEUX2009S151,
title = "A sulcal depth-based anatomical parcellation of the cerebral cortex.",
journal = "NeuroImage",
volume = "47",
number = "",
pages = "S151",
year = "2009",
note = "Organization for Human Brain Mapping 2009 Annual Meeting",
issn = "1053-8119",
doi = "http://dx.doi.org/10.1016/S1053-8119(09)71561-7",
url = "http://www.sciencedirect.com/science/article/pii/S1053811909715617",
author = "C Destrieux and B Fischl and AM Dale and E Halgren",
}

The error message is as follows:

File "/Users/michal/Documents/OHBM_Hackathon/duecredit/venvs/dev/lib/python3.6/site-packages/citeproc/source/bibtex/bibtex.py", line 120, in _bibtex_to_csl_pages
    pages = Pages(first=int(decimal))
ValueError: invalid literal for int() with base 10: 'S151'

Python 2.6 support

setup.py includes the classifier Programming Language :: Python :: 2.6 , however https://github.com/xZise/flake8-string-format reports 31 instances where the code is not Python 2.6 compatible, and the test suite is not run by Travis-CI on Python 2.6 , and Python 2.6 is not in tox.ini.

e.g. https://github.com/brechtm/citeproc-py/blob/cabe7a6/citeproc/formatter/html.py#L23 is an error, as it should be {0} instead of {}.

Rather than fix Python 2.6, perhaps it is sensible to not support Python 2.6 and remove any Python 2.6-isms, as there doesn't appear to be anyone screaming about the existing Python 2.6 bugs.

Please release 0.4.1

As people run into what has been fixed since 0.4.0 (e.g. #78 )

Handle non-ASCII characters in BibTeXParser

From python 2.7, loading a .bib file with non-ASCII characters in it crashes the parser.

(See pull request with testcase for sample .bib file.)

Escaping tags in html.py

Although the preformat() function in citeproc/formatter/html.py correctly escapes standard text, it does not escape text inside wrapped elements. I would suggest adding at least something like text=preformat(text) to line 28, to solve a problem appearing with Journal names containing an ampersand.

Question: formatter name for rst output

I am interested in rst output, so this project looks very intersting!
So far, I could only find formatter.plain and formatter.html.
What formatter name do I have to use for restructured text output?
Or is that not yet supported?

BibTeX to CSL month conversion bug

Hi,
there is a simple conceptual error in the conversion from BibTeX-style months to CSL-style months. The BibTeX months are mapped from 0 to 11 in contrast to the CSL months which are mapped from 1 to 12.

#source/bibtex.py
months = ('jan', 'feb', 'mar', 'apr', 'may', 'jun', 'jul', 'aug', 'sep',
              'oct', 'nov', 'dec')
...
def _parse_month(self, month):
...
    begin['month'] = self.months.index(begin['month'][:3].lower())
    end['month'] = self.months.index(end['month'][:3].lower())
    return begin, end
...

For January, the return value is 0, for February it's 1 and so on.

However, Date_Part.process() seems to assume a range from 1 to 12 though since the input value 0 would return "month-00" which cannot be found in the terms schema:

#model.py
class Date_Part(CitationStylesElement, Formatted, Affixed, TextCased,
                StrippedPeriods):
    def process(self, date, context=None):
...
        elif name == 'month':
...
            if form == 'long':
                text = context.get_term('{}-{:02}'.format(term, index)).single
            elif form == 'short':
                term = context.get_term('{}-{:02}'.format(term, index), 'short')
...

If I do not miss anything crucial, a simple '+1' to line 150 and 151 in bibtex.py should fix the issue.
Best Regards,
Alex

Extend/improve mapping of BibTeX fields to CSL fields

Resources:

• http://www.docear.org/2012/08/08/docear4word-mapping-bibtex-fields-and-types-with-the-citation-style-language/
• https://github.com/citation-style-language/schema/blob/master/csl-data.json
• https://github.com/citation-style-language/schema/blob/master/csl-citation.json

wishlist: support @data field in BibTeX

apparently doi -> bibtex resolution service (see http://labs.crossref.org/citation-formatting-service/) uses @data type of entry (not sure why not @misc):

$> curl -LH "Accept: application/x-bibtex; charset=utf-8" http://dx.doi.org/10.5281/zenodo.50186             
@data{61534ef1-173b-4047-ac04-5e7340ac968a,
  doi = {10.5281/zenodo.50186},
  url = {http://dx.doi.org/10.5281/zenodo.50186},
  author = {Satrajit Ghosh; Chris Filo Gorgolewski; Oscar Esteban; Erik Ziegler; David Ellis; cindeem; Michael Waskom; Dav Clark; Michael; Fred Loney; Alexandre M. S.; Michael Notter; Hans Johnson; Anisha Keshavan; Yaroslav Halchenko; Carlo Hamalainen; Blake Dewey; Ben Cipollini; Daniel Clark; Julia Huntenburg; Drew Erickson; Michael Hanke; moloney; Jason W; Demian Wassermann; cdla; Nolan Nichols; Chris Markiewicz; Jarrod Millman; Arman Eshaghi; },
  publisher = {Zenodo},
  title = {nipype: Release candidate 1 for version 0.12.0},
  year = {2016}
}

I wondered if it could be supported by citeproc-py I guess. Meanwhile I will email crossref (will CC you)

pep8-naming

This package almost passes the rules for https://pypi.python.org/pypi/pep8-naming

The four violations are all in model.py

• Date_Part
• Name_Part
• Et_Al
• Else_If

If we fix those, backwards compatible names can be added so existing users dont break.

UnicodeEncodeError: 'charmap' codec can't encode character '\u2013' in position 90: character maps to <undefined>

When it performs the Parser by a JSON structure where the type is "article-journal" when iterar:

for item in bibliography.bibliography():
    print(item)

Return the error:

Traceback (most recent call last):
  File "bibliografia.py", line 62, in <module>
    print(item)
  File "C:\Anaconda3\lib\encodings\cp437.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2013' in position 90: character maps to <undefined>

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 35: ordinal not in range(128)

Build fails on FreeBSD 11.2, python 2.7:

===>  Building for py27-citeproc-py-0.4.0
Attempting to get version number from git...
Assume we are running from a source distribution.
running build
running build_py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 128, in <module>
    'Topic :: Software Development :: Libraries :: Python Modules',
  File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 131, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.7/distutils/command/build.py", line 127, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "setup.py", line 74, in run
    convert_rnc()
  File "setup.py", line 69, in convert_rnc
    rnc2rng.dump(root, rng)
  File "/usr/local/lib/python2.7/site-packages/rnc2rng/__init__.py", line 10, in dump
    f.write(serializer.XMLSerializer(indent).toxml(root))
  File "/usr/local/lib/python2.7/site-packages/rnc2rng/serializer.py", line 76, in toxml
    return '\n'.join(prelude + self.buf)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 35: ordinal not in range(128)

Name clash with Python types module

The types module from Python3 clashes with the types.py file here. This can cause issues, in particular when using emacs flycheck integration. Maybe a minor inconvenience, but it's also a pretty minor fix. Just rename everything here from types to csltypes for example.

Appveyor builds

Do you mind if I add an Appveyor build script, either as the default appveyor.yml or using .appveyor.yml ?

http://ci.appveyor.com/

Personally I rather that it downloads the build scripts from the https://github.com/ogrisel/python-appveyor-demo project, similar to how it is done here:

https://github.com/wikimedia/pywikibot-core/blob/master/.appveyor.yml

Also, I see you recently added https://coverage.io. Would you mind terribly if we switched to https://codecov.io , as it groups jobs by commit, merging Appveyor and Travis jobs into the same coverage report.

Adding locales from external directory

I'm missing some locales. I've successfully installed missing locales copying *.xml files from https://github.com/citation-style-language/locales into my local citeproc-py clone into citeproc/data/locales/.

As far I've seen in the code there's no way to configure another external directory to look for *.xml locale files.

bibtex.py crashes

With 726bfd5 I get this (after python setup.py develop --user and python 2.7.5):

matej@mitmanek: examples (master)$ ls
bibtex.py  citeproc_json.py  xampl.bib
matej@mitmanek: examples (master)$ python bibtex.py 
/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/source/bibtex/bibtex.py:86: UserWarning: Unsupported BibTeX field 'key'
  warn("Unsupported BibTeX field '{}'".format(field))
/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/source/bibtex/bibtex.py:86: UserWarning: Unsupported BibTeX field 'organization'
  warn("Unsupported BibTeX field '{}'".format(field))
/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/source/bibtex/bibtex.py:86: UserWarning: Unsupported BibTeX field 'howpublished'
  warn("Unsupported BibTeX field '{}'".format(field))
/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/source/bibtex/bibtex.py:86: UserWarning: Unsupported BibTeX field 'type'
  warn("Unsupported BibTeX field '{}'".format(field))
/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/source/bibtex/bibtex.py:86: UserWarning: Unsupported BibTeX field 'institution'
  warn("Unsupported BibTeX field '{}'".format(field))
/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/source/bibtex/bibtex.py:86: UserWarning: Unsupported BibTeX field 'school'
  warn("Unsupported BibTeX field '{}'".format(field))
/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/source/bibtex/bibtex.py:86: UserWarning: Unsupported BibTeX field 'crossref'
  warn("Unsupported BibTeX field '{}'".format(field))
Citations
---------
Traceback (most recent call last):
  File "bibtex.py", line 69, in <module>
    print(bibliography.cite(citation1, warn))
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/frontend.py", line 102, in cite
    return self.style.render_citation(citation, self._cites, callback)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/frontend.py", line 69, in render_citation
    return self.root.citation.render(citation, cites, callback)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 239, in render
    return self.layout.render_citation(citation, callback)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 619, in render_citation
    output = self.render_children(item)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 582, in render_children
    text = child.render(item, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 1383, in process
    child_text = child.render(item, context=context, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 1383, in process
    child_text = child.render(item, context=context, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 660, in render
    text, language = self.process(*args, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 674, in process
    text = self.get_macro(self.get('macro')).render(item, context)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 600, in render
    sort_options=sort_options)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 582, in render_children
    text = child.render(item, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 1090, in process
    text = substitute.render(item, context=context, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 1299, in render
    context=context, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 1060, in process
    text = name_elem.render(item, role, context=context, **kwargs)
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 1168, in process
    et_al = self.et_al()
  File "/home/matej/archiv/knihovna/repos/citeproc-py/citeproc/model.py", line 1131, in et_al
    result = self.get_term('et-al').single
AttributeError: 'NoneType' object has no attribute 'single'
matej@mitmanek: examples (master)$

Locale fallback mechanism

The reason for the uncaught exceptions in the following tests was analysed in #45:
bugreports_GreekStyleProblems
bugreports_GreekStyleTwoEditors

Improve the BibTeX parser

• make the BibTeX parser more robust
• parse mathematics
• support for crossref and cite keys?

These resources contain detailed information on the BibTeX database format:

• http://maverick.inria.fr/~Xavier.Decoret/resources/xdkbibtex/bibtex_summary.html
• http://www.lsv.ens-cachan.fr/~markey/bibla.php?lang=en

pypy SyntaxWarning for name CslTest

When enabling the pypy & pypy3 builds, they both complain about

/home/travis/build/jayvdb/citeproc-py/tests/citeproc-test.py:307: SyntaxWarning: name 'CslTest' is assigned to before global declaration

https://travis-ci.org/jayvdb/citeproc-py/jobs/116792712

Label failing tests according to severity

We should inspect all failing tests and decide whether they represent major or minor issues. This will help to decide which we should focus on fixing first.

pages range

As discussed here
http://tex.stackexchange.com/q/58668/8569
the pages range should use -- (en-dash).
But not all .bib files follow this advice.
Currently, parsing fails if only a single hyphen is used.
Not sure if it makes sense to make parsing more stable or to require clean input.
If you go for the second option, a warning might be nice.

Both example scripts crash on both Python 2 and Python 3

On the current HEAD (9800dad), running citeproc_json.py with Python 3.5.2 yields

Citations
---------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 1129, in et_al
    result = self.xpath_search(expr)[0].render()
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "citeproc_json.py", line 147, in <module>
    print(bibliography.cite(citation1, warn))
  File "/usr/lib/python3/dist-packages/citeproc/frontend.py", line 102, in cite
    return self.style.render_citation(citation, self._cites, callback)
  File "/usr/lib/python3/dist-packages/citeproc/frontend.py", line 69, in render_citation
    return self.root.citation.render(citation, cites, callback)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 239, in render
    return self.layout.render_citation(citation, callback)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 619, in render_citation
    output = self.render_children(item)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 582, in render_children
    text = child.render(item, **kwargs)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 1383, in process
    child_text = child.render(item, context=context, **kwargs)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 1383, in process
    child_text = child.render(item, context=context, **kwargs)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 660, in render
    text, language = self.process(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 674, in process
    text = self.get_macro(self.get('macro')).render(item, context)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 600, in render
    sort_options=sort_options)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 582, in render_children
    text = child.render(item, **kwargs)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 1060, in process
    text = name_elem.render(item, role, context=context, **kwargs)
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 97, in render
    return self.markup(self.process(*args, **kwargs))
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 1168, in process
    et_al = self.et_al()
  File "/usr/lib/python3/dist-packages/citeproc/model.py", line 1131, in et_al
    result = self.get_term('et-al').single
AttributeError: 'NoneType' object has no attribute 'single'

(See #38 for the second error.) Running it with Python 2.7.12 yields:

Traceback (most recent call last):
  File "citeproc_json.py", line 5, in <module>
    from citeproc.py2compat import *
ImportError: No module named citeproc.py2compat

The other example script, bibtex.py, produces similar errors.

csl test suite has moved to github

The bitbucket repo for the test suite has been moved to
https://github.com/citation-style-language/test-suite

See
https://github.com/brechtm/citeproc-py/blob/master/tests/citeproc-test.py#L23 and friends.

Custom data path

Hi,
i'm currently having problems with the old schema files in the citeproc data folder. They are more than two years old...

Do you think that it might make sense to be able to pass a custom data path to the library. Styles, locales and schema directories could be updated by the user himself.

Thank's again!

Tarball inconsistencies

The distributed tarballs of 0.3.0, the one at Pypi (e23da3b6c75485f82e1e5a725235731e) doesn't contain CHANGES.rst (not that big problem), but the one from Github (ccabf6bb628a598c3737aea5572b457e) misses citeproc/version.py, which results in an installation break of setup.py.

Version 0.1.1

Thank you for your great work! Would it be possible for you to make a 0.1.1 release with the recent updates, in particular with respect to the formatter, and push it on pypi (I am using some features in bibgen on pypi)? Also, updating the setup.py in the git repo to a later version than the one released would be great. Best regards, Xīcò.

Proper way to generate HTML outout

I've tried to generate html output, following the examples, but I only get lists of strings with no meaning at all.
I'd like to understand, how to get html from the parser and how to modify the html formatter, so that each semantic element has some form of tag around it.

Non-standard bibtex field 'issue' used by bibutils

The correct BibTeX field for journal issue is confusingly called number, and number is used for other output types - e.g. series number for books, etc..

Before 0254247 , BibTeX number was mapped to CSL number, but that was corrected to map to 'issue'.

When a record contains both an issue and a number, bibutils emits both.
If a record contains either an issue or a number, bibutils emits a number.

There is a comment explaining why in the bibutils source.
https://github.com/jayvdb/bibutils-archive/blame/master/lib/bibtexout.c#L508

/*
 * from Tim Hicks:
 * I'm no expert on bibtex, but those who know more than I on our mailing 
 * list suggest that 'issue' isn't a recognised key for bibtex and 
 * therefore that bibutils should be aliasing IS to number at some point in 
 * the conversion.
 *
 * Therefore prefer outputting issue/number as number and only keep
 * a distinction if both issue and number are present for a particular
 * reference.
 */

static void
output_issue_number( FILE *fp, fields *info, int format_opts )
...

I suggest that citeproc-py follows suit.

Please release 0.3.1 or 0.4.0

There was a reasonable number of fixes introduced since 0.3.0 (just ran into pages parsing in bibtex fixed in 8161762). Given that the last release (0.3.0) was almost 2 years ago, it would be awesome if new, at least a bugfix, release was cut

thanks in advance!

Remove lxml dependency

ElementTree from the Python standard library should be fast enough. However, it is missing some of lxml's features:

• ElementTree only has limited support for XPath expressions
• RelaxNG validation

The XPath searches in model.py might require a lot of code to replace.

Losing validation is not a big deal IMO, as a downloaded CSL style file will probably have been validated already.