Pandoc integration

[brrd]

Pandoc integration

[seanlano]

Abricotine is my new favourite program! I'd also love to see some kind of basic pandoc integration, it's about the only thing that it cannot do (for my use case, at least).

It could be something (relatively) simple, perhaps export is only enabled through an advanced option where the user would need to configure the path and parameters for pandoc on their system. It would be up to the user to manually check that pandoc can run and take the Markdown file produced by Abricotine, but this command could be called by an "Export to PDF" button. For me at least, this is pretty much how I create the PDF output -- so this option would really just provide a nice integrated button for my terminal command.

The advanced option could take some argument that is directly run on the system, i.e. pandoc --latex-engine=xelatex --filter pandoc-crossref --template=/usr/share/local/abricotine.latex %infile -o %outfile where Abricotine replaces the % values with the document path.

[brrd]

Hi,

I think that Pandoc as so many options, especially for Markdown, that this can be a quite huge work if we want to handle them all.

But I can imagine a postSave hook in config that could trigger any command (such as pandoc path/to/temp/md-file.md -o user-defined/output/path [add your options here]) when the file is saved with a specific command/menu. Such a system could also be used with any other command than Pandoc.

Do you think this could do the job?

[brrd]

@seanlano Of course PR are welcome!

Here are my advices:

(1) First you will probably need to add an entry in config.json (https://github.com/brrd/Abricotine/blob/develop/default/config.json), like:

  "tasks": [
    {
      "name": "Pandoc PDF",
      "command": "pandoc --pdf-engine=xelatex --template=path-to-template %in.md -o %out.pdf"
    }
  ],

Beware "command" already exists as a concept in Abricotine. This is why I would rather prefer to name those actions "tasks" or "scripts" or something else.

(2) Then you will have to dynamically create the menu in abr-menu.js. You can get inspired with other menus (spelling, html export and themes) which are already dynamically generated: https://github.com/brrd/Abricotine/blob/1f0b1da749616ef3fb457f3429576ccc8f811218/app/abr-menu.js#L53

task.name would be the label of the menuItem. Maybe here you will also decide to store task.command in the menu item in order to pass it to the command when the menu is clicked?

(3) To make the menu work, you first need to write the function which executes the command (probably using child_process.exec). I think this should be included in abr-document.js (one day I would like to split this big file in smaller modules, but never mind) because there you will be able to export the editor's contents.

(4) We need to ask the user for a save location, so you will probably also have to do something in dialogs.js.

(5) Finally we need to create a "task" command in renderer/commands.js. The command is the one which will be called by the menu on click.

Knowing Abricotine architecture, this is the way I would try to implement this feature, but feel free to adjust it if you have a better idea.

[guillecro]

+1 for PDF export <3

[seanlano]

@brrd I've not done much anything with Electron-based apps before, but I've done some JS and plenty of Python and C++, and this would probably be a not-too-difficult task to try my hand at - where would you suggest I start with trying to implement this? I can have a go and make a pull request.

[nathanlesage]

@inoryy I've kept the exporting class of my own app, , rather modular. It would be easy to rewrite it and offer it as an NPM package, so in case @brrd still wants to implement it, I would offer to do the refactoring and publishing of the class via npm. In this way, one could simply require it and give it the source file, some options and, optional, a destination file (default: Source filename with different extension).

The necessary API is basically one single line (plus additional error handling, because the class implements throw).

It currently exports markdown files to 12 different formats (in the current dev build, not yet on master) using pandoc and, in case of PDF, the xelatex engine.

The class: https://github.com/Zettlr/Zettlr/blob/master/source/main/zettlr-export.js

How it's called within my editor: https://github.com/Zettlr/Zettlr/blob/346db5cc148e4b6f796adea341b2135f24ddc029/source/main/zettlr.js#L556

[brrd]

@nathanlesage Oh ok, sorry ! That's my mistake 😊

[tmr232]

+1

[rrooij]

Yes, it would be a lot of work to handle all the options.

Before discovering this project, I thought about making a full fledged document editor using Markdown + Pandoc. It would provide a good cross platform solution to most Office document editors. Then I discovered this project and found almost exactly what I was thinking about, except without the pandoc integration.

However, I understand that it is a lot of work. A postSave hook could definitely be a good option, but it is only a good option for power users. My aim was an approach for the masses as a document editor.

But having a postSave is obviously better than having no option at all. And I understand the need to be pragmatic in this case, as real pandoc integration would be lots of work.

[framatophe]

Yes it would be a lot of work ! I suggest the following options instead :

1. select only some of pandoc's elements which are useful (but you have to explain your choice, etc.)
2. create a new feature which aims to authorize the user to write its own "enlightenment" . For example: I would like that the Abricotine's editor put an orange color on all sequence that begin with [@ and ends with ] ... (in the config.json ??). Then I will be able to customize my own Abricotine with my own style (including a pandoc-style-like).

[CxRes]

How about leveraging a library like node-pdc for Pandoc Document Conversion support as a start with vanilla Markdown.

In the interim one could use something like PanDoctor or PanConvert.

[Sunsheep]

+1 for Run script files or custom (export) commands for post-processing via pandoc to latex, pdf or other formats. This would greatly simplify the export process and also bring the power of markdown to non technics.

Pandoc commands would work like @seanlano meantioned:

pandoc --pdf-engine=xelatex --template=path-to-template %in.md -o %out.pdf

[seanlano]

Thanks! I'm pretty busy at the moment too, but I will get onto this at some point.

I like your idea of the scripts config item - I was thinking that there would just be one "Custom Export" option, but there's no reason why it couldn't be a dynamic list of scripts that could be run as required - so I'll progress along that path.

[brrd]

@seanlano This is just an idea, of course we can start with a single custom export first and develop the dynamic tasks later.

[inoryy]

@brrd @seanlano what's the status of this ticket?

[seanlano]

@nathanlesage Unfortunately I've not had the time to look at it these past few months - so it's still just as it was (i.e. not implemented). I like the sounds of your suggestion though!

[nathanlesage]

@seanlano Should I read this as a 'Go' for releasing my export engine separately? If so, it would take time either way to fully port it out of Zettlr and into the npm database, but the earlier I know the earlier I can figure out when to do this!

[seanlano]

Well, not my call really - it's not my project, I think all I can say is that it's not looking like I'll be doing anything myself to try and add the export feature any time soon, so feel free to take it and run with it. Good luck!

[inoryy]

Actually turns out it is fairly easy to achieve on Linux with pandoc.
Here's a one-liner solution (also fixes the annoying $$ format):

sed 's/\$\$/\$/g' text.md | pandoc -o text.pdf

[brrd]

@inoryy The status of this ticket remains the same until someone develop this feature.

@nathanlesage Thank you very much, but I think this option would be more complicated to implement compared to the main idea which was to add the ability to trigger simple custom command lines with custom entries in menu. I don't want to dissuade you from releasing a package from your code, though. It can be useful to others of course.

[nathanlesage]

@brrd so basically you want to automate only certain nasty commands? It sounded different, but alright. Maybe I’ll get to it some day!

_{Sent with GitHawk}

[brrd]

@nathanlesage Why is this nasty? My idea is to give users the freedom to use the tool and the options they want by declaring them in the config. IMHO command-line is very suitable for this, since Pandoc is firstly a command-line tool (as I know, libraries built upon Pandoc are just wrappers around its commands). It is more simple to implement, but it is also very suitable for average users (we just have to declare a basic export with Pandoc in default config, which would work well for mostly everybody) and also for users who want more options (they can declare new CLI with different options in config.json). So in my opinion this solution is not that bad. But maybe there is something I don't see?

[nathanlesage]

@brrd No, no, this is a misunderstanding! I meant "nasty" with respect to the commands you'd have to type in to get pandoc do what you want. It's always a hassle to re-type those commands, which is why I think your idea is great for maximum freedom.

Actually, not having to write those pandoc commands all over again was the initial idea I began Zettlr with (firstly, it was only designed as an app you could throw documents at and drag out of it a converted one).

That's what nasty meant, and some pre-configured commands may work very well to get users started. I actually already might have an idea for how to do it. But I'd need to dig into the logic of Abricotine a little bit more, I haven't managed to understand it fully, yet (especially the MathJax-rendering, which I'm currently trying to implement using my approach of new commands).