Support for asynchronous content plugins?

[jnordberg]

Support for asynchronous content plugins?

[jnordberg]

You could do it in the render method

  render: (locals, contents, templates, callback) ->
    ### *callback* with a ReadStream/Buffer or null if the contents should not be rendered
        *locals* rendering context variables
        *contents* is the full content tree
        *templates* is a map of all templates as: {filename: templateInstance} ###

[lhagan]

I'm pretty sure this will work for regular pages, but not things like the blog example's main page since it pulls in multiple articles from other markdown files.render would only be called once for the main page and not for any of the articles, right?

[jnordberg]

Ah yes, you are correct.

The problem touches upon the same underlying issue as #45 – all view logic is done in templates. And in this case you would need a templating language capable of doing async calls to avoid preloading any async content you (maybe) want to access.

I'm thinking of introducing a view layer (like contents -> view -> template) in wintersmith 2.0. But i'm not sure yet how to go about it without adding another layer of complexity :)

I think pre-rendering it in fromFile is the best option right now.

[lhagan]

The issue with fromFile is that in preview mode, sometimes the page gets rendered before Pandoc is done with the content. Not sure if that's a bug.

[jnordberg]

That sounds really strange. Are you sure you are not calling any callbacks prematurely?

[lhagan]

I don't think so, but I'm no async expert. The call to pandoc is near the end of the waterfall and I didn't make any other changes to the process:

PandocPage.fromFile = (filename, base, callback) ->
    async.waterfall [
      (callback) ->
        fs.readFile path.join(base, filename), callback
      (buffer, callback) ->
        extractMetadata buffer.toString(), callback
      (result, callback) =>
        {markdown, metadata} = result
        page = new this filename, markdown, metadata
        callback null, page
      (page, callback) =>
        pandocRender page, callback
      (page, callback) =>
        callback null, page
    ], callback

[jnordberg]

The waterfall looks ok, try dropping the async queue in your pandocRender method and see if that helps.

I also did a quick test and wintersmith behaves well even if a plugin takes a long time to complete

module.exports = (ws, callback) ->
  delay = 10000
  page = ws.defaultPlugins.MarkdownPage

  class Delay extends page

    render: (args...) ->
      setTimeout (=> page::render.apply(this, args)), delay

  Delay.fromFile = (args...) ->
    setTimeout (=> page.fromFile.apply(this, args)), delay

  ws.registerContentPlugin 'pages', '**/*.*(markdown|mkd|md)', Delay
  callback()

[lhagan]

Unfortunately, without the queue node runs up against a limit for number of open files and crashes on larger sites. I'll try to use your example code above to see if queue is the culprit.

[lhagan]

It looks like the issue must be a subtle problem with pandoc or its wrapper. Everything works fine with another converter.

[jnordberg]

Ok, closing this