How to do multilingual websites?

[docpad]

How to do multilingual websites?

[balupton]

The obvious way for this to work is to specify a locationlisation file which is loaded in, then any text on the website is passed through a translation function. E.g.

<%= @t 'This text will be translated' %>

[balupton]

Perhaps we could take this a step further and create a plugin that uses the MyGengo API...

[kapooostin]

At the moment Docpad fails to compile and outputs row data from src if I use Cyrillic in variables, for instance in title.

[balupton]

Hrmmm, crap. Can you send me an example file to [email protected] or provide me with the repository that fails.

[kapooostin]

It disappeared on its own. I'll let you know if it happens again.

DocPad is wonderful, the only thing I miss now is documentation on
variables available inside documents and layouts. At the moment I use
layout itself to iterate over document properties.

[balupton]

Just released an update v5.1.2 which has a different binary/utf8 encoding detection, try an upgrade and let me know how it goes :)

[kapooostin]

Like a charm :)

On 26 April 2012 04:32, Benjamin Arthur Lupton <
[email protected]

wrote:

Just released an update v5.1.2 which has a different binary/utf8 encoding
detection, try an upgrade and let me know how it goes :)

---

Reply to this email directly or view it on GitHub:
#17 (comment)

[balupton]

Bumped. The infrastructure is more or less in place now to make this happen.

[balupton]

Different ways I've ome up with for i18n:

Way 1. move all text on your website to your templateData, then re-generate your website for each langauge rather than your own and put these regenerations into a sub directory (e.g. /out/fr/ /out/de/) etc

Way 2. write a plugin that scans for text on each of your documents, and looks for the corresponding text in your templateData

[magnusdahlstrand]

Way 3. Use multiple directories in your "documents" folder, one for each language (the default language doesn't need its own folder): https://gist.github.com/3958441#file_documents%20folder%20layout.

These subfolders are mirrors of each other, just that the content is in different languages. I used templateData for site wide translated content, with helper functions on the site-variable that extracted the locale from the current file's path, which made usage easier.
I've used this method for a recent project, and it worked fine with five languages and 60+ pages/language.

[balupton]

For dates I've done up this example: https://gist.github.com/4166882

Still need a more solid way to translate text.

[Artmobile]

I think the most natural way is through the routing (rails style). Usually shops that provide multi language support do so by controlling a CNAME, a resource or the query string:

http://en.mysite.com
http://www.mysite.com/en
http://www.mysite.com?lang=en

In Rails I would use something like the following:

In the routing I decide which controller page I go to depending on the route:

scope "(:locale)", :locale => /fr|de|en/ do
    match '/home' => 'main#home' 
    match '/flights' => 'main#flights'   
    match '/hotels' => 'main#hotels'
    match '/' => 'main#main' 
end

In the controller (or in middleware):

def set_locale
    I18n.locale = params[:locale] || extract_locale_from_subdomain || I18n.default_locale
end 

Then in the ejs/erb it's just a matter of a t translation:

<%=t :travel_place_airport%>

I wonder how the above is possible in DocPad

I would assume that using express on top of DocPad will do the trick. Unfortunately the example provided in http://bevry.me/learn/docpad-api runs express alongside with DocPad on a different port so the express routing cannot be used.

[balupton]

So perhaps the solution can be:

1. Language files that are imported into the template data.
2. Generations for each language.
3. Routing to go to the correct language site.

Seems pretty straightforward. I'd love to see jed or messageformat used in this. As well as a plugin done up to do this automatically.

Alternatively, for a half assed approach, a plugin could be done that passes each page through google translate and saves them to the appropriate language site...

[balupton]

Must read for everyone wanting to attempt this: http://alexsexton.com/blog/2012/03/the-ux-of-language/ and http://alexsexton.com/blog/2012/10/introducing-the-jed-toolkit/ :)

[Artmobile]

Thanks Ben. Jed seems to be a nice approach. How can put a Jed dictionary into my metadata section? I did not find a way to do that.

[balupton]

@zenorocha any chance of getting a nice guide on how you fellows are doing it? I know you're rocking it up on the localisation front!

[zenorocha]

Sure, I can work on it, maybe next week :)

[nkhine]

how about using the i18next library https://github.com/jamuhl/i18next-node

[kendonaldson]

@zenorocha Did you ever get a guide put together. I'm starting to add Mandarin support for my site and I'd love to see how you did it.

[glukki]

// humble bump

[adam-p]

If anyone would like to see an example of a reasonably-well-functioning Docpad-based localized site, you can check out ours: here's the source (on Bitbucket... boo), here's the Transifex project for the translation, and here's the site itself. Not all of the languages are as fully translated as others, but these ones are pretty complete: English, Spanish, Farsi, Arabic, Chinese, Russian.

I always meant to write up a description of how it work, but I didn't, so... If anyone has any questions about it I'll be happy to answer them. The two main issues that come to mind are: it would be great if Docpad layout metadata were combined into the document's metadata (Docpad issue #458), and the compilation of the site is slooooooow -- a full minute on a pretty fast machine. I don't have a clue why it's so slow.

[nylnook]

// Humble bump too

I'm looking for a solution for my website, and not really convinced with your solution adam-p (look like too much work : each string have to be referenced ... but the Transifex solution is seducing), nor the browser-diet website approach : it's ok to duplicate pages for each language if you have only 3 pages... but what happen when you have one hundred ?

So a jed solution look like the more future proof one ;)
I'm too newbie to docpad to implement it, I'm afraid...

[balupton]

A new guide by @sapegin is up here: http://blog.sapegin.me/all/multilingual-docpad

[nylnook]

Thanks a lot balupton and sapegin !
I hadn't found that one ;)

It's almost what I was looking for, I'll try to adapt it to my use case !

[balupton]

Sweet. Let's give it more prominence in our documentation before we close this issue.

[nylnook]

Sorry, this is not working...
I do some tests since yesterday to find a solution, but I can't figure out a proper way.
sapegin solution work for templates and template data, but it still duplicate everything.

Maybe my use case is out of the scope of this bug, but here is a full description of the issue :

• I'd like to translate to two languages at a minimum (english and french), and maybe add more languages when time come
• I don't care to automatically redirect visitors to a sub-directory or a sub-domain, but I'd like to display the text in the correctly detected language (somehow with a server req.headers["accept-language"] detection). URL may be the same, but language is visitor specific.
• I have a lot of files (images), actually around 300, just starting, and plan to have thousands. That's the reason why I can't duplicate "out" directories all the way : disk usage would go crazy (multiplied with each language)
• So I need to render only text documents (the document directory) for each language. Not the files or any other directory (duplicating the minimum).
• you can imagine that with a "documents/en/" dir, next to a "documents/fr/" dir, and so on, or with parts of file names containing language string like "firstpost_en.html.md" and "firstpost_fr.html.md", or a conditional statement in template files.

The part I can't figure out is how to tell docpad if visitor language is this one, use this html file server-side. I can do that client side with javascript detection and "localstorage" I suppose, but that's not optimal, require visitor approbation, with the no-script user case, and so on.

Any idea ?

[sapegin]

@nylnook Just put your images outside DocPad src/files directory.

[nylnook]

Thanks for your answer, but I'm afraid that's not a solution : I also need to parse them with associatedfiles...
...
or I have to think to another way of managing them...

[renoirb]

I have the same pain @nylnook and i doubt that de-duplicating file names with filename patterns (e.g. _fr.html.whatever) from folders is better. At least its dead obvious, the right file would be loaded. Remember that there's no server-side code to decide which file. Unless you make that distinction with mod_rewrite (you could add _fr there, though).

We will possibly work on the issue ourselves too for webplatform.org as we need more than one language for our homepage documents. I'll keep this issue updated whenever I see progress.

[nylnook]

So... maybe with node-reverse-proxy ?

[adam-p]

There is no duplication of files (images, etc.) in our site. If you go to http://play.psiphon3.com/en/user-guide.html you'll see that the images on that page load from /images/user-guide, regardless of the language of the page. It's only the documents that get duplicated.

I can't help with server-side language routing, since our site is statically hosted in S3. (We do some client-side detection and redirecting. I've been meaning to also add a cookie+JS to remember manual language selection.)

[nylnook]

Hi @adam-p,

I must admit I underestimated your work... It's your templates files with a lot of strings that confused me : it looked like everything should be into variables.
But your solution on the blog part look like what I need !

On the redirect part, maybe we can do something with the "serverExtend" event at the end of docpad config file... to redirect visitor to an appropriate subdomain.

So I'll try it next week, and keep you updated ;)

[zenorocha]

IMHO the DocPad team should prioritize a solution to address this problem. It's affecting lots of big projects.

[balupton]

The DocPad team has prioritized a solution to address this problem. It affects lots of big projects. The solution we've prioritized is to have those coding multi-lingual sites explain how they've done it #17 (comment) so that we can focus on the priorities that affect everyone that only we can do, utilising everyone's limited time the best we can.

[nylnook]

Not much time this week, but I'm slowly progressing on the subject, and starting to have a working solution inspired from everyone work. Once ready and documented I'll report here.

[mikeumus]

👍 @nylnook Cheers! 🍻

[nylnook]

After a lot of messing around, I found a working solution for my case at least...
And I'm proud to announce I documented it !
I'm not a coder, I'm a designer, so don't expect it to be perfect. But I humbly believe it's a good basis for future works.

My code : https://github.com/nylnook/nylnook-website
The blog post / documentation : http://nylnook.com/en/blog/docpad-i18n (need proofreading, but it's late in the night for me)
My localized website : http://nylnook.com/

Many thanks to @balupton @zenorocha @adam-p and @sapegin for their work, without you, I wouldn't have achieved this work.

[zenorocha]

Really nice article @nylnook, thanks for all details you've provided :)

[mikeumus]

Agreed. @nylnook your article is the real deal. Thanks for putting that beast together. If @nylnook's approach works for eveyone trying to do localization then I'd say we can close this issue and refer people to it and/or make a Wiki link to the guide.

[nylnook]

No more compliments ;p
As already said, I'm glad it will be useful !

If the Docpad team want to copy and paste it (or parts) to Docpad documentation (maybe with some english correction, I'm afraid it's not perfect), that's totally ok consider it's under MIT licence and my tribute back to this great project !

[greduan]

I can look into testing this stuff and adding it into the documentation. Just @balupton has to give his OK to make sure there's nothing wrong before I do so. :)

The DocPad documentation (which I think includes the code in it) is licensed under the http://creativecommons.org/licenses/by/3.0/ license, is it OK if the MIT license is dropped (of course providing plenty of credit) or would you prefer it it keeps the MIT license?

[nylnook]

CC-by is ok ;) I use Creative Commons for my pictures already, so why not for text, and MIT is very similar to CC-by.

I'm glad to contribute. However, my code clearly need a review : some months ago, I didn't write anything in nodejs, and I learned with Docpad. So take your time to check it.

[balupton]

So here's my advice.

Use different environments for different languages. Each environment should:

1. add their own documentsPaths for that language
2. add a templateData property for the locale so your templates can render the menus etc
3. change outPath to something specific to that locale

You then run docpad generate --env <locale> for each locale.

That way:

1. your templates all stay the same, but the content can be locale specific
2. it will generate different sites for each language
3. they can go at a subdomain or subfolder
4. routing to the different sites can be handled by cloudflare or whatever