Multiple language support?

[ponzu-cms]

Multiple language support?

[nilslice]

Hi @paultyng -

Sorry for the delay in response.. I wanted to give this enough thought to offer some actionable feedback for you or anyone who might want to work on this feature. Lately, I've just been swamped with a client project that is about to launch.

I would imagine this could be best achieved through a new system package for i18n, containing a Translateable interface which when implemented would do a few things:

1. have a method to define the languages supported, which would in turn create a new bucket for each language, like MyType__translated__zh_CN, MyType__translated__en_US (if English is not the primary input language). This would look something like:

func (m *MyType) Translations() map[string]string{} {
        return map[string]string{
                "zh_CN": "Chinese (Simplified)", 
                "en_US": "English (US)", 
        }
}

2. create and register new API endpoints to return or add/edit a translation, i.e. /api/content/translated & /api/contents/translated which would mirror the various actions of /api/content but instead of changing or retrieving data in the main content buckets, it would do so on the bucket dictated by some query param in the URL.. for example, /api/content/translated?type=MyType&id=3&lang=zn_CN would return the results for a call to db.Content("MyType__translated_zn_CN:3"). Maybe these routes are registered in the init func, so they're only activated if this package is imported somewhere.

3. define some UI toggle in that would appear in the Editor or Manager for the MyType inside the CMS, which would return the same UI as seen currently, but would set some query param to tell the DB which language bucket to pull content from or to set content in. This would be similar to how the editor.Mergeable/api.Createable interfaces impact the UI.

I'm sure I am missing some details, but I think this could be done without too much impact on the existing design.

[nilslice]

@martinsotirov -

The bucket structure would be better suited to have the translations a sibling bucket rather than a child. The reason being, all of the Content APIs and Database APIs rely on this for fast lookups, and tend to favor quick iteration through a bucket to find a key.

The separation of these buckets would allow us to continue this pattern and choose a different language very easily just by issuing a param name to append to the DB lookup. See the system/db package, particularly content.go to see what I mean.

[mordillo123]

Hi all, no any updates about this request feature?
Best regards!

[olliephillips]

Hey @mordillo123 thanks for your question. The issue's labelled "help wanted" so remains on the radar, but like all open source, time resource is constrained. With this issue also, there are particular skills/experience re i18n needed, which is why - I would guess - there's been no movement.

[nilslice]

Hi @paultyng -

I haven't really put much thought into i18n specifically for content (added to the CMS by a user). I think the first step for any i18n would be for the hard-coded names for things in the CMS itself, like navigation, section titles, buttons etc.

My experience in this area is fairly limited, so any suggestions you have would be fantastic to hear. I currently don't have a plan for this.

I'm not sure there would be an elegant way to do i18n for the content a consumer ends up adding though, at least not in a way that would be 'automated' (i.e. generating a sibling field for adding content in different languages)

What do you think?

[paultyng]

Adding it to the UI I think makes sense, and would be relatively straightforward with some of the existing libraries. Below are some of my basic thoughts on the adding it to the consumer system, but I'm not sure its worth it, mainly I was curious if it was on your roadmap in the first place.

In some internationalized systems I work with now, when a person creates/edits content in their language, its submitted for translation to some third party SaaS queue that handles managing translators and all that and then gets fed back in automatically when translated via like a webhook, although asynchronously and attached to the original record as an additional version of it. Occasionally though the consumer knows multiple languages and can edit them both in real time in the UI. One of the weird things is that the titles (and slugs) can change between languages and so forth, so it may just not make sense to add on to this system, I'm not sure as it would probably quite significantly affect the architecture.

If anything I could see changing the localizable fields to a new type that is something like map[LanguageTag]string for "translatable" resources / fields. And then you just need the ability to specify the requested languages (typically a coalescing structure, of like chosen language, accepted languages, app default language) values when requesting it in the api (so you don't just return all languages to consumers unnecessarily).

[joeblew99]

awesome that this is being looked at...
The ponzu API for i18n looks like a good start.

Here is a golang lib for it. i did a fair bit of i18n and there is a huge amount to it once you dog deeper, so its important to choose your weapons well :)

https://github.com/corestoreio/csfw/tree/master/i18n

• Has brought all the golang libs into one and covered ALL aspects for i18n
• covers both text, but also numbers, currency and RTL / LTR.

[martinsotirov]

i18n is really important for ponzu to be viable CMS, I think.

I'm not a BoltDB expert but can't we have each content type have a root bucket and then sub-buckets for each language?

[torniker]

I'll try to tackle this.

I'll create translation branch and do the work there. @nilslice If you'll have some time please review the code from time to time.

I'll post updates here so I wouldn't do something out of scope or make some breaking changes.

Wish me luck :)
Cheers

[nilslice]

Thanks, @torniker - just a heads up, with the CI broken, we're not in a good spot to accept PRs. I'd ask that any incoming code include some cleanup of the CI so that the project is tested and built upon each PR. Once the CI is fixed, I'm happy to look through PRs.

Also, that is just my opinion, but the project is now under the ownership of apilayer.com via @julian-zehetmayr - who may have a more flexible policy on CI.