leolabs/json-autotranslate

Translate a folder of JSON files containing translations into multiple languages.

json-autotranslate's Issues

how solve this issue about new type of json for autotranslate?

https://gist.github.com/ab1bc793fd801ebf3c125520229f0584

🔍 Looking for key-value inconsistencies in source files...
├── en.json contains 459 inconsistent key(s)
└── Found key-value inconsistencies in 1 file(s).

When files do not exist in 'destination' language, vague error is thrown

Line 388 here will error with a vague '.name doesn't exist' error when I have a 'help.json' file in my 'en' directory, but 'help.json' does not exist in my de directory.

After looking through your source, I understood what was going on and made those files by hand, but a clearer error message might be helpful.

Thank you for the library! It's been very useful and saved me hours of my life. I hope you're well!

Does not translate to French.

I ran command json-autotranslate -f -i locales -m i18next -c .\grand-store-292020-1d6d5dbdee41.json and this is what I got.
My french json has all keys but no values.

Found 1 target language(s):
-> fr

🏭 Loading source files...
├── translation.json (natural)
└── Done

🔍 Looking for key-value inconsistencies in source files...
└── No inconsistencies found

🔍 Looking for invalid keys in source files...
└── No invalid keys found

💬 Translating strings from en to fr...
├── Translating translation.json (0 changes from cache) (+0)
└── All strings have been translated.

🗂 Caching source translation files...
└── Translation files have been cached.

0 new translations have been added!

Throws error with double-brackets "{{" "}}" in json

I need to translate strings with {{ }}, which is a nesting variable symbol of the popular i18next library. Here is an example.

"KEY": "Words {{var1}} words {{var2}}",

While single { } works just fine, {{ }} throws below error

💬 Translating strings from en to ar...
├── Translating common.json (1 changes from cache)
An error has occurred:
Expected identifier but "{" found.
SyntaxError: Expected identifier but "{" found.
    at peg$buildStructuredError (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/messageformat-parser/parser.js:407:12)
    at Object.peg$parse [as parse] (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/messageformat-parser/parser.js:1978:11)
    at exports.matchIcu (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/json-autotranslate/lib/matchers/icu.js:6:40)
    at Object.exports.replaceInterpolations (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/json-autotranslate/lib/matchers/index.js:20:24)
    at GoogleTranslate.<anonymous> (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/json-autotranslate/lib/services/google-translate.js:113:57)
    at step (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/json-autotranslate/lib/services/google-translate.js:33:23)
    at Object.next (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/json-autotranslate/lib/services/google-translate.js:14:53)
    at /Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/json-autotranslate/lib/services/google-translate.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/ipadtinkamo/Documents/Tinkamo/tinkamo-mono/node_modules/json-autotranslate/lib/services/google-translate.js:4:12)

I have to manually replace {{ }} to << >> and reverse it before / after using json-autotranslate. Please fix it if possible. Thx!

Update cache with already translated key-value pairs

I'm migrating from intl to i18next and using json-autotranslate to add new translations. When i run script to generate new translations:

json-autotranslate -i public/locales -s manual --type keybased -l en

It asking for the already existing key-value pairs in all files (the cache is not updated, i think).

There is a way to update cache with already translated key-value pairs in all files?

Id like this feature because i add .json-autotranslate-cache/ to .gitignore and the idea is run something like 'json-autotranslate -updatecache' to verify and update the local cache before run the generate translations script.

REQUEST: XML, YML, JS autotranslate

i hope have you a little of my time for create version for xml, yml and js instead json
Greetings

Doesn't work with the new "DeepL Free plan"

Looks like the API endpoint need to be changed from

var API_ENDPOINT = 'https://api.deepl.com/v2';
to
var API_ENDPOINT = 'https://api-free.deepl.com/v2';

when using a free plan.

(I guess that's a new plan on deepL, I didn't saw it before )

maybe we should have a service "deepl-pro" or "deepl-free" ?

i18next plurals

Hi @leolabs, first, thank you for this wonderful library, it is really useful for projects that are translated into multiple languages. We are using plurals in our i18next project, something like this:

English:

    "Show older comments_one": "Show {{count}} older comment",
    "Show older comments_other": "Show {{count}} older comments",

Polish:

    "Show older comments_one": "Pokaż {{count}} starszy komentarz",
    "Show older comments_few": "Pokaż {{count}} starsze komentarze",
    "Show older comments_many": "Pokaż {{count}} starszych komentarzy",

As you can see, Polish has different plural forms. When I ran json-autotranslate with -d, it removed all these and added the key with _other as the last entry under the parent key. Is there a way to do it correctly for languages like this or this is something that's not implemented?

Stop automatically updating translations

I would like to know if there's a way to avoid translation updates as I am having the following issue. I have the texts back in a button which is translated by Google to dutch as rug, but I actually want terug. When I make the correction directly to the nl.json file and call json-autotranslate again I see rug is back.

Note I haven't changed the original text, I have only changed the dutch translation.

i18next style string interpolation causes an error

If I have a string value that uses i18next interpolation, such as:

"I am {{author.name}}"

then I get the following error:

An error has occured:
Expected identifier but "{" found.
SyntaxError: Expected identifier but "{" found.

---

BTW, thanks for building this, it's super useful!

zh-CN and zh-TW are not supported

Google Translate service doesn't support Chinese. It seems it's connected with the letter case.

Found 108 target language(s):
-> af, am, ar, az, be, bg, bn, bs, ca, ceb, co, cs, cy, da, de, el, eo, es, et, eu, fa, fi, fr, fy, ga, gd, gl, gu, ha, haw, he, hi, hmn, hr, ht, hu, hy, id, ig, is, it, ja, jw, ka, kk, km, kn, ko, ku, ky, la, lb, lo, lt, lv, mg, mi, mk, ml, mn, mr, ms, mt, my, ne, nl, no, ny, or, pa, pl, ps, pt, ro, ru, rw, sd, si, sk, sl, sm, sn, so, sq, sr, st, su, sv, sw, ta, te, tg, th, tk, tl, tr, tt, ug, uk, ur, uz, vi, xh, yi, yo, zh-CN, zh-TW, zu

✨ Initializing Google Translate...

...

  { code: 'zh-CN', name: 'Chinese (Simplified)' },
  { code: 'zh-TW', name: 'Chinese (Traditional)' },

...

🙈 Google Translate doesn't support zh-CN. Skipping this language.
🙈 Google Translate doesn't support zh-TW. Skipping this language.

During debugging I found that supportedLanguages from the service contains zh-cn, zh-tw in lowercase.

json-autotranslate/src/services/google-translate.ts

Line 18 in a1c1420

private supportedLanguages: string[] = [];

i18next - Interpolation throwing partial errors

@leolabs It seems there are some issues in supporting the interpolations. It's working well in some places but it's considering for translations at some other places. It's completely random. Find the example here.

I have a key: value like this -> "title": "{{underlying_asset}}-{{quoting_asset}} Index"

After converting it to Arabic, the result is: "title": "<span><span>0-1</span></span> مؤشر"

Ideally, it shouldn't ignore those keys underlying_asset, quoting_asset

issue with google translate

npx json-autotranslate -i locales -s google-translate -c traducciones-352415-79659df1c6f1.json
Found 2 target language(s):
-> ar, de

🏭 Loading source files...
├── helper-messages.json (key-based)
├── messages.json (key-based)
└── Done

✨ Initializing Google Translate...
[
  { code: 'af', name: 'Afrikaans' },
  { code: 'sq', name: 'Albanian' },
  { code: 'am', name: 'Amharic' },
  { code: 'ar', name: 'Arabic' },
  { code: 'hy', name: 'Armenian' },
  { code: 'az', name: 'Azerbaijani' },
  { code: 'eu', name: 'Basque' },
  { code: 'be', name: 'Belarusian' },
  { code: 'bn', name: 'Bengali' },
  { code: 'bs', name: 'Bosnian' },
  { code: 'bg', name: 'Bulgarian' },
  { code: 'ca', name: 'Catalan' },
  { code: 'ceb', name: 'Cebuano' },
  { code: 'ny', name: 'Chichewa' },
  { code: 'zh-CN', name: 'Chinese (Simplified)' },
  { code: 'zh-TW', name: 'Chinese (Traditional)' },
  { code: 'co', name: 'Corsican' },
  { code: 'hr', name: 'Croatian' },
  { code: 'cs', name: 'Czech' },
  { code: 'da', name: 'Danish' },
  { code: 'nl', name: 'Dutch' },
  { code: 'en', name: 'English' },
  { code: 'eo', name: 'Esperanto' },
  { code: 'et', name: 'Estonian' },
  { code: 'tl', name: 'Filipino' },
  { code: 'fi', name: 'Finnish' },
  { code: 'fr', name: 'French' },
  { code: 'fy', name: 'Frisian' },
  { code: 'gl', name: 'Galician' },
  { code: 'ka', name: 'Georgian' },
  { code: 'de', name: 'German' },
  { code: 'el', name: 'Greek' },
  { code: 'gu', name: 'Gujarati' },
  { code: 'ht', name: 'Haitian Creole' },
  { code: 'ha', name: 'Hausa' },
  { code: 'haw', name: 'Hawaiian' },
  { code: 'iw', name: 'Hebrew' },
  { code: 'hi', name: 'Hindi' },
  { code: 'hmn', name: 'Hmong' },
  { code: 'hu', name: 'Hungarian' },
  { code: 'is', name: 'Icelandic' },
  { code: 'ig', name: 'Igbo' },
  { code: 'id', name: 'Indonesian' },
  { code: 'ga', name: 'Irish' },
  { code: 'it', name: 'Italian' },
  { code: 'ja', name: 'Japanese' },
  { code: 'jw', name: 'Javanese' },
  { code: 'kn', name: 'Kannada' },
  { code: 'kk', name: 'Kazakh' },
  { code: 'km', name: 'Khmer' },
  { code: 'rw', name: 'Kinyarwanda' },
  { code: 'ko', name: 'Korean' },
  { code: 'ku', name: 'Kurdish (Kurmanji)' },
  { code: 'ky', name: 'Kyrgyz' },
  { code: 'lo', name: 'Lao' },
  { code: 'la', name: 'Latin' },
  { code: 'lv', name: 'Latvian' },
  { code: 'lt', name: 'Lithuanian' },
  { code: 'lb', name: 'Luxembourgish' },
  { code: 'mk', name: 'Macedonian' },
  { code: 'mg', name: 'Malagasy' },
  { code: 'ms', name: 'Malay' },
  { code: 'ml', name: 'Malayalam' },
  { code: 'mt', name: 'Maltese' },
  { code: 'mi', name: 'Maori' },
  { code: 'mr', name: 'Marathi' },
  { code: 'mn', name: 'Mongolian' },
  { code: 'my', name: 'Myanmar (Burmese)' },
  { code: 'ne', name: 'Nepali' },
  { code: 'no', name: 'Norwegian' },
  { code: 'or', name: 'Odia (Oriya)' },
  { code: 'ps', name: 'Pashto' },
  { code: 'fa', name: 'Persian' },
  { code: 'pl', name: 'Polish' },
  { code: 'pt', name: 'Portuguese' },
  { code: 'pa', name: 'Punjabi' },
  { code: 'ro', name: 'Romanian' },
  { code: 'ru', name: 'Russian' },
  { code: 'sm', name: 'Samoan' },
  { code: 'gd', name: 'Scots Gaelic' },
  { code: 'sr', name: 'Serbian' },
  { code: 'st', name: 'Sesotho' },
  { code: 'sn', name: 'Shona' },
  { code: 'sd', name: 'Sindhi' },
  { code: 'si', name: 'Sinhala' },
  { code: 'sk', name: 'Slovak' },
  { code: 'sl', name: 'Slovenian' },
  { code: 'so', name: 'Somali' },
  { code: 'es', name: 'Spanish' },
  { code: 'su', name: 'Sundanese' },
  { code: 'sw', name: 'Swahili' },
  { code: 'sv', name: 'Swedish' },
  { code: 'tg', name: 'Tajik' },
  { code: 'ta', name: 'Tamil' },
  { code: 'tt', name: 'Tatar' },
  { code: 'te', name: 'Telugu' },
  { code: 'th', name: 'Thai' },
  { code: 'tr', name: 'Turkish' },
  { code: 'tk', name: 'Turkmen' },
  { code: 'uk', name: 'Ukrainian' },
  ... 11 more items
]
└── Done

🔍 Looking for key-value inconsistencies in source files...
└── No inconsistencies found

🔍 Looking for invalid keys in source files...
└── No invalid keys found

💬 Translating strings from en to ar...
├── Translating helper-messages.json (+84)
├── Translating messages.json
An error has occurred:
input.charCodeAt is not a function
TypeError: input.charCodeAt is not a function
    at peg$parseargument (/root/json-autotranslate/node_modules/messageformat-parser/parser.js:497:15)
    at peg$parsetoken (/root/json-autotranslate/node_modules/messageformat-parser/parser.js:431:10)
    at peg$parsestart (/root/json-autotranslate/node_modules/messageformat-parser/parser.js:419:10)
    at Object.peg$parse [as parse] (/root/json-autotranslate/node_modules/messageformat-parser/parser.js:1969:16)
    at exports.matchIcu (/root/json-autotranslate/node_modules/json-autotranslate/lib/matchers/icu.js:6:40)
    at Object.exports.replaceInterpolations (/root/json-autotranslate/node_modules/json-autotranslate/lib/matchers/index.js:20:24)
    at GoogleTranslate.<anonymous> (/root/json-autotranslate/node_modules/json-autotranslate/lib/services/google-translate.js:113:57)
    at step (/root/json-autotranslate/node_modules/json-autotranslate/lib/services/google-translate.js:33:23)
    at Object.next (/root/json-autotranslate/node_modules/json-autotranslate/lib/services/google-translate.js:14:53)
    at /root/json-autotranslate/node_modules/json-autotranslate/lib/services/google-translate.js:8:71

source json files here:

https://github.com/KnowageLabs/Knowage-Server/tree/master/knowage-vue/src/i18n/en_US

pls help me to solve this

Google translate | losing interpolations

Hey @leolabs !

Nice library and thanks for this!
I noticed that upon translating with i18-next format & using google-translate service, the interpolations at the end were not added back after the translation and this is because Google service translates <0 /> <1 /> etc to: < 0 > so you are not able to correcty replace them back again. I assume this is probably due to invalid html nodes? Maybe this change to the service happened later on.

I tested it locally and seems to work if you just change the xmlStyleReplacer to <br className="${index}" /> for example.

json-autotranslate/src/matchers/index.ts

Line 5 in cb6c9f6

export const xmlStyleReplacer = (index: number) => `<${index} />`;

Cheers!

Some dependencies should be devDependencies

I have json-autotranslate in the devDependencies of my project and when I install my project I get a deprecation warning from tslint. That's why I've checked where it's from and it's a dependency of json-autotranslate. The dependencies referenced in package.json could be cleaned up so that the dependencies only include those that are really required on runtime (so that they aren't added to other projects on install).

From the looks of it, you might be able to move these dependencies to devDependencies:

• @types/node
• dotenv
• ts-node
• tslint
• typescript

can u create md-autotranslate / xml-autotranslate / html-autotranslate?

issue with type of json with azure

messages.zip
helper-messages.zip

Found 1 target language(s):
-> es

🏭 Loading source files...
├── helper-messages.json (key-based)
├── messages.json (key-based)
└── Done

✨ Initializing Azure...
└── Done

🔍 Looking for key-value inconsistencies in source files...
└── No inconsistencies found

🔍 Looking for invalid keys in source files...
└── No invalid keys found

💬 Translating strings from en to es...
├── Translating helper-messages.json (+84)
├── Translating messages.json
An error has occurred:
input.match is not a function
TypeError: input.match is not a function
	at exports.matchI18Next (/root/json-autotranslate/node_modules/json-autotranslate/lib/matchers/i18next.js:5:25)
	at Object.exports.replaceInterpolations (/root/json-autotranslate/node_modules/json-autotranslate/lib/matchers/index.js:20:24)
	at /root/json-autotranslate/node_modules/json-autotranslate/lib/services/azure-translator.js:107:49
	at Array.map (<anonymous>)
	at AzureTranslator.<anonymous> (/root/json-autotranslate/node_modules/json-autotranslate/lib/services/azure-translator.js:105:45)
	at step (/root/json-autotranslate/node_modules/json-autotranslate/lib/services/azure-translator.js:33:23)
	at Object.next (/root/json-autotranslate/node_modules/json-autotranslate/lib/services/azure-translator.js:14:53)
	at /root/json-autotranslate/node_modules/json-autotranslate/lib/services/azure-translator.js:8:71
	at new Promise (<anonymous>)
	at __awaiter (/root/json-autotranslate/node_modules/json-autotranslate/lib/services/azure-translator.js:4:12)

source language en doesnt exist

I created the directory localed added subfolders of languages I want to translate en to, inside en folder I have my source json file that needs translating, I'm using deepl which I did signed up with

Not all keys are translated

Thanks for an amazing tool!

The only problem I have is that not all keys are translated but available in en. In es for example, only 10/40 files are translated.

My command looks like this:
json-autotranslate -i src/locales -d -c service-account.json -m i18next -t natural -f

File structure:

Example JSON:

{
  "description": "",
  "title": ""
}

Better ICU handling

Hey me again!

I've found a few issues with the mechanism of using placeholders in html and icu syntax, specifically the plural stuff i added, but it could effect other things.

I have a solution, but i may not be able to contribute it any time soon, partly because it would require a large-ish change to how matching/replacing works for ICU strings that might not be wanted for this library. I thought I would at least open this issue to demonstrate how the AST parsing can be achieved.

The problem

(I'm aware this is a bad translation string but it shows the problem well)

Input: "defaultMessage": "Select all {totalRows, select, undefined {available} other {{totalRows}}} rows"

Google: "defaultMessage": "Sélectionnez toutes {totalRows, select, undefined { lignes } other {{totalRows}}} disponibles"
Aws: "defaultMessage": "Toutes les} other {{totalRows}}} lignes{totalRows, select, undefined { disponibles sont sélectionnées."

Why does this happen?

Because the translation services decides to move the html around, to make the sentence better in other langs
(i've put this back into english to demonstrate the issue)

All <span translate="no">1</span><span translate="no">0</span>available rows
selected.

So it appears you can't always guarantee the ordering of html placeholders once its been the through the translator.

I appreciate that this was a feature that I contributed and it doesn't quite hit all the cases, so sorry for that.

A solution

To solve this the library could parse the ast in a different way, and translate the specific values which should translate, i've done this here for google and amazon

https://github.com/leolabs/json-autotranslate/compare/master...Georgegriff:json-autotranslate:icu-parsing?expand=1#diff-b15d6586b5ec27fb11a3ca41ba1fa178626842745da09f66554ed7058eec840e

I had to introduce a new concept, Matchers api wont work, which is why i'm not jumping to a contribution here. I also haven't fixed up other translation services

This may or may not turn into a PR, your call, but I thought I would at least document a solution in case it's helpful in the future

Code detail

• I swapped out the ast parser for more friendly one to TS and one which has nice helpers.
• the "core" of whats going on here is this recursive function.

import {
  isLiteralElement,
  isTagElement,
  isPluralElement,
  isSelectElement,
} from '@formatjs/icu-messageformat-parser';

async function doTranslateAst(
  messageElements: MessageFormatElement[],
  translateText,
): Promise<MessageFormatElement[]> {
  const translatedElements = await Promise.all(
    messageElements.map(async (messageElement) => {
      if (isLiteralElement(messageElement)) {
        const originalValue = messageElement.value;
        // some services don't play well with whitespace, they remove it, manually handle that.
        const trimmedValue = originalValue.trim();
        const startingWhitespace =
          originalValue[0] !== trimmedValue[0] ? originalValue[0] : '';
        const endingWhitespace =
          originalValue[originalValue.length - 1] !==
          trimmedValue[trimmedValue.length - 1]
            ? originalValue[originalValue.length - 1]
            : '';
        messageElement.value = `${startingWhitespace}${await translateText(
          trimmedValue,
        )}${endingWhitespace}`;
      }

      if (isPluralElement(messageElement) || isSelectElement(messageElement)) {
        const optionEntries = await Promise.all(
          Object.keys(messageElement.options).map(async (id) => {
            const pluralChildElement = messageElement.options[id];
            const value = await doTranslateAst(
              pluralChildElement.value,
              translateText,
            );
            pluralChildElement.value = value;

            return [id, pluralChildElement];
          }),
        );
        messageElement.options = Object.fromEntries(optionEntries);
      }

      if (isTagElement(messageElement)) {
        const elementChildren = await doTranslateAst(
          messageElement.children,
          translateText,
        );
        messageElement.children = elementChildren;
      }

      return messageElement;
    }),
  );

  return translatedElements;
}

import { printAST } from '@formatjs/icu-messageformat-parser/printer';
import {
  parse,
} from '@formatjs/icu-messageformat-parser';

nst astMessageElements = parse(input);
  const translatedMessaged = await doTranslateAst(
    astMessageElements,
    translateText,
  );
  // convert back to string
  return printAST(translatedMessaged);

Here is a good way of playing with the AST's

how can create al locales i18n folders?

i have en folder, and i need all languages options in locales folder...

Investigate starting from a grammatically complex language to improve pronoun translation precision

If we have partial strings such as "Registered on {date}" it might be important to know what the argument is in order to select the correct pronoun in other languages. English, however, is a very simple language that doesn't cover the different cases in other languages, so on the part of the translator there is a lot of guess work involved.

Perhaps if we start from a more complex language by default (such as german, french or spanish), we might get better translations.

Add command to change a key

Every so often I need to change the value of a key. Usually it's a better message or I need to completely change a label. Right now, if I make a change I update the English translation file then delete that key from 14 other language files. It's a bit tedious.

Would would be nice is the ability to re-translate just one key.

I don't want to regenerate all the translation files because I've made some manual changes and don't want to lose them.

Thanks for making this great tool!

Support azure translator service?

The API reference is here:
https://docs.microsoft.com/en-us/azure/cognitive-services/translator/reference/v3-0-reference

If ya'll are open to it I could probably implement it in a PR

How to pass Google Translate API key in --config

I'm running the command: npx json-autotranslate --type keybased --matcher i18next --input locales --config locales/key.json

and I'm getting an error: "key must be a string, a buffer or an object". I have API key from Google Cloud and I don't know how to pass it to json-autotranslate. How my config JSON file should look like?

The source language en doesn't exist.

I'm having troubles running this.

When I run npx json-autotranslate

I get this error:

An error has occurred:
The source language en doesn't exist.
Error: The source language en doesn't exist.
    at /Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:129:31
    at step (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:68:23)
    at Object.next (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:49:53)
    at /Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:43:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:39:12)
    at translate (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:115:12)
    at Object.<anonymous> (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:345:1)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)

I also get the same error when using yarn yarn json-autotranslate

% yarn json-autotranslate
yarn run v1.22.18
warning package.json: No license field
$ /Users/xxx/Documents/dev/translate/node_modules/.bin/json-autotranslate

An error has occurred:
The source language en doesn't exist.
Error: The source language en doesn't exist.
    at /Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:129:31
    at step (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:68:23)
    at Object.next (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:49:53)
    at /Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:43:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:39:12)
    at translate (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:115:12)
    at Object.<anonymous> (/Users/xxx/Documents/dev/translate/node_modules/json-autotranslate/lib/index.js:345:1)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I'm trying to use it with deepl-free

can you please show me a command to make it run with deepl-free with translation files inside the locales directory en.default.json

Use custom terminology from Amazon Translate

How can I include in my translation Amazon Translate custom terminologies? Should I do something in my config file?

Please add license file

Hi - thank you for this library. We want to use your library but our legal team requires a license file before they allow us to use a library. We would be grateful If you could please add a license file. Thank you.

Translations written to en.json

So far json-autotranslate has always worked fine for me. I now suddenly get a weird error: When I run it, it translated as expected but all changes are written into en.json and not the actual locales.

I start it like this:
json-autotranslate -d --directory-structure ngx-translate -c google_translate.json -s google-translate --decode-escapes -t key-based -i locales/lumi -m i18next

The logs show that the direction is fine:
💬 Translating strings from en to de... (66 changes from cache)Lower case: en

But all the German strings are written to the en.json file.

I use version 1.9.0.

DeepL Target Language Support

Please support more target languages like
"HR",
"SR",
...

DeepL formality

Is it possible to add the formality parameter to the deepL API?

More information for this parameter find here

Sets whether the translated text should lean towards formal or informal language. This feature currently only works for target languages "DE" (German), "FR" (French), "IT" (Italian), "ES" (Spanish), "NL" (Dutch), "PL" (Polish), "PT-PT", "PT-BR" (Portuguese) and "RU" (Russian).Possible options are.

Questions

Sorry, I wasn't sure the best way to send you some questions about the library ...

1. I'm seeing something where a phrase like "Hop Farm" does not get translated but "Hop farm" does get translated properly. I don't really have any proper nouns in my translation file. Is there any way to force the translation to always translate uppercase words?

2. The word "hops" and "hop" is particularly troublesome. In my context I'm referring to the ingredient used in beer: lúpulo, hopfen, houblon. Aside from interpolation, is there a way to force the translator to prefer one translation of a word over another?

Google Translator oddities

When using the tool with Google Translator and i18next interpolation (using {{placeholder}}), there are cases in which google translate put strings into the 0 tag passed to Google Translate.

Example: Google translate returns
Det uppstod ett fel när filen <span>0 skulle</span> raderas från innehållsobjektet.
instead of

Det uppstod ett fel när filen <span>0</span> raderas från innehållsobjektet.

It looks like the translator tries to make sense of the sentence by putting in words inside the span. You can prevent this behavior by adding a translate="no" atribute to the span like this in src/matchers/index.ts:

export const xmlStyleReplacer = (index: number) => `<span translate="no">${index}</span>`;

I haven't submitted a PR on this, as I'm unsure if this will break other translators. Maybe you need a Google Translate specific replacer...

Another thing I've noticed is that Google Translate escapes all characters considered unsafe in HTML. So an apostrophe become '. It would be great if the json-autotranslate decodes these characters back to their actual value. It might be worth adding an option for this, as source strings might already contain HTML encoded characters.

Curly brackets inside JSON value throw an error

Hello,

when I run auto-translate on a JSON file that has this record in it:

"LABEL_SEQUENCE_STEP": "Step {value:step}",

it will throw this error:

An error has occured:
Expected "," or "}" but ":" found.
SyntaxError: Expected "," or "}" but ":" found.
    at peg$buildStructuredError (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/messageformat-parser/parser.js:407:12)
    at Object.peg$parse [as parse] (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/messageformat-parser/parser.js:1978:
11)
    at exports.matchIcu (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/json-autotranslate/lib/matchers/icu.js:5:40)
    at Object.exports.replaceInterpolations (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/json-autotranslate/lib/matc
hers/index.js:17:24)

    at GoogleTranslate.<anonymous> (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/json-autotranslate/lib/services/goog
le-translate.js:78:57)
    at step (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/json-autotranslate/lib/services/google-translate.js:32:23)
    at Object.next (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/json-autotranslate/lib/services/google-translate.js:
13:53)
    at /Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/json-autotranslate/lib/services/google-translate.js:7:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/petrchutny/Tinkamo/Tinkamo App – All/node_modules/json-autotranslate/lib/services/google-translate.js:3:
12)

When I remove the curly brackets from inside the double quotes, everything is OK.
Trying to escape curly brackets with \ doesn't work (like this: "LABEL_SEQUENCE_STEP": "Step \{value:step\}"
I previously thought it's caused only by curly brackets, but it seems it's the combination of them and the colon ':' in between them, because there is also this record, which doesn't throw any error: "MSG_CONNECTED": "{name}<br>is connected",

How to handle input keys with "." in names

I have some strings to translate which have the following structure:

{
  "path.to.Component.yourProgress": {
    "defaultMessage": "Your progress"
  },
}

I'm finding that by default the library is unflattening my object, i suspect because internally flattenjs uses the same character for splitting objects

example output ends up breaking my json input:

{
"path": {
  "to": {
    "Component": {
      "yourProgress": {
             "defaultMessage": "Your progress"
         }
    }
  }
}
}

Ideally the library wouldn't transform the JSON is there some way to fix this, or might have i have to flatten the files myself?

Variable placeholders getting corrupted while using Google translate API

@leolabs Thanks for building this. This is a fantastic tool with excellent documentation and made my life a lot easier.

But I did face an issue while using this tool.

So I was trying to translate the strings without arabic translations in the following for fixing zulip/zulip#18933.

https://github.com/zulip/zulip/blob/05352f417a8f163ee8bda1fc1593875c432c5297/locale/ar/translations.json.

npx json-autotranslate -t natural --matcher icu -d -i locale -c google-key.json

The command completed sucesfully. But 17 strings ended up having the variable names corrupted. There are 55 strings with a variables in the file. So around 30% of the strings ended up being corrupted. I guess the good news is 70% got translated correctly :)

Here are the list of strings that were corrupted. The first line contains the string in English and the second line contains the translation in Arabic.

```bash
Are you sure you want to mention all <strong>__count__</strong> people in this stream? <br> This will send email and mobile push notifications to most of those <strong>__count__</strong> users. <br> If you don't want to do that, please edit your message to remove the <strong>@__mention__</strong> mention.
هل تريد بالتأكيد ذكر جميع <strong>__count__ من</strong> الأشخاص في مجموعة البث هذه؟<br>سيؤدي هذا إلى إرسال إشعارات عبر البريد الإلكتروني والجوال إلى معظم هؤلاء <strong>__العدد__ من</strong> المستخدمين.<br>إذا كنت لا تريد القيام بذلك ، يرجى تعديل رسالتك <strong>لإزالة__mention__</strong> الإشارة.

Message __- recipient_names__
الرسالة __- أسماء المستلمين__

Narrow to stream &quot;__display_recipient__&quot;
ضيق للدفق & quot؛ __ display_recipient __ & quot؛

Narrow to stream &quot;__display_recipient__&quot;, topic &quot;__topic__&quot;
ضيق للدفق & quot؛ __ display_recipient __ & quot ؛، موضوع & quot؛ __ موضوع __ & quot؛

Narrow to your private messages with __display_reply_to__
تضييق نطاق رسائلك الخاصة باستخدام __عرض_الرد على_تو__

Organization using __percent_used__% of __upload_quota__.
منظمة تستخدم __percent_used __٪ من __upload_quota__.

Owner: __name__
اسم المالك__

This organization is configured to restrict editing of message content to __minutes_to_edit__ minutes after it is sent.
تم تكوين هذه المؤسسة لتقييد تحرير محتوى الرسالة بـ __دقائق_لتحرير__ دقائق بعد إرسالها.

You (click to remove) reacted with __- emoji_name__
لقد (انقر للإزالة) تفاعلت بـ __- اسم_إيموجي__

You and __display_reply_to__
أنت و __عرض_الرد على_إلى__

You and __recipients__
أنت و __المستلمون_

You are not subscribed to stream __stream__
أنت غير مشترك في البث __ Stream__

You subscribed to stream __stream__
لقد اشتركت في البث __ Stream__

You unsubscribed from stream __stream__
لقد ألغيت اشتراكك في تيار __ Stream__

__- username__ reacted with __- emoji_name__
__- اسم المستخدم__ تفاعل بـ __- اسم_إيموجي__

__starred_status__ this message
__ starred_status__ هذه الرسالة

private messages with __- recipient__
رسائل خاصة مع __- المستلم__

Any idea why this happened?

i want use azure service

root@ubuntu21:~/json-autotranslate# npx json-autotranslate

An error has occurred:
The source language en doesn't exist.
Error: The source language en doesn't exist.
	at /root/json-autotranslate/node_modules/json-autotranslate/lib/index.js:129:31
	at step (/root/json-autotranslate/node_modules/json-autotranslate/lib/index.js:68:23)
	at Object.next (/root/json-autotranslate/node_modules/json-autotranslate/lib/index.js:49:53)
	at /root/json-autotranslate/node_modules/json-autotranslate/lib/index.js:43:71
	at new Promise (<anonymous>)
	at __awaiter (/root/json-autotranslate/node_modules/json-autotranslate/lib/index.js:39:12)
	at translate (/root/json-autotranslate/node_modules/json-autotranslate/lib/index.js:115:12)
	at Object.<anonymous> (/root/json-autotranslate/node_modules/json-autotranslate/lib/index.js:345:1)
	at Module._compile (node:internal/modules/cjs/loader:1105:14)
	at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)

how solve this?

Rate Limit Exceeded

Problem:
Google-translate service reaches the rate limit, and translation fails with the following error:

An error has occurred:
User Rate Limit Exceeded
Error: User Rate Limit Exceeded

How to add a delay between translation calls or retry the translation call to google service after some period in case the rate limit is reached?

Error empty other language directory

When I tried to use it with:

json-autotranslate -i locales --config google_key.json -s google-translate

and I let the fr and de language directories empty I had the following error:

An error has occurred:
Cannot read property 'name' of undefined
TypeError: Cannot read property 'name' of undefined
at /Users/catalin/node_modules/json-autotranslate/lib/index.js:390:141
at step (/Users/catalin/node_modules/json-autotranslate/lib/index.js:68:23)
at Object.next (/Users/catalin/node_modules/json-autotranslate/lib/index.js:49:53)
at fulfilled (/Users/catalin/node_modules/json-autotranslate/lib/index.js:40:58)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:95:5)

Cannot read properties of undefined (reading 'name')

As per the package instructions, we just need to create the language folder and this package will create translation files in the folder.

I used this command:

npx json-autotranslate -i public/locales -m i18next -c gcloud_sa.json -d

error log:

💬 Translating strings from en to de...
├── Translating common.json (0 changes from cache)
An error has occurred:
Cannot read properties of undefined (reading 'name')
TypeError: Cannot read properties of undefined (reading 'name')
    at /Users/captain/Desktop/saiberio/repos/saiberio/node_modules/json-autotranslate/lib/index.js:390:141
    at step (/Users/captain/Desktop/saiberio/repos/saiberio/node_modules/json-autotranslate/lib/index.js:68:23)
    at Object.next (/Users/captain/Desktop/saiberio/repos/saiberio/node_modules/json-autotranslate/lib/index.js:49:53)
    at fulfilled (/Users/captain/Desktop/saiberio/repos/saiberio/node_modules/json-autotranslate/lib/index.js:40:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

If I create the placeholder JSON files inside the language folder, with an empty JSON ({}), it creates the translations successfully.

Any ideas why I see this behavior?

Release new version

Hi, great tool! I was a bit confused at first that the NPM version isn't the same as the master on GitHub. It would be great if you could release a new version!

Windows file formats UTF-8-BOM (CR LF)

Ran into an issue running json-autotranslate for the second time on Windows platform. It appears that when reading the files from the cache folder an exception will be thrown parsing JSON if the files are not UTF-8 with Unix (LF) EOL. In my case one of the files was UTF-8-BOM with Windows (CR LF) EOL, I assume that the file format is carried over from the original file. (I happen to auto generate one of the json files from a RESX file hence one file was in a windows file format).

To fix the problem I manually converted the JSON files to UTF-8 with Unix (LF) EOL, including those in the cache folder.

Adding a note about Windows compatibility would be sufficient, or even better adding support for windows file formats.

Thanks for the great work!

Version 1.5.2

Unexpected token  in JSON at position 0
SyntaxError: Unexpected token  in JSON at position 0
    at JSON.parse (<anonymous>)
    at D:\repos\project\node_modules\json-autotranslate\lib\index.js:248:91

Unable to translate to traditional chinese with Google translate.

According to this page traditional Chinese is supported in the Google translate API.

So in theory with a folder named zh-TW it should give a translation.

When using a folder named zh-tw, it gives a error that indicates an invalid language on google API's end.

Azure requires region

Tried this great library out with Azure and they need a region. I monkied with the js on my node_module and it works by adding the extra header item.

json-autotranslate/src/services/azure-translator.ts

Line 81 in 0fd6d1c

'Ocp-Apim-Subscription-Key': this.apiKey,

https://docs.microsoft.com/en-us/azure/cognitive-services/translator/reference/v3-0-reference#authenticating-with-a-regional-resource

attranslate - A rewrite of json-autotranslate

First of all, thank you for writing json-autotranslate 👍
json-autotranslate worked great for all my i18next-projects 😎

Nevertheless, I came to the conclusion that a complete rewrite of json-autotranslate is appropriate.
The resulting rewrite is attranslate: https://github.com/fkirc/attranslate
attranslate addresses the following features that I was missing in JSON-autotranslate:

• Do not enforce any specific folder-structure. Not every project uses a folder-structure that is prescribed by i18next.
• Support additional file-formats other than JSON (XML, Plist, YAML, PO/POT).
• Generate caches only for source-files, but not for target-files. This reduces noise in version-control.
• Place a higher emphasis on file-stability, that is, only change a minimum number of lines.
• Ensure quality with a comprehensive test-suite.
• Optimize performance. For example, remove available-language-checks that were painful to keep up-to-date.
• Make it simpler to add new services or new file-formats. For example, move matching-logic from individual services into the core-unit.

However, despite of all those differences, there is still a lot of duplication between attranslate and json-autotranslate.
This raises the question whether it is really necessary to maintain both codebases in the long-term.
Ideally, all maintenance effort would go into one codebase.
Please let me know whether you would be open to a collaboration instead of maintaining competing sub-projects.

Support for DeepL Free API

Hi,

it looks like only the Pro DeepL version is supported, since the API request is made to the PRO endpoint. Could you add a parameter to support the free version as well (without Pro you have 500'000 characters/month).

Example to get all languages: https://api-free.deepl.com/v2/languages instead of https://api.deepl.com/v2/languages

json-autotranslate/src/services/deepl.ts

Line 11 in d8d7ab7

const API_ENDPOINT = 'https://api.deepl.com/v2';

Feature wanted: Add new languages not supported by Google Translate

Hi,

I want to tell you that json-autotranslate works like charm! I really enjoy it. Recently I have encountered a problem.

I already have my JSON translated into Quechua (Peru Spanish). But it keeps saying "Translation doesn't support pe. Skipping this language." Neither auto-translation nor manual works. I hope it can support languages out of Google Translate's scope. Once I have the folder and JSON ready, it can always use the local resource to translate.

Anyway, I want a walkaround.

Feature suggestion: Translating updated strings again

Hello,

thanks for making this amazing script. It helps us greatly to save so much time translating at Tinkamo. I know this might be a big idea requiring more coding, but what do you think about the functionality of being able to detect whether a source string has changed and translating it again?
What do you think about it? :)

Some problem

After install i have some problem:

Can you tell me what the problem is?

ICU Plurals Support fix

Hey, first off this project is straight up 🔥 I'm really hoping to be able leverage it to help kick off internationalisation with the product that I work on.

I notice this project does not currently support plurals for ICU e.g.

{value, plural, =0 {No unread messages} one {{value} unread message} other {{value} unread messages} }

I've given a shot at adding support in the fork below. Is this something you'd be interested in me opening a PR for?
Some of the code/types may need tidying up, happy to receive pointers once/if get to PR stage

https://github.com/leolabs/json-autotranslate/compare/master...Georgegriff:json-autotranslate:icu-plurals-support?expand=1

This change will mean that "No unread messages" "unread message" and "unread messages" would be sent to be translated instead of just all placeholdered