Giter Site home page Giter Site logo

vscode-l10n's Introduction

Localization tooling for Visual Studio Code

This repository contains tooling for localizing Visual Studio Code extensions. Localization for VS Code extension's source code has 4 important parts:

  • vscode.l10n.t - The API for translating strings in your extension's code
  • @vscode/l10n-dev - The tooling used for extracting l10n strings from vscode extensions and working with XLF files
  • @vscode/l10n - The library used for loading the translations into subprocesses of your extension
  • package.nls.json - The file used for translating static contributions in your extension's package.json

Additionally, for a sample of how to use these tools, see the l10n-sample in the vscode-extension-samples repo.

vscode.l10n.t

This API, introduced in VS Code 1.73, is used for translating strings in your extension's code. It is a part of the main VS Code extension API and is further documented here.

Note

Make sure you your VS Code engine and @types/vscode version in your extension manifest is at least ^1.73.0.

@vscode/l10n-dev

Tooling used for extracting l10n strings from vscode extensions and working with XLF files. See it's dedicated README for usage instructions.

@vscode/l10n

Library used for loading the translations into subprocesses of your extension. See it's dedicated README for usage instructions.

Note

You should NOT use this library in your extension's main process. The translations are loaded into the main process by VS Code itself.

package.nls.json

This file, along with package.nls.{locale}.json files, are used for translating static contributions in your extension's package.json. Here's an example:

Your ./package.json:

{
  "name": "my-extension",
  "version": "0.0.1",
  "main": "./out/extension.js",
  "l10n": "./l10n",
  //...
  "contributes": {
    "commands": [
      {
        "command": "my-extension.helloWorld",
        // The key is surrounded by % characters
        "title": "%my-extension.helloWorld.title%"
      }
    ]
  }
}

Your ./package.nls.json:

{
  // That same key from the package.json
  "my-extension.helloWorld.title": "Hello World"
}

Your ./package.nls.de.json:

{
  // That same key from the package.json
  "my-extension.helloWorld.title": "Hallo Welt"
}

VS Code will automatically load the correct package.nls.{locale}.json (or package.nls.json for English) file based on the locale of the user. If no translation is available for a given key, VS Code will fall back to the English translation.

Note

@vscode/l10n-dev has some tooling around these files (converting them to XLIFF files, generating Pseudo-Localization files, etc.) that you can use.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Build steps

First, install all of the dependencies using npm install.

If you plan on working with l10n-dev it has one additional step. This package requires building the tree-sitter WASM files for the two grammars that we consume. To do this, you can run the following commands:

cd l10n-dev
npm run build-wasm

Note

On macOS or Windows, you will need to have Docker running in order to build the WASM files. The CLI runs a linux container to build the WASM files.

If you've done this correctly, you should see two .wasm files in the l10n-dev folder.

At this point you can run the build task in the repo to build in the background and run the tests with npm test.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

vscode-l10n's People

Contributors

connor4312 avatar deepak1556 avatar dependabot[bot] avatar joyceerhl avatar jruales avatar lramos15 avatar lszomoru avatar microsoft-github-operations[bot] avatar microsoftopensource avatar olguzzar avatar rzhao271 avatar sbatten avatar tylerleonhardt avatar yiliang114 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vscode-l10n's Issues

@vscode/l10n-dev export misses template strings

Running npx @vscode/l10n-dev export --outDir ./test src in js-debug on the l10n branch, the tool only finds 70 of the 210 usages of l10n.t.

Some examples of missed usages, I don't see any particular pattern:

https://github.com/microsoft/vscode-js-debug/blob/c10e6f58749c23ca4be0531db606e7e046b61f5f/src/binder.ts#L110

            params.breakpoints?.map(() => ({
              id: ++lastBreakpointId,
              verified: false,
              message: l10n.t(`Unbound breakpoint`),
            })) ?? [],

https://github.com/microsoft/vscode-js-debug/blob/c10e6f58749c23ca4be0531db606e7e046b61f5f/src/vsDebugServer.ts#L63

    const rootSession = new VSDebugSession(
      'root',
      l10n.t('JavaScript debug adapter'),
      deferredConnection.promise,
      { type: DebugType.Chrome, name: 'root', request: 'launch' },
    );

https://github.com/microsoft/vscode-js-debug/blob/c10e6f58749c23ca4be0531db606e7e046b61f5f/src/adapter/customBreakpoints.ts#L119

    i('requestAnimationFrame', l10n.t('Request Animation Frame')),

Support for ICU formatting?

Is there any plan to support ICU formatting in the future? This would allow for proper translation into locales that don't use the same singular/plural convention as English. For example, Czech treats the plural differently for numbers 2-4 vs. for numbers 5 or greater. Russian uses the singular form for all numbers that end in 1, even if they are big numbers. French and Brazilian Portuguese treat 0 as singular and not as plural as English does. Here is a list of plural rules compiled by Mozilla for reference.

Provide XLF Description and Resources

The documentation for @vscode/l10n-dev mentions XLF files. I’ve worked with a number of localization file types (.po, .csv, .xlsx, and .json from VS Code’s previous localization solution) but XLF is a new one for me. I would guess that I’m not alone here. A google search for that file type returns a a suggestion for the Wikipedia article for XLIFF which appears to be what you’re referring to with XLF.

It would be nice if you could provide some more information than "XLF files" for users whom may not have encountered the file type before. This includes links to helpful resources when dealing with XLF files. It appears that a cross-platform tool for handling these is available here (though there are likely others - I just stopped after identifying one).

Improve CLI (@vscode/l10n-dev ) - Validate if pre-requisites are correctly defined in `package.json`

Hi,

After updating half dozen of my extensions to support l10n I can say that both API and tooling, are really easy to use. Congrats!

And, as a feedback to the CLI (@vscode/l10n-dev), specially for newcomers, I would suggest a few improvements related to error messages and default values.

  • Validate if the pre-requisites are correctly set in package.json, like:
    • The engine and @types/vscode entries are defined to 1.73.0 minimum
    • The l10n entry is also defined
  • [ ] Use the value defined in l10n entry, if --outDir param is missing (moved to #66)

Thank you

Scope should be larger

While it's great that you are looking at providing a formalised solution for localisation of extension strings this is not all that hard once you get past the total lack of documentation. What's hard is coordinating incremental updates when you have a contributor per translation. For example, I do English and French, and I have colleagues who speak English and Spanish, or English and Italian, or English and Icelandic. Every time I release a minor version, I have to coordinate changes to

  • readme
  • changelog
  • UI (settings descriptions etc)
  • UI dialog messages
  • manual

I can almost hear you drawing breath to say that github does diffs.

It's not good enough.

  • It's spread all over the place.
  • There's no coherent way to see what languages have been updated and which haven't.

What's required is tooling to manage localisation.

  • Tooling to identify translation candidates (literal strings)
    • ignore replacement tokens because these are already being localised
    • ignore literals found in an ignore list
  • Tooling to emit difference files requiring translation as a single file for the translator
  • Tooling to update the NLS source files from the translated difference file.

If you ask me this begs for implementation as a VS Code extension.

What are your thoughts?

Support newlines in comments?

Something I realized in #93 is that we don't support newlines in comments... and instead you have to arrayify your comments.

This is tricky to support because the localization team represents "multiple comments" as new lines in their XLF format... and the XML parser we use turns newlines (as actual newlines and 
) into... well... newlines. In other words they're treated the same way. So that makes it tough to support newlines in comments and multiple comments via an array.

I don't think this is really that important... but wanted to capture the issue somewhere. We could consider encoding newlines somehow, but I worry that that will overcomplicate things.

New line characters not normalized

image

Take a look at this big "To have a great conversation" string... that uses newlines in it and in the bundle, they are \r\n but on non-Windows we'll be asking for \n... so we need to normalize these newlines.

l10n-dev | Produce a formatted, sorted and auto commented bundle.l10n.json file

Hi,

The l10n-dev export command produces a bundle.l10n.json file which is not actually used by l10n but which serves as a reference for constructing bundle.l10n.<locale>.json files.

Apparently, the bundle.l10n.json file is built based on the discovery of the texts used by the vscode.l10n.t function and the text strings are saved, without formatting, in the order they appear in TypeScript or JavaScript files.
Any modification (moving a portion of code) in the TypeScript or JavaScript files has an impact on the construction of the bundle.l10n.json file.

The work of creating and especially maintaining bundle.l10n.<locale>.json files would be easier if the bundle.l10n.json file was formatted and sorted on key texts, possibly with options to enable formatting and / or sorting.

Formatted: this can be an effect of working on Windows platform... CRLF expected instead of LF, add replace("\r(\n)?","\r\n") on result of JSON.stringify? use "\t" insteadd of 2 as space value?
See:

Sorted: add a sort on key of mergedJson in getL10nJson?

And perhaps add source path and source line number, (position.row), column number, (position.column), as a automatic comment.

Thanks.

l10n | New function l10n.k to manage translation by key instead of English natural text translation

Hi,

Extract from:

The l10n solution based on the use of the natural language (English) as a search key for localization poses the problem that each change in text used as a search key for localization immediately has an impact on all bundle language files: a typography correction in the text used as a search key requires to modify all the bundle files of the proposed localization languages.

The l10n solution should at least offer a way to define key values, truly neutral, as for package.nls.json files, not linked to a particular language, with bundle files containing the texts for the different languages, including a default bundle that can be English for its universal aspect.

To limit the impacts on existing extensions, this could be implemented by offering a new function other than l10n.t('text') to manage searches by key and not searches by text, for example a l10n.k('key') function:

  • the VS Code core must load at least one bundle file, either the one corresponding to current active language, bundle.l10n.<locale>.json, or the default bundle file (without explicit language), bundle.l10n.json,
  • searching for the text associated with the key using l10n.k('key') must be done regardless of the active language, including English.

As summary, the difference between the two functions would be:

  • l10n.t('text'): ignores the bundle.l10n.json file, and imposes the use of the English language in texts used in the extension, also imposes a definitive version of the texts used in the extension
  • l10n.k('key'): takes into account the bundle.l10n.json file, and does not impose the use of the English language in the texts used in the extension, which also allows deferring and delegating translation to different translators

See also:

Thanks.

Issue building vscode against >=electron-21

I've posted this into vscode repo, but maybe you guys have any idea what may be causing this?

The error tree_sitter_tsx_external_scanner_create comes from

static #tsGrammar: Promise<Parser.Language> = (async () => {
await initParser;
return await Parser.Language.load(
path.resolve(__dirname, 'tree-sitter-typescript.wasm')
);
})();
static #tsxGrammar: Promise<Parser.Language> = (async () => {
await initParser;
return await Parser.Language.load(
path.resolve(__dirname, 'tree-sitter-tsx.wasm')
);
})();

Both wasm files come pre-built from @vscode/l10n-dev.
Changing the dependency to version 0.0.24 changes the error (although that may be a race condition between the two):

Unhandled Rejection at: Promise Promise {
  <rejected> Error: bad export type for `tree_sitter_typescript_external_scanner_create`: undefined
      at reportUndefinedSymbols (/var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/web-tree-sitter/tree-sitter.js:1:19748)
      at postInstantiation (/var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/web-tree-sitter/tree-sitter.js:1:17027)
      at /var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/web-tree-sitter/tree-sitter.js:1:17752
      at async /var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/@vscode/l10n-dev/dist/main.js:270:10
      at async /var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/@vscode/l10n-dev/dist/main.js:259:22
} reason: Error: bad export type for `tree_sitter_typescript_external_scanner_create`: undefined
    at reportUndefinedSymbols (/var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/web-tree-sitter/tree-sitter.js:1:19748)
    at postInstantiation (/var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/web-tree-sitter/tree-sitter.js:1:17027)
    at /var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/web-tree-sitter/tree-sitter.js:1:17752
    at async /var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/@vscode/l10n-dev/dist/main.js:270:10
    at async /var/tmp/portage/app-editors/vscode-9999/work/vscode-9999/node_modules/@vscode/l10n-dev/dist/main.js:259:22

Lines 259 and 270 from the error look as follows:

__privateAdd(ScriptAnalyzer, _tsxParser, (async () => {
  await initParser;
  const parser = new import_web_tree_sitter.default();
  parser.setLanguage(await __privateGet(_ScriptAnalyzer, _tsxGrammar));
  return parser;
})());

__privateAdd(ScriptAnalyzer, _tsxGrammar, (async () => {
  await initParser;
  return await import_web_tree_sitter.default.Language.load(
    path.resolve(__dirname, "tree-sitter-tsx.wasm")
  );
})());

In this case tree_sitter_typescript_external_scanner_create is absent from tree-sitter-tsx.wasm, but present in tree-sitter-typescript.wasm.

I have no idea, why is it trying to load a function that is absent from wasm file. Broken wasm files in @vscode/l10n-dev?

It is interesting that electron-19 and electron-20 show no such behaviour. See fresh logs here.

Multiline Comments Issue

Multiline Comments:

Running npx @vscode/l10n-dev export on

const comment = vscode.l10n.t(
{
  message: "string with multiple comments",
  comment: [
	"comment 1",
	"comment 2",
	"comment 3"
  ]
});
vscode.window.showInformationMessage(comment);

produces a bundle.l10n.json with

{
  "string with multiple comments/comment 1comment 2comment 3": {
    "message": "string with multiple comments",
    "comment": [
      "comment 1",
      "comment 2",
      "comment 3"
    ]
  }
}

note that the lookup key contains all three lines of comments, but has no marker separating the comment lines.
Copying this value into a translated bundle.l10n.*.json with a fake translation like

{
    "string with multiple comments/comment 1comment 2comment 3": "FAKE TRANSLATION - string with multiple comments"
}

causes a lookup miss at runtime, loading the untranslated English default text.

Way to easily identify duplicate strings

Originally from microsoft/vscode-discussions#159 (comment)

One discussion that @philliphoff and I had yesterday relates to strings that are identical in, say, English, but not in other languages. These could be differentiated by using the comment feature for a message, because the comment is used as part of the key in the bundle.

However, suppose you are converting an existing extension that has been using vscode-nls. It would not be scalable to identify every reused string on a manual basis. Can the @vscode/l10n-dev package have a feature to help find places where the same string is reused, so the authors can decide if a comment needs to be added to ensure uniqueness in the bundles?

Weird treesitter crash with v0.0.29, not on v0.0.16

Cannot generate l10n files on my computer with this crash:

(base) PS D:\prj\vscext\pepb-helper> npx @vscode/l10n-dev export -o ./l10n ./src
Need to install the following packages:
  @vscode/[email protected]
Ok to proceed? (y)
Searching for TypeScript/JavaScript files...
Found 5 TypeScript files. Extracting strings...
vscode-l10n-dev export [args] <path..>

Export strings from source files. Supports glob patterns.

位置:
  path  TypeScript files to extract strings from. Supports folders and glob
        patterns.                                     [数组] [必需] [默认值: []]

选项:
      --version  显示版本号                                               [布尔]
  -v, --verbose  Enable verbose logging                                   [布尔]
  -d, --debug    Enable debug logging                                     [布尔]
      --help     显示帮助信息                                             [布尔]
  -o, --outDir   Output directory                                       [字符串]

Error: bad export type for `tree_sitter_tsx_external_scanner_create`: undefined
    at reportUndefinedSymbols (C:\Users\RigoLigo\AppData\Local\npm-cache\_npx\68546583dcb2c24b\node_modules\web-tree-sitter\tree-sitter.js:1:19748)
    at postInstantiation (C:\Users\RigoLigo\AppData\Local\npm-cache\_npx\68546583dcb2c24b\node_modules\web-tree-sitter\tree-sitter.js:1:17027)
    at C:\Users\RigoLigo\AppData\Local\npm-cache\_npx\68546583dcb2c24b\node_modules\web-tree-sitter\tree-sitter.js:1:17752
    at async C:\Users\RigoLigo\AppData\Local\npm-cache\_npx\68546583dcb2c24b\node_modules\@vscode\l10n-dev\dist\cli.js:334:10

Rolling back to v0.0.16 will work:

(base) PS D:\prj\vscext\pepb-helper> npm uninstall @vscode/l10n-dev

added 33 packages, changed 1 package, and audited 231 packages in 3s

1 package is looking for funding
  run `npm fund` for details

found 0 vulnerabilities
(base) PS D:\prj\vscext\pepb-helper> cnpm install @vscode/[email protected]
√ Linked 11 latest versions fallback to D:\prj\vscext\pepb-helper\node_modules\.store\node_modules
√ Installed 1 packages on D:\prj\vscext\pepb-helper
√ All packages installed (5 packages installed from npm registry, used 737ms(network 734ms), speed 29.31KB/s, json 0(0B), tarball 21.51KB, manifests cache hit 11, etag hit 10 / miss 0)

dependencies:
+ @vscode/l10n-dev ^0.0.16

(base) PS D:\prj\vscext\pepb-helper> npx @vscode/l10n-dev export -o ./l10n ./src
Searching for TypeScript files...
Found 5 TypeScript files. Extracting strings...
Writing exported strings to: D:\prj\vscext\pepb-helper\l10n\bundle.l10n.json

Issues with `@vscode/l10n-dev` tool (documentation?)

Hi,

I'm having a few issues with the @vscode/l10n-dev tool, to extract the strings from my source. I guess the instructions in README.md are outdated. If that's the case, I would be glad to open a PR to fix it

  1. The npx @vscode/l10n-dev ./src --out ./l10n command didn't work for me. Neither installing the tool with npm install -g. After digging a bit, I find out I had to add an export command

  2. It seems there is a typo in the --out parameter. Looking at the source, I find out it should be --o or --outDir instead

In the end, the correct command to me was

    npx @vscode/l10n-dev export ./src --outDir ./l10n

Thank you

esm variant (`module`) does not not implement `fsPath`

The package.json defines a module field to provide a esm variant.

Packagers like esbuild and webpack use that to tree shake.

The problem is that module is currently set to ./dist/browser.esm.js and is not defined for NodeJS: it does not implement the fsPath config and uses fetch (requires > Node 17.5). It should instead match the NodeJS variant (main.esm.js`)

See https://esbuild.github.io/api/#main-fields:

For package authors: Some packages incorrectly use the module field for browser-specific code, leaving node-specific code for the main field. This is probably because node ignores the module field and people typically only use bundlers for browser-specific code. However, bundling node-specific code is valuable too (e.g. it decreases download and boot time) and packages that put browser-specific code in module prevent bundlers from being able to do tree shaking effectively. If you are trying to publish browser-specific code in a package, use the browser field instead.

import-xlf prettified output

npx @vscode/l10n-dev import-xlf ... produces the translated bundle.l10n.*.json files as single lines, which makes diffing the output annoying without manually invoking the VS Code Format Document command for each translated bundle after every import.

It would be much more convenient if the tool would prettify/format the outputted bundles.

Do not minify the main entry files

The main file (browser, main, module`) are all minify which makes it very hard to debug problems in the code, even when the application uses the code is running from sources.

Libraries should not minify (or even package) themselves but leave this to the application that uses them.

Feature request: Setting custom localization string keys

VS Code localization currently uses a hash of [English string] + [comment] as a key for localization strings in .xlf files. It would be beneficial to allow having a custom, stable key for each localization string. For example, if we make a small change to an English string that has already been translated, it would be good to keep the old translation for as long as it takes to re-translate the new version, instead of showing untranslated English strings during that period of time. Similarly, it could be useful to have the old translation at hand to use as a reference for the new translation.

In terms of implementation, the custom key could be provided as an optional parameter to the vscode.l10n.t() function and parsed by @vscode/l10n-dev together with the English strings and comments.

No strings found. skipping writing to a bundle.l10n.json

Whenever I execute npx @vscode/l10n-dev export --outDir ./l10n ./src I receive searching for typescript/javascript files... found 6 typescript files. extracting strings... no strings found. skipping writing to a bundle.l10n.json.
I'm not sure if this issue is from the source file, or from my execution.
What I did exactly is I entered the command inside the extension folder.

Further multiline issues

While there were a couple PRs fixing #63, looks like there are still issues with multiline strings and multiline comments.

Multiline Strings:

Running npx @vscode/l10n-dev export on

vscode.l10n.t("multiline\ntest")

produces a bundle.l10n.json with

{
  "multiline\\ntest": "multiline\\ntest"
}

Generating an XLF with npx @vscode/l10n-dev generate-xlf removes that extra escape slash to

<trans-unit id="++CODE++60babcaf5bf8760815f018b47248ffc2935c9f5c6a2a0cc34cea133139bce3c7">
  <source xml:lang="en">multiline\ntest</source>
</trans-unit>

but importing a translated version with npx @vscode/l10n-dev import-xlf produces a bundle.l10n.*.json with

{
    "multiline\\ntest": "FAKE TRANSLATION multiline\\ntest",
}

At runtime, the bundle matching fails, as the search string is just using \n, but the bundles (both created via export and import-xlf) have \\n causing a lookup miss and displaying the original English text.

Don't bundle `@vscode/l10n` to save space

It's a library so there's no need to. Plus, on the node side it pulls in request-light which should be declared as a normal dependency rather than a devDependency to make the resulting package smaller.

xml2js NPM Vulnerability reported

a vscode extension of ours references this @vscode/l10n-dev and I saw a vulnerability reported today:

xml2js  <=0.4.23
Severity: high
xml2js is vulnerable to prototype pollution  - https://github.com/advisories/GHSA-776f-qx25-q3cc
No fix available
node_modules/xml2js
  @vscode/l10n-dev  *
  Depends on vulnerable versions of xml2js
  node_modules/@vscode/l10n-dev

Will this be addressed on the @vscode/l10n-dev package side?

Conflicting documentation

Dear me. :-) I know it's not the best place to ask but how do you use it? Documentation seems to be very contradicting, some suggest to use the vscode/l10n dependency, others say it's built-in already if you only want to use in the main extension. package.nls.json gets picked up all right but I simply can't make the bundle.l10n.json work.

Without the dependency, trying to use vscode.l10n.t() gives an error of Property 'l10n' does not exist on type 'typeof import("vscode").

If I do depend on the package, then I can import:

import { t } from "@vscode/l10n";

And it looks like it might work but then comes a very nasty surprise: is it real that, in the bundle files, I have to use the English original as a key? Can't I use a key like in the package level? Eg:

"extension.key1.text": "first text",
"extension.key2.text": "second text",
"extension.key3.text": "third text",

Really? I hope not because this would be rather problematic. After all, the same original won't always be translated to the same localized string. There are homonyms in English. There might be situations when the same expression has to be spelled, cased or translated differently, depending on context. Tell me I misunderstood something...

messages not found in bundle

The language bundle for the VS Code HTML extension looks like that:

{
    "": [
        "--------------------------------------------------------------------------------------------",
        "Copyright (c) Microsoft Corporation. All rights reserved.",
        "Licensed under the MIT License. See License.txt in the project root for license information.",
        "--------------------------------------------------------------------------------------------",
        "Do not edit this file. It is machine generated."
    ],
    "version": "1.0.0",
    "contents": {
        "bundle": {
            "\"store\" a boolean test for later evaluation in a guard or if().": "„store“ i.e. Speichern eines booleschen Tests für die spätere Auswertung in einem Wächter oder if().",
            "'from' expected": "„from2 erwartet",
            "'in' expected": "„in“ erwartet",
            "'through' or 'to' expected": "„through“ oder „to“ erwartet"
            ...
       },
        "package": {
             ...
       }
}
  • I get the location of the file with l10n.uri?.toString(), I pass it to the HTML language server
  • The language server reads the file (workaround for #81), and passes it to config
	const contents = await (await fs.readFile(uri.fsPath)).toString();
	l10n.config({ contents });
  • no strings are found

Problem is that l10n looks up the strings in the root, not contents.bundle

bundle = JSON.parse(config.contents);

  • same when loading from the file or URI

XLF Export/Import issues with multiline strings

The XLF handling appears to break things when either the Message or Comment contains a newline \n or apostrophe '

vscode.window.showInformationMessage(vscode.l10n.t({
    message: "Untranslated Test String 1", comment: ["Translation notes 1"]}));
vscode.window.showInformationMessage(vscode.l10n.t({
    message: "Untranslated Test String 2", comment: ["comment contains ' apostrophe"]}));
vscode.window.showInformationMessage(vscode.l10n.t({
    message: "Untranslated Test String 3\nSecond Line",
    comment: ["This is a multi line string "]}));
vscode.window.showInformationMessage(vscode.l10n.t({
    message: "Untranslated Test String 4\nSecond Line",
    comment: ["This comment has' an apostrophe "]}));

Running npx @vscode/l10n-dev export --outDir ./l10n ./src builds a correct looking bundle.l10n.json:

{
  "Untranslated Test String 1/Translation notes 1": {
    "message": "Untranslated Test String 1",
    "comment": [
      "Translation notes 1"
    ]
  },
  "Untranslated Test String 2/This comment has' an apostrophe": {
    "message": "Untranslated Test String 2",
    "comment": [
      "This comment has' an apostrophe"
    ]
  },
  "Untranslated Test String 3\nSecond Line/This is a multi line string ": {
    "message": "Untranslated Test String 3\\nSecond Line",
    "comment": [
      "This is a multi line string "
    ]
  },
  "Untranslated Test String 4\nSecond Line/This comment has' an apostrophe ": {
    "message": "Untranslated Test String 4\\nSecond Line",
    "comment": [
      "This comment has' an apostrophe "
    ]
  }
}

but running npx @vscode/l10n-dev generate-xlf ./package.nls.json ./l10n/bundle.l10n.json -o ./loc/translations-export.xlf produces an XLF with

  1. ' replaced with &apos;
  2. the multi-line string's trnas-unit id is the string literal instead of the expected "++CODE++[hash-of-source-string]" format that all the other strings use
<?xml version="1.0" encoding="utf-8"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
  <file original="bundle" source-language="en" datatype="plaintext"><body>
    <trans-unit id="++CODE++304bc3c345a64372f0f8358c17e4f9d35fa33d96449c6765a7d103e5a8c39eb6">
      <source xml:lang="en">Untranslated Test String 1</source>
      <note>Translation notes 1</note>
    </trans-unit>
    <trans-unit id="++CODE++cc3487b2f5e8652da11876778a63ba5ccbfa9c61e44354dbe607c741c2a36320">
      <source xml:lang="en">Untranslated Test String 2</source>
      <note>This comment has&apos; an apostrophe</note>
    </trans-unit>
    <trans-unit id="Untranslated Test String 3
Second Line/This is a multi line string ">
      <source xml:lang="en">Untranslated Test String 3\nSecond Line</source>
      <note>This is a multi line string </note>
    </trans-unit>
    <trans-unit id="Untranslated Test String 4
Second Line/This comment has&amp;apos; an apostrophe ">
      <source xml:lang="en">Untranslated Test String 4\nSecond Line</source>
      <note>This comment has&apos; an apostrophe </note>
    </trans-unit>
  </body></file>
  <file original="package" source-language="en" datatype="plaintext"><body>
    <SNIP/>
  </body></file>
</xliff>

Using this to build a fake French import.fr.xlf

<?xml version="1.0" encoding="utf-8"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
  <file original="bundle" source-language="en" datatype="plaintext" target-language="fr"><body>
    <trans-unit id="++CODE++304bc3c345a64372f0f8358c17e4f9d35fa33d96449c6765a7d103e5a8c39eb6">
      <source xml:lang="en">Untranslated Test String 1</source>
      <note>Translation notes 1</note>
      <target state="translated">FAKE Translation 1</target>
    </trans-unit>
    <trans-unit id="++CODE++cc3487b2f5e8652da11876778a63ba5ccbfa9c61e44354dbe607c741c2a36320">
      <source xml:lang="en">Untranslated Test String 2</source>
      <note>This comment has&apos; an apostrophe</note>
      <target state="translated">FAKE Translation 2</target>
    </trans-unit>
    <trans-unit id="Untranslated Test String 3
Second Line/This is a multi line string ">
      <source xml:lang="en">Untranslated Test String 3\nSecond Line</source>
      <note>This is a multi line string </note>
      <target state="translated">FAKE Translation 3\nSecond Line</target>
    </trans-unit>
    <trans-unit id="Untranslated Test String 4
Second Line/This comment has&amp;apos; an apostrophe ">
      <source xml:lang="en">Untranslated Test String 4\nSecond Line</source>
      <note>This comment has&apos; an apostrophe </note>
      <target state="translated">FAKE Translation 4\nSecond Line</target>
    </trans-unit>
  </body></file>
  <file original="package" source-language="en" datatype="plaintext" target-language="fr">
    <body>
      <SNIP/>
    </body>
  </file>
</xliff>

and running npx @vscode/l10n-dev import-xlf ./loc/translations-import/*.xlf --outDir ./l10n produces this bundle.l10n.fr.json

{
    "Untranslated Test String 1/Translation notes 1": "FAKE Translation 1",
    "Untranslated Test String 2/This comment has' an apostrophe": "FAKE Translation 2",
    "Untranslated Test String 3\r\nSecond Line/This is a multi line string ": "FAKE Translation 3\\nSecond Line",
    "Untranslated Test String 4\r\nSecond Line/This comment has&apos; an apostrophe ": "FAKE Translation 4\\nSecond Line"
}

which results in the translation lookups failing and displaying the untranslated strings:
image

Issues here are

  1. because the XLF step translated the character \n to a string literal line break, this became a \r\n on the Windows build machine, and the resulting test strings 3 and 4 to miss, and not display the translated string.
  2. The ' apostrophe in String 2 was correctly returned from it's &apos; form, but the one within String 4 was not. This means that a manual or scripted fix of swapping \r\n to \n fixes the string lookup for string 3, but not for string 4 as the lookup is expecting the literal ' character

Workaround:
Everything works when we add steps to both replace the \r\n with \n and the &apos; to '

Cannot export value on deconstruction

For example, the following two cases cannot be scanned by l10-dev:

import * as vscode from 'vscode';
const {
  l10n: { t },
} = vscode;
const { l10n: aaa } = vscode;

console.log(t('Hello {0}', ['yiliang114']));
console.log(aaa.t('Hi {0}', ['yiliang114']));

Passing outDir to command always outputs warning that it doesn't match packageJSON.l10n value.

In package.json, I have

  "l10n": "./l10n",

When I run the command npx @vscode/l10n-dev export -o ./l10n ./src, It always outputs

The l10n property in the package.json does not match the outDir specified. For an extension to work correctly, l10n must be set to the location of the bundle files.

But the code seems to succeed.

I think the comparison here just needs to use !==?

if (outDir) {
if (!packageJSON.l10n || path.resolve(packageJSON.l10n) === path.resolve(outDir)) {
console.warn('The l10n property in the package.json does not match the outDir specified. For an extension to work correctly, l10n must be set to the location of the bundle files.');
}
} else {
outDir = packageJSON.l10n ?? '.';
}

Unexpected Behavior - Translation notes are appended to the English string when no translation is provided

When a translation is not provided for a string, I would expect that just the default English string would be used as a fallback. However, it appears that vscode.l10n.t(...) is instead using a concatenation of the string and the translation notes.

For a simple example:

vscode.window.showInformationMessage(vscode.l10n.t({
    message: "Untranslated Test String 1", comment: ["Translation notes 1"]}));
vscode.window.showInformationMessage(vscode.l10n.t({
    message: "Untranslated Test String 2", comment: ["Translation notes 2"]}));

with a bundle.l10n.fr.json which only translates one of those strings like

{
  "Untranslated Test String 1/Translation notes 1": "Fake Translation String 1"
}

we get the expected output when VS Code is set to English:
image
but switching to French ends up appending the translation notes for the one untranslated string:
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.