📎 Embedded language formatting

[ah-yu]

📎 Embedded language formatting

[Sec-ant]

Great great work 🚀 . I'd like to add some additional notes. I will update them in this comment later.

---

TLDR, I think we should broaden the concept of embedded language. I'll provide my rationale below, gradually (been struggling with a tight schedule).

The Concept and Scope of Embedded Language

We should envision the introduction of embedded language to serve two primary purposes:

1. Enhancing the Developer Experience: This feature aims to improve the developer experience when formatting foreign languages within their code. For example, when writing CSS template literals in a JavaScript file, it could be structured as follows and the formatter should be able to format the CSS code:

   const style = css`
     div {
       color: red;
     }
   `;

2. Supporting Composable Parsing Infrastructure: This feature should also strengthen our parser infrastructure to support languages composed of multiple embedded languages. For instance, an HTML file can contain <style> tags with embedded CSS and <script> tags with embedded JavaScript. This composable parsing infrastructure allows us to fully support modern frameworks like Vue, Svelte, and Astro. By reusing the same parsing infrastructure for each embedded language (HTML, JavaScript, and CSS), we can seamlessly integrate and compose them in various ways.

Configuration of Embedded Language

Possible Configuration

I believe the configuration of embedded languages includes these aspects at least:

1. Identifying CST Structures: The first step is to determine which CST (Concrete Syntax Tree) structures within a language can be considered as embedded languages. For example, in JavaScript, we might consider JsTemplateExpression as a block to have embedded languages in it.

2. Determining the Language/Parser: Next, we need to identify the language or parser for the embedded language. For instance, we would use the tag within a JsTemplateExpression to identify the language and select the appropriate parser to parse this embedded language.

3. Providing Language-Specific Options: We should offer language-specific options for embedded languages that can override the global language-specific options. So they can be parsed / formatted / linted with a different set of options.

4. A Switch to Opt Out: There should be a switch for users to fully opt out from embedded language detection.

For the first two configuration items, I believe we can leverage our plugin infrastructure. This means users can configure different Grit patterns to inform Biome which CST structures in a language should be considered embedded language blocks and which parser should be used to parse them.

Regarding the third configuration item, we should utilize our existing configuration file structure for language-specific options. This can be done in a way similar to overrides, allowing users to specify different configurations for embedded languages.

Regarding the fourth configuration item, the opt-out option should be configurable for each language, and it should also be a top-level option.

Extent of Configurability

Relating to the two primary purposes mentioned earlier:

For the first purpose (enhancing the developer experience), users should be able to configure the above aforementioned settings freely. This is because the support for embedded languages in this context is not inherently part of the language itself. For example, different libraries may have different APIs for embedding CSS, such as css`...`, style`...`, or css.style`...`. Additionally, users might have their own SDKs or coding styles. Therefore, we should not hardcode these configurations into our infrastructure. Instead, we should provide users with a configuration interface using Grit patterns. I believe we can provide sane default presets to provide a nice out-of-box experience, but users should be able to override them as they want.

For the second purpose (supporting composable parsing infrastructure), there should be more restrictions. For example, when parsing Astro files, the frontmatter is defined as JavaScript code according to Astro's specification. In this case, we should not let users to override this behavior, but rather enforce the logic by our parser. Users shouldn't be allowed to opt out from the enforced logic. However, I think we should still offer the configurablity to the extent that users are allowed to add more patterns to target certain CST nodes as embedded language blocks.

Note

I will add an example of how the configuration would be like here.

Integration Phase of Embedded Language

Another critical consideration is determining at which phase embedded language support should be implemented: the parsing phase or the formatting phase. In #3228, we placed it in the formatting phase, which aligns with Prettier's approach. However, I believe Biome can improve on this.

The ideal phase for supporting embedded languages is the parsing phase. The rationale behind this is that, for languages like Astro (related to purpose 2), in the CST, the frontmatter can be mapped to JavaScript CST nodes, rather than being treated as a literal text node. This approach enhances the support experience for such compositional languages. Additionally, it allows our plugin and linter systems to reuse the same CST structure to handle the content within these embedded languages, so we can add embedded linting/checking support for them later.

One thing to consider in this approach is that we might also want to preserve the original CST nodes, so, for example, linter or formatter rules will still work for the container of the embedded language. Also, for languages that have interpolations, such as template literals in JavaScript, we should also keep the nodes of the expressions in the CST.

Nested Embedded Languages

Note

TBD. One example you can think of is embedded HTML in a JS file, in which there're inline <style> tags.

Indentation Handling in Embedded Language Formatting

Note

This is a tricky one because spaces at line starting can be significant when they appear in multi-line template literals. I'll elaborate later.

Interpolation Handling

Note

This has been discussed in #3228

Examples of Embedded Languages

Note

TBD. I'll come back later.