dmnd/dedent

⬅️ ES6 string tag that strips indentation from multi-line strings.

dedent's Issues

Onboard to create-typescript-app@1

Splitting out the last step from #36: I'd like to onboard most or all of the tooling set up in http://github.com/JoshuaKGoldberg/template-typescript-node-package https://github.com/JoshuaKGoldberg/create-typescript-app.

Predictive transformation

i tried to used dedent and came accros two issues :
First i had some bug with \` being transformed the wrong way (i can't remember exactly)
Second is if you try for example to add a first blank line or you want to tab all you text you can't.

Here is the short module i ended up writing to overcome both. If you want i could make it a pr.
Code here : https://github.com/ekino/veggies/blob/master/src/extensions/snapshot/dedent.js
tests here : https://github.com/ekino/veggies/blob/master/tests/extensions/snapshot/dedent.test.js

For commodity here is the code

/**
 * Extract spaces length
 * @param {string} text
 * @returns {number} length tab and space before first char
 */
const getSpacesLength = text => {
    let length = 0

    while (length < text.length) {
        const char = text[length]
        if (char !== ' ' && char !== '\t') break
        length += 1
    }

    return length
}

/**
 * Used to remove indentation from a text. Usefull with multine string in backticks.
 *
 * Two way to use it : `
 *     My text
 *       Another line
 *     Another line again
 * `
 * the result text will be :
 * "My text
 *   Another line
 * Another line again"
 *
 * In this case, alignment is done on the length of the first character that is not a space or a tab of all lines
 *
 * Or
 *
 * Another way : `
 *     """
 *       My text
 *         Another line
 *      Another line again
 *     """
 * `
 * the result text will be :
 * "  My text
 *      Another line
 *  Another line again"
 *
 *  In this case, alignment is donne on the spaces or tab before """
 *
 * Warning : First line and last line will always be ignored
 *
 * @param {string} text
 * @return {string}
 */
module.exports = text => {
    if (typeof text !== 'string') text = text[0]

    let lines = text.split('\n')
    if (lines.length < 3) return text

    lines = lines.slice(1, lines.length - 1)

    let skipLength = getSpacesLength(lines[0])

    if (
        lines[0].substr(skipLength, 3) === '"""' &&
        lines[lines.length - 1].substr(skipLength, 3) === '"""'
    ) {
        lines = lines.slice(1, lines.length - 1)
    } else {
        for (let i = 0; i < lines.length; i++) {
            const line = lines[i]
            skipLength = Math.min(skipLength, getSpacesLength(line))
        }
    }

    const resultLines = []
    for (let i = 0; i < lines.length; i++) {
        resultLines.push(lines[i].substring(skipLength))
    }

    return resultLines.join('\n')
}

Repeated indentation

Is this test actually testing the expected behaviour:

it("works with removing same number of spaces", () => {
  const dd = require("../dedent");
  const result = dd`
                    first
                       second
                          third
                   `;
  expect(result).toBe("first\nsecond\n   third");
});

I thought this was a bug, and have a fix for it, but then there's a test for it.

I would have expected the result to be "first\n___second\n______third" (MD is eating up my whitespace).

Let me know if it is actually a bug and I'll make a PR

Bump devDependencies to latest (as of May 2023)

Filing this issue as part of the plan in #36 for tracking. This package's package.json hasn't had dev dependencies bumped in quite a few years.

#32 includes a good reference for some of the updates that could be made.

Convert macro.js to TypeScript

Following #50 -> #51: I didn't try to convert macro.js to TypeScript because I'm not very familiar with https://github.com/kentcdodds/babel-plugin-macros. But in theory it'd be nice for someone who is more familiar to:

• Delete macro.d.ts
• Rename macro.js to macro.ts
• Adjust the build tooling so that macro.d.ts gets recreated, akin to today's build:types for dedent.js -> dist/dedent.d.ts

Convert dedent.ts and tests to TypeScript

Splitting out from #36: I'm going to convert the main source files in this repo to TypeScript.

I would convert macro.js but it's not something I'm particularly familiar with. Will allow that to be a followup.

escaped `$` and `{` characters are not handled correctly

Follow on from #2 - currently dedent does not treat $ or { as special when determining what it should do with escaped characters:

dedent`\$` === '\\$'
dedent`\{` === '\\{'

`babel-plugin-macros` should be an optional peer dependency

While I've not used macros myself, from what I can tell dedent/macro.js is only expected to be used when babel-plugin-marcos is being used and is a special file that is consumed by the plugin (i.e. even if I'm using babel-plugin-macros, I still myself wouldn't write require('dedent/macros.js')).

As such, I think it's safe to list it as an optional peer dependency so that folks who are not using the macro don't have to carry around this extra dependency.

🐛 Bug: babel-plugin-macros isn't an optional peer anymore

Bug Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Expected

babel-plugin-macros being an optional peer

Actual

it no longer is an optional peer right now in main

Additional Info

It looks like #60 got undone in #55, which might be a mistake as there's not a good reason to include a babel plugin as a package dependency. Maybe it should be a devDependency if the maintainers aren't willing to move it as a peer dep like the plugin says it should be installed

🛠 Tooling: Set up tokens for automated publishing & contributor recognition

Bug Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Overview

Just tracking for visibility: I haven't been able to put the ACCESS_TOKEN or NPM_TOKEN tokens in place per https://github.com/JoshuaKGoldberg/create-typescript-app/blob/7aa6d6b57e6c84e1ff134bb528a72e0c5ecd61a7/docs/Creation.md. #55 added in tooling that uses them. Until those are added to this repo, the automated contributors and release jobs will fail. I've reached out to try to get access or have them added for me.

Additional Info

No response

🐛 Bug: Upgrading Node to verison 18 broke builds

Bug Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Expected

A major node version dependency should not be upgraded in a bug fix.

Actual

NPM is showing the latest version is 1.5.2, which requires Node 18. This is breaking our application since Jest 29.5.0 has this somewhere in the dependency tree only supporting Node 16.

Additional Info

It looks like that tag was removed from GitHub. A new tag that reverts functionality back to 1.5.1 should be tagged as 1.5.3.

🚀 Feature: Add an option to disable special character escaping

This pulls together #2, #26, #45, #46:

• #2: Backticks were confirmed as expected to be (un-)escaped properly when dedent is used with template literal strings.
  • By extension, it makes sense to consider other native template literal string escapings as unwanted: #45 -> #46
• #26: On the other hand, when dedent is used as a function, users aren't expecting that special character escaping

I think the "right" (or: least unexpected) way would be to add an option with a not-very-savory name like escapeSpecialCharacters. The option would default to:

• true when called for a template literal string
• false when called as a function

Proposal:

• Let's add this option in separately from #46, to resolve this issue and 26
• I can release a new minor version that includes this option and 26
  • I'm a little hesitant to merge that PR before this option is in, in case folks are relying on the existing behavior

Thoughts @G-Rath @sirian?

`dedent` and template tag functions

Possibly interesting for the future (but I don’t have a use case for it, myself): allow dedent to be combined with template tag functions:

console.log(dedent(html)`
    <div>
    Hello!
    </div>
`);

However, things would be relatively complex, because you’d want to dedent before passing the input on to the tag function.

error with npm install on firebase project

npm ERR! code E404
npm ERR! 404 Not Found - GET https://repo.huaweicloud.com/repository/npm/dedent/-/dedent-1.5.1.tgz
npm ERR! 404
npm ERR! 404 'dedent@https://repo.huaweicloud.com/repository/npm/dedent/-/dedent-1.5.1.tgz' is not in this registry.
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

ES Module

Please publish as an ES Module.

If users need CJS, it could be compiled from the standard javascript.

1.0.1 type change

Argument of type 'string' is not assignable to parameter of type 'TemplateStringsArray'.

Means this code in the readme (and the code I wrote 😂) is no longer valid:

const third = dedent(`
    Wait! I lied. Dedent can also be used as a function.
`);

The workaround is to use ES6 string tags:

const first = dedent`A string`

Add a changelog

Hello, you've recently updated dedent from 0.7.0 to 1.0.2 which means in semver that this new version is major and potentially breaking.

Where can I found your changelog?

Dedent is massively used https://www.npmjs.com/package/dedent

The community will highly benefits from having an easy to find changelog (with a copy within https://github.com/dmnd/dedent/releases).

And also please could you also use GitHub relase -> https://github.com/dmnd/dedent/releases that makes it really easy to find. Specially for developers and maintainers like me that deals with hundreds of packages to updates 🙏

Thanks for your awesome work

Node ESM compatibility issue

When using dedent in a node ESM environment, a import dedent from 'dedent' statement will result a {default: dedent} object.

> const dedent = await import('dedent')
undefined
> dedent`abc`
Uncaught TypeError: dedent is not a function
> dedent('abc')
Uncaught TypeError: dedent is not a function
> dedent
[Module: null prototype] {
  __esModule: undefined,
  default: <ref *1> [Function: dedent] { default: [Circular *1] }
}
> dedent.default`abc`
'abc'

This issue comes from several reasones:

1. In package.json a module filed is specified to dist/dedent.mjs, but NodeJS doesn't support this field
2. The main field in package.json references to dist/dedent.js which is a CommonJS module, NodeJS's ESM then transforms module.exports into a default export

To address this, we need a exports field in package.json:

{
  "exports": {
    ".": {
      "types": "./index.d.ts",
      "import": "./dist/dedent.mjs",
      "default": "./dist/dedent.js"
    }
  }
}

This will solve the import issue, but is potentially a breaking change to NodeJS ESM environment, would it worse a major version?

Project Status: Still Alive!

👋 Hi all! I'm going to be helping out with maintenance of the project for a little while. Pinning this issue to explain project context & next steps.

Project Context

The dedent package has been one of the more common, popular packages in the JS ecosystem for quite a while. Per dedent package on npm there are >16 million downloads a week. Quite impressive given that the last published version was 6 years ago.

In the long term, https://github.com/tc39/proposal-string-dedent is mostly going to become the standard way to dedent strings in JavaScript. But even if that were to be ratified today, it'd be some time before all engines run by users supported it. So this dedent package is here to stay for a while.

Both this project's popularity and the presence of a TC39 proposal are strong indicators that dedenting strings is a common, important user need in JS development!

Next Steps

Here's what I'm going to do with this repository:

• Triage the open issues
• Review the open PRs
• Generally bump dev dependencies to their latest versions: #37
• Switch from Flow to TypeScript: #51
• Add in some more tooling (issue templates, allcontributors bot, etc.) from template-typescript-node-package : #55

Publishing New Versions

I'm going to try to publish a new beta version with changes that have been merged over the last few years. I'll try to tag it as a beta so folks can preview it. Just to be safe. #16

Update July 2nd, 2023: [email protected] is published under the beta tag. I'll promote it to stable next week.

Update July 10th, 2023: [email protected] is published as stable. 🚀

Out of Scope

For now, adding any large new features to dedent is out of scope. This package is going to stay mostly as-is.

Bug fixes will of course be accepted & released.

---

Please do post feedback if you have any questions or suggestions. Cheers all! ❤️

🛠 Tooling: Build failures on main from references to yarn

Bug Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Overview

From https://github.com/dmnd/dedent/actions/runs/7278084706/job/19831511946?pr=84:

Run pnpm build

> [email protected] build /home/runner/work/dedent/dedent
> yarn build:legacy && yarn build:modern && yarn build:types

error This project's package.json defines "packageManager": "yarn@pnpm@8.[7](https://github.com/dmnd/dedent/actions/runs/7278084706/job/19831511946?pr=84#step:4:8).0". However the current global version of Yarn is 1.22.21.

Looks like I missed some references to yarn.

Additional Info

No response

Should only trim one newline at start and end so that you can add custom newlines

Looks like the module removes all newlines from start and end, but I think it makes more sense to only remove one newline at start and end so that I can add newlines without adding \n.

// Adds a newline at the end. Currently the newline would be deleted.
dedent`
  foo
  bar

`

Strips trailing newline

dedent(`
  <p>Hello world!</p>\n
`);

becomes <p>Hello world!</p> without newline in the end.

Handle escaped backticks is wrong when dedent used as function

dedent("\\`") === "`"

Bug in this line https://github.com/dmnd/dedent/blob/master/dedent.js#L17

Expose a util function

Hi there. Thanks for the package. It works really well.

One small piece of feedback: as someone who is using this in my own library and not as a template writer, I don't actually have any templates in my own code. Instead, users pass templates to my library. I just need to know that the templates passed to my package methods have been dedented. So, really I just want to call a simple function that returns a new, dedented template.

Here is the function that I'm currently using to do do this:

var dedent = require('dedent');

function dd(str) {
    return dedent`${str}`;
}

Then calling dd(template); will obviously return a dedented version of template. It would be nice if this method was accessible somewhere.

Cheers.

Support babel-macros

It would be nice to have this at build time instead of runtime, would you be open to a PR adding /macro.js with glue code for babel-plugin-macros? I'll gladly make that PR 😄

cc @kentcdodds

Tabs?

Hey,

Is it me or doesn't dedent support dedenting tabs?

Cheers

Please tag release

npm has 0.7.0, but here on github, 0.6.0 is the latest : please just tag the new release.

📝 Docs: add an 'install' section to the README?

Bug Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Expected

• Have no misleading error during dev
• Be able to auto-import dedent

Actual

I installed dedent with pnpm install dmnd/dedent

Then Typescript can't find the declaration but at runtime (Vite devServer) my strings are dedented:

Also trying to auto-import it with https://github.com/unplugin/unplugin-auto-import configured with

              'dedent': [
                [`default`, `dedent`],
              ],

I get

Additional Info

• My project is a pnpm monorepo with one tsconfig per package. dedent is a dep of only one of the packages

• When I go to node_modules/dedent, the dist folder is missing.

• When I install the dev dependencies and build it, the dist folder is created and everything works well

• Investigating I realized that my package.json was odd
  

• so I run a pnpm remove dedent then pnpm install dedent and everythin works well now (removing it first is mandatory)

• I open this bug anyway as it may be useful to add an 'install' section to the README to avoid this. I personally didn't know that pnpm would automatically map to github if I added the vendor in the install command.

Newline issue

Started here indentjs/endent#8

More details here graphql-nexus/nexus-prisma#50

The issue is that if \n shows up in a template string it gets read as a newline.

Consider being `String.dedent`

I realise that string-dedent is apparently the polyfill (I don't know if its technically official, but it is the library used in the REPL), but I think there is value in mirroring the behaviour laid out in the String.dedent proposal because:

• it seems fairly stable
• it will enable users to more easily transition to using String.dedent if/when it lands
• it should help save on mental energy since behaviours have already been bikeshedded for the proposal

Again while string-dedent is a package that can be installed through npm, unless there is something people really disagree with in the String.dedent proposal (which I think would be of serious concern anyway), I think there is value is having one of the most popular dedent libraries match its behaviour more.

At the least the REPL indicates:

• escapes are always consumed - you have to do \\ to escape an escape (relates to #69)
• only the first start and end newline are trimmed (#41)
• considers the first unintended line (#20)
• gives the expected results for nested dedents (#12)

My personal concern is that String.dedent will come out with these subtle differences that mean I have to relearn how dedenting works - so having dedent start to mirror String.dedent more closely would address that.

Is this project dead?

There are several pull requests and unanswered issues. Is this project dead?

Does not support escape sequences

$ node
> dedent = require(".").default
[Function: dedent]
> `a\xa0\tb`
'a \tb'
> dedent`a\xa0b`
'a\\xa0\\tb'
> ⏎       

(For people stumbling upon this issue: https://github.com/MartinKolarik/dedent-js might work better for you.)

🐛 Bug: \\n -> \n should only happen with template strings

Bug Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Expected

dedent("\\nu") should return the string unchanged.

Actual

dedent("\\nu") turns the \\n into a newline, i.e. returns a newline character + u.

Additional Info

This happens because result.replace(/\\n/g, "\n") is called regardless of the value of escapeSpecialCharacters, i.e. regardless of whether dedent is called as a tag on a template or as a function on a string.
An easy fix would be to make the replacement conditional, solving this particular problem. (However, it would still not work for dedent`\\nu` because here, still the \\n would be replaced 🤷‍♂️)

Bug(?) JSON.serialize(..., null, 2) throws off indentation

not sure if this is a bug or somehow expected behaviour that i don't understand yet, but here's a lil repro:

$ cat test.js 
const dedent = require('dedent');

console.log(
    dedent`
        Foo
        ${JSON.stringify({ a: 'b' }, null, 2)}
        Bar
    `,
);

$ node src/test.js 
Foo
      {
"a": "b"
}
      Bar

Observations:

• the Bar line is not at the same level of indentation as Foo.
• the serialized json string has odd indentation, but i sort of understand why this happens, and might be outside the scope of this library to fix....but would be super nice if it did :P
• same thing happens with util.inspect

thanks!

Add an option to recursively dedent nested template literal strings

Hijacked by @JoshuaKGoldberg on September 4th, 2023: see #12 (comment). Now accepting PRs for a dedent({ recursive: true }) option.

---

const a = dedent`
hello
  world
`;
const b = dedent`
  foo.
  ${a}
  bar.
`;

results in

foo.
hello
world
bar.

instead of

foo.
hello
  world
bar.

This is unfortunate because it would be nice to compose dedented strings.

If you decide to fix this, any test cases should also check that inlined dedents make sense:

const weird = dedent`
  foo.
  ${things.map(x => dedent`
    hello
      world
  `).join('\n')}
  bar.
`;

CommonJS breakage: many consumers still use dedent = require("dedent")

In the (old) current version 0.7.0, dedent is published with a dist/dedent.js that sets module.exports to the dedent function:

if (typeof module !== "undefined") {
  module.exports = dedent;
}

That's meant to be used with older-style Node.js code like:

const dedent = require("dedent");

However, in e13e334, the style was switched over to ES modules style default export:

exports.default = dedent;

This is a breaking change that just hasn't been published yet. Per this Sourcegraph query for const dedent = require("dedent"), many users are still using the old style API.

Since I'm trying to keep this package stable (#36), I'm going to use https://www.npmjs.com/package/babel-plugin-add-module-exports to dial back the breaking change. That way folks can import both with the old const dedent = require("dedent") and the new import dedent from "dedent" ways.

Feature request: support `join` option

I would like a "join" option, so that multi-line strings could be both dedented and joined into a single line.
Example:

example = dedent(' ', `
  This is a
  multiline string
  that is joined with spaces
  instead of newlines.
`); // "This is a multiline string that is joined with spaces instead of newlines."

A little backstory, I actually have my own dedent function implemented in several projects, and use it constantly! I found this repo when thinking about publishing my dedent. I'm glad to see it already exists!
This 'join' feature is something I use quite often, too, so I'm hoping its something I could add to this project.
Let's discuss?

Non-indented lines are ignored when calculating minimum indentation

Consider this example

dedent`
test
  test2
`

Outputs

test
test2

Regex in line 43 (l.match(/^(\s+)\S+/);) should probably read \s* instead of \s+.

Publish new version

The ES6 default export is a good thing, but hasn't been published yet.

Could you please npm version & npm publish?

Allow first line to be blank

E.g.:

      console.log(dedent`
        if foo
          br
          hr
      `)

Cheers

dedent.d.mts missing in package.json > files

Ref: #67 (comment)

Leading whitespace should not be stripped from interpolated components

As currently written, this module performs interpolation before
stripping leading whitespace. In particular, whitespace is stripped from
the interpolated components. I think that this is the wrong behavior.

My understanding is that dedent is supposed to be effectively
syntactic sugar. Writing

function foo() {
  const things = dedent`\
    lorem ipsum
    dolor sit amet
  `;
  return things;
}

should be completely equivalent to writing

function foo() {
  const things = `\
lorem ipsum
dolor sit amet
`;
  return things;
}

with the sole benefit that the code indentation doesn’t have to become
wonky just to accommodate the text.

This suggests that

function foo() {
  const stuff = "abc\n    xyz";
  const things = dedent`\
    lorem ${stuff}
    dolor sit amet
  `;
  return things;
}

should also be equivalent to

function foo() {
  const stuff = "abc\n    xyz";
  const things = `\
lorem ${stuff}
dolor sit amet
`;
  return things;
}

which means that the output should be

lorem abc
    xyz
dolor sit amet

but with the current version of dedent the result is actually

lorem abc
xyz
dolor sit amet

Let me provide a concrete case where this matters. I was using dedent
to server-side render static pages on my site. The code looked roughly
like this:

            // lots of context, so this code is indented a bunch...
            const page = dedent`\
                <!DOCTYPE html>
                <html>
                <body>
                <div id="container">${html}</div>
                <script src="${pathToBundle}"></script>
                </body>
                </html>
                `;

This all worked well—I wrote it and forgot about it. But when I added
code blocks to my site, I noticed that they rendered incorrectly: code
like

<pre><code>function foo(bar, baz) {
    while (true) {
        if (bar()) {
            console.log("stuff");
            if (baz()) {
                console.log(
                    "even more stuff"
                );
            }
        }
    }
}</code></pre>

was being rendered as

<pre><code>function foo(bar, baz) {
while (true) {
if (bar()) {
console.log("stuff");
if (baz()) {
console.log(
    "even more stuff"
);
}
}
}
}</code></pre>

in the static HTML. Because pre sets white-space: pre, the resulting
code is rendered incorrectly in the browser, too. This was somewhat
difficult to track down!

I ended up replacing this module with a hand-written version that fit my
purposes better. (There was an additional change that I needed to make
to properly render code blocks with line-trailing backslashes.) You can
see that change here:

wchargin/wchargin.github.io@06475d4

Is this something that you’re interested in pulling upstream? I realize
that it is a breaking change, so I’d understand if you’re not
interested.

(Also, 👋 !)

Avoid escaping $ without a { after it?

Cross-posting @Haroenv's #65 (comment):

I wonder if instead of an option, the real issue is that $ without a { after it doesn't need to be escaped, right?

@G-Rath - do you have thoughts here?

🐛 Bug: Doesn't do dedent if first character is a space character

Bug Report Checklist

• I have tried restarting my IDE and the issue persists.
• I have pulled the latest main branch of the repository.
• I have searched for related issues and found none that matched my issue.

Expected

const d = require('dedent')
const result = d(' first line\n   second line'); // has space character at the beginning

// result is  'first line\nsecond line'

Expecting first line and second lines to be dedented

Actual

const d = require('dedent')
const result = d(' first line\n   second line'); // has space character at the beginning

// result is  'first line\n   second line'

Actual behavior is that the first space is stripped and the second line is not dedented

Additional Info

All code was executed and tested on the library version 1.5.1

node.js - v16.15.1

Doesn't handle \` escapes correctly

Expected:

dd`\`` === '`'

Actual:

dd`\`` === '\\`'