ryancavanaugh/learn-a

Sample repo using lerna with TypeScript project references

Lerna + Project References

This is a "bare minimum" repo that shows one way to configure TypeScript Project References with lerna. There are a lot of different ways you can set things up and this isn't intended to be authoratitive guidance or exclusionary of other ways that might work better in your project.

Setting up this repo

> git clone https://github.com/RyanCavanaugh/learn-a.git
> cd learn-a
> npm install
> lerna bootstrap
> tsc -b packages

Note that you'll need a 3.0 version of tsc (currently available at npm install -g typescript@next).

General Structure

As with a normal lerna repo, there's a packages folder. Inside we have three creatively named packages pkg1, pkg2, and pkg3.

packages/
| tsconfig.settings.json
| tsconfig.json
| pkg1/
  | tsconfig.json
  | src/
  | | (typescript files)        
  | lib/
  | | (javascript files)
  | | (.d.ts files)
| pkg2/
  | (same as pkg1)
| pkg3/
  | (same as pkg1)

Let's review each file in the repo and explain what's going on

tsconfig.settings.json

{
    "compilerOptions": {
        // Always include these settings
        "composite": true,
        "declaration": true,
        "declarationMap": true,
        "sourceMap": true,

        // These settings are totally up to you        
        "esModuleInterop": true,
        "target": "es5",
        "module": "commonjs",
        "strict": true
    }
}

This file contains the "default" settings that all packages will use for compilation. You will definitely want the composite, declaration, declarationMap, and sourceMap settings enabled for all projects, so include those in this file. Other settings, like target and strict, can be specified here if you'd like to enable them by default. You'll also be able to override these settings on a per-package basis if needed.

tsconfig.json

{
    "files": [],
    "references": [
        { "path": "pkg1" },
        { "path": "pkg2" },
        { "path": "pkg3" }
    ]
}

This file is pretty simple - simply list the packages that need to be built with tsc in the references array. You should also include "files": [] in this file - this will prevent an incorrect invocation of tsc without -b from trying to build the entire packages folder source files as one compilation (which will fail, but drop a bunch of .js files in random places as an annoying side effect).

packages/pkg2/tsconfig.json

We'll just cover one of the pkg1 / pkg2 / pkg3 packages since they're basically identical for the purposes of this demo. Here's pkg2's tsconfig.json:

{
  "extends": "../tsconfig.settings.json",
  "compilerOptions": {
    "outDir": "lib",
    "rootDir": "src"
  },
  "references": [
    { "path": "../pkg1" }
  ]
}

The extends property pulls in the settings we wrote in tsconfig.settings.json, so we don't have to duplicate any settings described there.

In compilerOptions, we've set outDir to lib and rootDir to src, then placed all my .ts files in src. This means src/index.ts will build to lib/index.js and lib/index.d.ts. This is also the place where you could override settings like strict or target if you needed to change them on a per-project basis.

In the references array, we list the paths to the other projects' tsconfig.json files (or containing folders, as shown here). This will both ensure that we locate the .d.ts files correctly, and set up a proper build ordering.

packages/pkg2/src/index.ts

import * as p1 from '@ryancavanaugh/pkg1';

export function fn4() {
    p1.fn();
}

Nothing unusual going on here. We import and export with the usual syntax. Notably, if you open this repo in an editor, you can still "Go to Definition (F12)" on p1.fn here and land in pkg1/foo.ts - the original sourcecode - even though "under the covers" it's using the much faster .d.ts file for typechecking.

packages/pkg2/package.json

Here are the relevant excerpts from the package.json:

{
  "main": "lib/index.js",
  "typings": "lib/index.d.ts",
  "scripts": {
    "prepublishOnly": "tsc -b ."
  },
  "devDependencies": {
    "typescript": "^3.0.0-dev.20180626"
  }
}

Because we build to lib, we need to set main to the .js file there and typings to the .d.ts file.

In scripts, we use the local copy of tsc (listed here as a dev dependency) to run a build mode compilation on the project. This will ensure that the lib folder is always built before npm publish, and blocks any publishes that try to push non-compiling code.

packages/pkg2/.npmignore / packages/pkg2/.gitignore

.gitignore

lib/

.npmignore

# Empty, but needs to exist

The .gitignore stops us from checking in build outputs, which is generally a good idea. By default, npm won't publish files that are ignored by git, so we need a separate .npmignore file so that the lib folder still gets published!

Workflow

All your lerna commands and workflow will work as expected here.

To build the TypeScript projects, you can run individual builds with tsc -b:

 > tsc -b packages/pkg3

Or just build everything:

 > tsc -b packages

learn-a's Issues

[ts] The files list is empty

{
  "files": [],
  "references": [
      { "path": "./packages/..." },
  ]
}

"typescript": "^3.0.1"

Creating a serverless bundle?

I've tried to use your example to create a similar example made for serverless:

https://github.com/tommedema/serverless-mono-example

The problem is that while everything builds, the serverless framework ultimately expects 1 folder with the entire application. How would you achieve this with typescript references?

Module not found.

In pkg2, I'm getting [ts] Cannot find module '@ryancavanaugh/pkg1'. unless I run tsc -b packages. This is a little annoying... I suppose you aren't supposed to work on both packages at the same time? Or just use --watch to constantly rebuild...

Also, would be nice if I could import specific files from these packages... Should I import directly from the "@ryancavanaugh/pkg1/src/*'?

Add unit test folder next to src folder

The simplicity of this example is the best thing about it. However, can you add a test folder next to src folder in pkg1? And place one .ts file in there as if that was some unit test for the fn function. I cannot figure out how to modify the tsconfig file(s) to avoid the tsc or runtime errors.

Automated CLI for setting up and managing a lerna TypeScript monorepo

I've written an npm CLI for setting up a repo like this one. Anyone who sees it please try it out and give me some feedback!
https://www.npmjs.com/package/@isomorphic-typescript/ts-monorepo

📃

.

Error message 'Cannot find module' can be better

I'm using monorepo (no lerana, with our own scripts, doesn't matter) with 9 TypeScript projects at work.
So, I'm super excited to see "references" feature.

I was playing around with this test learn-a project, and finally was able to make it working.
Here is an developer experience issue I found:

1. Just after git clone and npm install, I see:
   
   Somehow typescript depends on /packages/pkg2/node_modules/@ryancavanaugh/pkg1 even if /packages/pkg1 exists. It is somehow counterintuitive, at least error Cannot find module '@ryancavanaugh/pkg1' can be more descriptive.

2. Just after npm install inside each /package/pkg1, /package/pkg2, /package/pkg3, I see:
   
   It is a bit surprising, because now I have both pkg1 and pkg2 inside /packages/pkg3/node_modules/@ryancavanaugh:
   
   After digging I found that [email protected] on npm has wrong "main" property in the package.json:

{
  "main": "index.js",
}

Should be:

{
  "main": "lib/index.js",
}

Again, error tells nothing about 'main' property of package.json.

.npmignore can contain src/

Ignoring the src folder in .npmignore reduces package size.

Someone at MS knows haha: https://github.com/Microsoft/TypeScript/blob/master/.npmignore

A

Inconsistency between repository and npm for pkg2 and pkg3

Thanks for this sample, it hit the spot. However, It seems like the npm published packages @ryancavanaugh/pkg2 and @ryancavanaugh/pkg3 are not up-to-date with the latest checked-in package.json files. @ryancavanaugh/pkg1 seems OK.

Simple example from command line:

PS C:\> npm info @ryancavanaugh/pkg1 main   
lib/index.js
PS C:\> npm info @ryancavanaugh/pkg2 main
index.js
PS C:\> 

Expected: the main file for pkg2 should read lib/index.js, like pkg1.

I don't understand why this is inconstant, clearly the checked in pkg2/package.json has the lib/ prefix on it's main entry. However the lack of the proper entry causes code trying to use pkg2 or pkg3 to fail at runtime.

rename repo to lerna-typescript-monorepo

Or something like that. Could "learn-a" be a typo?

This would make this repo much easier to discover in google.

"rename symbol", "go-to-definition", and "find all references" in a multi-package repo

Hi @RyanCavanaugh , thanks a lot for this minimal repo, very helpful.

In your working with multi-package repos, have you found a way to use tsc language services like "find all references" or "go to definition" across packages? I haven't been able to find a way to do this thus far.... right now this is the only major drawback I can see to having a multi-package repo.

EDIT: forgot to mention this related issue in the vscode repo: microsoft/vscode#48905

Question on Editor Integration

First, thanks for putting this together Ryan! I had a look at it this evening. I downloaded, followed the instructions and then played around a bit. In VS Code, I did a rename on a function in package 2 from package 3 to see how it would work across packages. I noticed that it renamed the function in the d.ts file for package 2, rather than in the source. This resulted in no change to source, and thus no build update. It also resulted in a situation where the build would succeed even though there was an error (incorrect function name). I wasn't sure if this was because VS Code wasn't updated yet or if this was something in the TS project support itself, so I wanted to mention it here.

Intellisense referencing other packages in monorepo includes the "src" folder, instead of using only the "dist" folder structure

Hi 👋

I've wanted to complement this question here:

Just to give you a brief explanation here, my library setup is as the following:

packages/foo
├── README.md
├── babel.config.js
├── jest.config.js
├── package.json
├── setupTests.js
├── src
│   ├── components
│   │   ├── Foo
│   │   │   ├── Foo.component.tsx
│   │   │   ├── Foo.data.ts
│   │   │   ├── Foo.designs.ts
│   │   │   ├── Foo.props.ts
│   │   │   ├── Foo.style.ts
│   │   │   ├── Foo.test.tsx
│   │   │   └── index.ts
│   │   └── index.ts
│   ├── index.ts
│   ├── theme
│   │   └── index.ts
├── styled.d.ts
└── tsconfig.json

I have a rollup build that generates my libraries files in a way that allows me to import all the paths present under my library. In essence, it allows me to import:
import foo from 'foo';,
import foo from 'foo/components/Foo';,
import foo from 'foo/components/Foo/Foo.component.tsx'

Although my rollup build scheme is correct (I am able to import my foo library correctly using those imports statements - outside of the monorepo), I'm still facing an issue regarding the "src" folder when importing from my packages. After following this demo from @RyanCavanaugh, my intellisense offers me the following imports:
foo,
foo/src/components/Foo,
foo/src/components/Foo/Foo.component.tsx

Having src in the "nested" imports for me is problematic because it's going to raise issues when building / in production and I'd like to know if there'd be any way to "fix" the imports to not offer src as part of the intellisense.

My tsconfig at my packages/foo library is:

{
  "extends": "../tsconfig.settings.json",
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "ts"
  }
}

My tsconfig at my packages/baz library is:

{
  "extends": "../tsconfig.settings.json",
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "ts"
  },
  "references": [
    { "path": "../foo" }
  ]
}

My packages/tsconfig.json file:

{
  "files": [],
  "references": [
    { "path": "baz" },
    { "path": "foo" }
  ]
}

My packages/tsconfig.settings.json file:

{
  "compilerOptions": {
    "composite": true,
    "declaration": true,
    "emitDeclarationOnly": true,
    "declarationMap": true,
    "sourceMap": true,

    "module": "esnext",
    "target": "es5",
    "lib": ["es2020", "dom"],
    "jsx": "preserve",
    "moduleResolution": "node",
    "downlevelIteration": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "noErrorTruncation": false,
    "skipLibCheck": true
  }
}

The demonstration cleared 90% of my typescript issues when using a monorepo, thank you for that!

Question regarding reference/dependency association

I'm trying to get a simple POC going that somewhat mimics our monorepo and I'm running across a problem that I'm hoping someone can help with.

I set up everything seemingly like this example project, but when I run an npm i I get a message stating that my dependency can't be found. I can reproduce it by slightly modifying this repo...

If I change all the module names from @ryancavanaugh/* to @foo/* (the name of the module and the names of the deps) in the package.json files, I get a similar error...

I did a big find/replace to replace it with foo, go into pkg2, wipe out package-lock.json, and run npm i. I get npm ERR! 404 Not Found: @foo/pkg1@^3.0.2.

I can't tell if I'm doing something wrong or if I'm just missing something.

I really appreciate the repo though. It does seem to be a great example.