practicajs/practica

Node.js solution starter boilerplate that is production-ready, packed with ✅ best practices and built with simplicity in mind

practica's Issues

Enrich the test reporters with 2 cool and useful tools

🎯 Goal: - Boost the developer testing experience with reporters that show useful info: The slowest tests and those that were skipped. This might lead to an action by the viewer

🤔 More details:

• Jest slow test reporter
• Jest skipped test reporter
• Remember to test it, trigger a slow test and a skipped one

p.s. Share a picture here of the reports when done?

Decision: What is the right configuration library for a project?

Please share your thoughts and opinion, remember to mind the vendor choose principles.

The supporting decision table is here

📗 Hardened docker file - great article to follow and learn from 🗞

🎯 Goal: - Allow building a hardened docker image out of this code so it can be deployed to Docker based productions (e.g., Kubernetes)

🤔 Things to consider:

• It should be secured (e.g., avoid leaking secrets)
• Support development mode (i.e., debug image locally)
• Reasonably sized
• Receive signals from OS
• Conform to any other known best practice. @lirantal published a great checklist to follow!. I already wrote an article on this matter, I find Liran's version to be better:)
• Should be part of the example Microservice and not the repo root
• Test it. How? It's interesting to read and hunt techniques. At the basic level, the generated image should be inspected with tools like dive, should be executed manually and ensure that signals are received and more. Can we write automated tests for this? Maybe an option to consider

Create a basic Docusaurus doc website (based on GitHub)

Based on the chosen technology, create a skeleton app with the default features including:

• Our logo
• Title
• Sensible first TOC
• Domain
• Sync with GitHub
• Others?

🤖 Make the CI run against multiple versions of Node (Matrix)

🎯 Goal: - Currently our CI runs against a specific Node.js version (e.g., the one within .nvmrc), since we should support also other Node.js versions like 12 & 14, it stands to reason to run the CI with all the supported versions of Node. GitHub actions support this out of the box

🤔 More details:

• Github actions support this with the matrix feature
• Test it - try temporarily embedding a feature that is supported only in Node.js 16, the CI should fail. Did it?
• Bonus: Besides Linux, put 'windows' OS as well in the matrix testing. It might fail, it's OK for now, at least our CI is ready

Show CLI spinner while we generate the code

🎯 Goal: - When the non-interactive CLI command is invoked, we should show some friendly spinner and message until the code is generated. When it's ready, it could be nice to show some nice message like "All set, the code is ready for you with tons of good practices inside"

Here is how our UI looks like (no emoji, no text):

Try it yourself with this command:

npx @practica/create-node-app immediate

Here how cool it could be (taken from nx UI):

🤔 Things to consider:

• There are plenty of tools that can achieve this easily like this one
• It should be done in the code under ~/src/code-generator/entry-points/non-interactive-cli.tsx. You may just await on the main function that generates code, while we await we can show spinner and then a success message
• Optional bonus: Run E2E to ensure it's there

Create a PG migration

Based on the chosen tooling, define all tables and schema via a migration file

Write health library that tells when the server is in panic

🎯 Goal: - Write a library that is being initialized by the app which will expose endpoint telling whether the app is read or not. This is needed for the runtime to determine whether to kill this instance. Also useful for "ops" visits to diagnose issues

✅ Requirements:

• Should be implemented as library in ./src/code-templates/libraries
• Should expose an endpoint that returns 200 if all good and 500 if the component is facing severe issues. See Kubernetes guide as an example
• How to determine a bad state? This can lead to a philosophical discussion, for a start if the DB+API are responsive - It seems OK. Have other ideas for simple measures?
• Should check healthiness also on start time so the process immediately crashes
• Should log+fire metric when something goes wrong

✅ Requirements:

• ⚙️ Tech considerations

• Soon

Create a Postgres DB/Tables/Migration for new orders

Based on the testing practices repo, put the DAL code with enough fields to demonstrate

Core tasks:

• Create 2-3 tables with joins
• Use at least 1 JSON column
• Create migration
• Create a singleton that holds the connection pool reference

Core decisions:

• ORM or not? which?
• Which migration framework (if not ORM)?

Decision: What technology should power our documentation site?

We need a documentation site, which technology should be used (e.g., Docusaurus)?

Don't let new commits decrease the coverage

🎯 Goal: - 100% coverage or not? Big question, at least we can disallow new commits to decrease the existing coverage

🤔 Things to consider:

• codecov seems like a good tool for the mission. Feel free to consider other options
• Should be a standard PR check - This way all the quality checks are centralized in one place
• No absolute numbers rather per-commit measurement - It's hard to declare now which exact coverage number we aim for. Instead, we should define that the latest commit did not decrease comparing with the previous one (achievable with configuration only using codecov)
• Other informative and 'cool' visuals that can be added to the PR are welcome
• If the tool has other useful capabilities - bring it on!

What is the right OpenAPI tooling and approach with express?

Since both Fastify and Nest have a built-in and recommended approach, this discussion is focused on what are the right approach and tooling to implement OpenAPI with Express?

The corresponding decision table is here:

https://github.com/practicajs/practica/blob/main/docusaurus/docs/decisions/openapi.md

✨📗 Quick learning: interesting things that are happening now🗓✨

✨🗓 A selection of opportunities for contribution and learning✨

📰 Note: You can start quickly by reading the shortened contribution guide, only 2 paragraphs here. Or the comprehensive one

📗 Have you ever configured and wrapped a logger?

Our logging library is doing nothing but console.log now 😞. We need to pick our preferred npm logger (remember to include a comparison table) and get it inside. There is no need to understand the entire project, the logger is a self-contained isolated package - You can test it without the rest

🎬 Action item - The corresponding issue

📗 A chance to setup your first (or second) GitHub bot

Bots can handle lots of messy work for us like this bot that closes old issues or that one that warns when packages contain vulnerabilities. See here a full list of potential bots, take one or few.

Free T-shirttt! - Our cool t-shirt to whomever handles this task 👚👕

🎬 Action items - See all related issues

📗 Refresh your knowledge on how to manage configuration

Although most projects use dotenv for reading configuration, we thoughtfully use another library, convict. All is comprehensively rationalized here. Interested to understand why and consider it to your own project? Read the comparison table here, go through the code (30 lines only). Learn something new today!

🎬 Action items - Visit the configuration package

Generate the code with a templating library

🎯 Goal: - Currently we generate code by just copying the code-templates folder. We need to run the code files through a templating library so it can be modified based on the user requests. In the future, this will rich logic (e.g., change web framework, avoid requiring files), for now we wish to start with a small step

✅Requirements:

• Add new CLI flag that we accept: --emit-best-practices (file: ./src/code-generator/cli-entry-point.ts). The code contains sections with 'best practices comments' that highlight good practices in the code. The user might want or avoid these parts in the code (see the file global-setup,. line 22)
• Choose a templating library like handlebars or other. Ensure to pick a very popular one
• Wrap these 'Best Practices' parts in the code with a conditional expression that your templating library provide. Example with handlebar
• Try generating code w/o this flag and ensure the output code is as expected
• Write tests against this new functionality. Too complex for now? You may defer this task or just ask for advice

Add a CLI step which asks if should init the app with a git repo

Configure a base linter + custom useful plugins

🎯 Goal: - Catch significant issues early using the base known linters and other remarkable extension

🤔 Things to consider:

• There are a handful of eslint extensions, we want to pick those who are really valuable and not anything
• No cosmetics, we have prettier ones for this. Focus on production bugs (e.g., a promise never resolves)
• Remember that they are needed twice: Root and inside the generated project
• Some examples, not necessarily each one if valuable and should be added. We trust your judgment: 💪

• eslint-plugin-jest
• eslint-plugin-node
• eslint-plugin-security
• An article to read:
  https://blog.ghaiklor.com/2021/07/21/top-5-eslint-plugins-for-typescript-development/

[root project] Enforce commit message standards

When committing with standards we can later on auto generate versions and change log.

*If it will be convenient for this project we can add it to the templates as well

Configure prettier

Put a standard prettier setup

Copy the testing code from nodejs-testing repo to here

We have modern integration testing patterns implemented at the repo:

https://github.com/testjavascript/nodejs-integration-tests-best-practices

Copy the basics thoughtfully heer

Create CI (GitHub actions) for the generated code

🎯 Goal: - Currently we have a CI only for the root project, the root, we obviously need to include one with the generated code. For a start using GitHub actions, in the future we might add more vendors. This CI should include few typical steps like testing, linting and vulnerabilities checker

🤔 Things to consider:

- The basics - Add a simple GitHub action workflow as described in this article, we actually have one in the root of our project. For a start, we surely want it to run the tests (already exists in every package and Microservice)

• Where - In the code templates folder where the generated solution is
• How to test - The workflow that you're adding is not under the root, so it won't run for this repository. How can we ensure that it works? This is one of the challenges in this task
• Monorepo challenges - The generated code is not a single component rather multiple that are linked in a monorepo (Lerna). What exactly the CI should execute? This worth a discussion, as a simple first step it might run the tests of all sons:

lerna command test

Run spell-checker against our README

🎯 Goal: - Remove embarrassing grammar/typos mistakes from our docs 🤭

🤔 How:

• Open the readme
• Install some Chrome spell checker extension
• Fix some issues. No need to fix everything
• PR 💚

Configure a vulnerabilities checker

🎯 Goal: - Detect when some code has vulnerabilities or when a developer is fetching a suspicious package

🤔 More info:

• Tools like sync alike can provide E2E CVE shield free for OSS projects
• Run as part of CI
• Beyond just realizing CVE (vulnerabilities), they have advanced capabilities of warning/PR when dependencies are outdates

🤖 Configure stale issues bot that closes old issues

🎯 Goal: - When an issue is been idle for too long, then a label is applied. After some additional time it will get closed

Stale bot seems like a great tool for the mission, are there any other options?

🔐 JWT validator library

🎯 Goal: - Protect routes by verifying the JWT token and stopping requests if the token is invalid. Later we will add JWKS later and authorization...

🤔 Things to consider:

• Should be its own library with its own tests
• Should be a middleware so every router/route pick it or not
• Create also a public non-protected route to demonstrate how to bypass the test
• Follow as much best practice that you can find while still keeping the code short. No worries if the 1st iteration is simplistic, all good

Name for this project?

Propose here adequate names for this project? Finally we will run a poll through all the stakeholders and choose

Note: The name should reflect the project's philosophy: Robust AND simple.

Docusaurus images are not being referenced in the right way

When looking at Docusaurus content with images, like the decisions table, the images are missing

This guide explains why:

https://docusaurus.io/docs/static-assets

Interactive mode CLI select list does not allow setting color

🎯 Goal: - The CLI interactive mode has multiple select lists (e.g. choose framework), when hovering an item the color is always blue which is not part of our coloring schema. Ideally, it should be white. The vendor lib, ink-select-input, does not allow passing a prop with the color. We should submit a PR ourselves to make 'em support this

🤔 Things to consider:

• Consider opening an issue first
• Link to repo

📗 Learning: Do the ESM kata

We need to migrate our repo to ESM (JS modules), this is an opportunity for you to greatly learn about this fundamental technique. This should be done in two steps: At first adjust the syntax and remove all 'require' from the code, then actually make tsc transpile to ESM code

❣️ Why is this important: The new default soon in Node.js, new syntax, the standard JS mechanism that works in browsers as well. Under the hood it is doing things differently that must be understood to plan and troubleshoot modules

✅ Task:

• Replace all the CJS code with ESM (from 'require' to 'import')
• Ensure test mocking/stubbing works with ESM
• Some minor code paths might need to be changed. For example, __dir is not working with ESM and has an equivalant alternative
• Since this has high impact on the system, let's go with two phases:
  A. Syntax change - Replace all to 'import', whenever using __dir, put its value into a variable so later in the future we can replace the variable value with import.meta

⏰ Busy? Do just a small part: Replace only 2 files and tests

📗 Learning it from A-Z:

1️⃣ Become acquaintance with the basic syntax
2️⃣ Read a quick start guide, try first code edit s
3️⃣ Watch a great YouTube that gets into depth by Gil Tayar
4️⃣ Understand all the pieces by reading the official docs

Make a library 'publishable'

🎯 Goal: - The code we generate contains multiple libraries, let's show how one can get published to npm so the main Microservice can install and use it. The rest will follow.

🤔 Things to consider:

• Before publishing the code should transpile (build), version bump should happen (we can automate this later) and then publish to npm. Consider chaining all of these commands under one 'npm script' in the library package.json
• Ensure .npmignore exists to avoid leaking secrets or none-relevant code
• Test that the main Microservice can consume this library from the remote registry (currently it's linked locally)

Documentation decision tables format

@relsteiner The origin decision table was created intentionally with pure HTML to gain full control on its visibility. I noted that it was changed to markdown which is more limiting. See the differences between:

Markdown

HTML

Can Docusaurus support the HTML version? Can we omit the Docusaurus metadata header so visitors could read in GitHub?

Create a logger library - No need to understand the entire system

🎯 Goal: - Create a logger library that wraps reputable industry logger and add additional common features

🤔 Requirements:

• Choose a reputable industry logger, Pino and Winston seem to be on the top. Read the vendor choosing guide
• Add log levels - Based on the current configuration, write to log only if the configuration.log-level instructs so (e.g. info, error)
• Enable pretty print for beautiful terminal messages during development
• Hunt for other great logger practices
• Write a few tests against it
• Optionally: include a comparison table that explains why you preferred this specific logger lib

⚙️Tech instructions:

• There is already a logger library ./code-templates/libraries/logger, it's just empty
• Remember to also run the generated solution tests to ensure nothing broke
• Consider it assigning a scoped name @practicajs/logger and publish to npm

Write a superb error handling module

🎯 Goal: - Handle errors in the application in a robust way so the users highly respect the error handling approach

✅Requirements:

- Isolates tests - Write tests against all of these scenarios. Here are some nice examples. Those test belong to the error handler lib #iteration1

-Corner cases -

• Ensure any kind of thrower is covered: Web-request, startup failure, undhandled, others #iteration1
• Ensure any kind of error is covered: AppError, Error, string, emoji of duck 🐥, etc #iteration1
• On unknown error (!AppError), ensure it's handled + same Error structure is preserved which includes .message, .name, .stack #iteration1
• Handle also sigterm, sigint
• Ensure errors are being handled: logging, crash decision, response to user if a web request. Maybe also fire metric
• Start failures, should be explicitly caught and reported as non-trusted
• Error handler supports option to allow overrides

- Microservice test-

• Write also 1 or 2 tests against the Microservice that stretches the error handler. For example, try to add new order, mock an error, ensure logging was called with the right param and the user got the right HTTP status

🤔 Things to consider:

• Chunk into iterations. For example, we can have a simplistic error handling in few hours as a 1st iteration. Then the 2nd iteration could be about one more feature.
• The error handler may register the process level events (unhandled, ...)
• The error handler should log using the logger lib (not console)
• Later on, in future iterations, it may also handle process exit signal and shut-off the webserver
• The less amount of code we write, the more likely users feel comfortable with it (not another huge module to maintain)

Autocomplete tests name in watch mode

🎯 Goal: - Slightly improve the testing experience by introducing 2 useful watch plugins that are presented to the developer while running testing in watch mode (cli)

🤔 More details:

• Typeahead plugin will allow searching and filtering for a specific test
  https://github.com/jest-community/jest-watch-typeahead

• Suspend plugin will allow pausing the 'test noise' when a developer know that the test won't pass for a while
  https://github.com/unional/jest-watch-suspend

p.s. Share pictures here when it's set up?

Enhance the non-interactive CLI generator

🎯 Goal: - This is about when the user chooses to generate code with CLI flags, not the CLI UI that we have. Though there is some initial POC, the non-interactive CLI is barely functional and we wish to complete a whole flow where the user gets a decent CLI experience and code is generated

✅ Requirements

• Get basic flags from the user like --overrideIfExists
• If the destination folder is not empty and --overrideIfExists=false, quit with a meaningful error
• If an unknown error occurred, show a meaningful error
• After generating code, show success message to the user + some tips on how to start with the code
• There are plenty of libraries to generate nice texts and icons so the messages look good (e.g., chalk, log-symbols and more)
• Bring on any other good terminal practice or DX. @lirantal has a great article about this here
• Show progress bar while the code is being generated
• Consider showing our logo as well (can be found in the repo's docs/images)
• Write E2E and component tests against the new functionality. Feel free to ask if not sure how or defer this task and discuss later

🤔 Things to consider:

• The code is located under ./src/code-generator

🤖 Make the CI run against multiple versions of Node (Matrix)

🎯 Goal: - Currently our CI runs against a specific Node.js version (e.g., the one within .nvmrc), since we should support also other Node.js versions like 12 & 14, it stands to reason to run the CI with all the supported versions of Node. GitHub actions support this out of the box

🤔 More details:

• Github actions support this with the matrix feature
• Test it - try temporarily embedding a feature that is supported only in Node.js 16, the CI should fail. Did it?
• Bonus: Besides Linux, put 'windows' OS as well in the matrix testing. It might fail, it's OK for now, at least our CI is ready

🤖 Delete merged branch bot

🎯 Goal: - When a branch was merged, why keep it alive and mess our branches list... Why spend human being time on such a task, let the bot work for us! This bot can take care for this automatically, let's invite her to the party

🤔 Things to consider:

• If you need permissions for the installation phase, just ask and it will happen quickly
• Test it!:) Mind both when the branch should be deleted and also when it's not (CI fails)

Compare various config libraries

Compare the 3 most popular libraries that tick all the best practices boxes:

• Hierarchic config
• Fail fast
• 3 most popular
• Type validation
• Document themselves
• Cascading value obtain (env var, file, static)

Unify and enhance our libraries configuration

Concern - We have a bunch of libraries like logger, auth, and more. Currently, they hold different configuration/structure and also lack some fundamental standards, let's nail this?

The list:

• Make 'em have the same set of package.json scripts (e.g., test, build, dev, etc)
• Create index file for every lib that is solely an interface to public functions/modules
• Create simple title-only README per package and make package.json link to this README
• Generate d.ts as part of the build and include this in package.json
• Put code under "lib" folder
• Totally optional for now: Make 'em share Jest and TS config
• Totally optional for now: Use a publishing tool like np to prevent silly mistakes

Prevent commits that violates quality standards using git hooks (Husky)

🎯 Goal: As a developer, I will not mistakenly send code with linting/testing/CVE issues

🤔 Requirements:

• Configure GitHook that runs all quality tools and stops the commit upon failure. Husky is quite popular tool for this mission
• Put in both the generator project (root) and the generated app (code templates)
• Allow by-pass, we always have CI as last the safety net
• Make it active on developers machine automatically as they 'npm install' their code
• For now, we have tests only, so husky should ensure that 'npm test' passes. Feel free to configure linter

Add support for images in dark mode

Images are not displayed properly when dark mode is used, fortunately, GitHub added support for light/dark based images.

It's available by using the prefers-color-scheme in the picture element.

More info here

Light:

Dark:

Images from CONTRIBUTING.md file

Support OpenAPI (swagger)

🎯 Goal: - Auto-generate OpenAPI for routes so developers can benefit all the advantages of Swagger

🤔 Things to consider:

• Our first version is about express, more challenging
• As long as the developer sticks to the coding convention, she gets everything generated automatically for free
• Get as much typing/details as possible. If doable, not only the routes but also the types (we're doing TypeScript)
• As always, based on a reputable and mature library
• As always, keep it simple. Aim to avoid disrupting developer's workflow

Can't render image

Our interactive wizard is based on cli React framework ink. We badly need to show our logo within the cli UI but since ink3, the image component was broken

It could be awesome if we find out why and fix it

@rluvaton Not sure if you have time now, this is a task for 007. Not urgent, we can process also next month

Display eslint/prettier errors visually inside GitHub PR files

🎯 Goal: - When a build fails due to linting error, as a developer I can see visually where exactly the error is in the code

More details here

@itai-autodesk Maybe this one will appeal to Itai?

Add .editorconfig to sync across editors

Add a standard .editorconfig so all the popular IDEs will share standards. Whatever can be defined in .prettier should stay there, it meant to sync values that are not handled by prettier

Don't let new commits decrease the coverage

🎯 Goal: - 100% coverage or not? Big question, at least we can disallow new commits to decrease the existing coverage

🤔 Things to consider:

• codecov seems like a good tool for the mission. Feel free to consider other options
• Should be a standard PR check - This way all the quality checks are centralized in one place
• No absolute numbers rather per-commit measurement - It's hard to declare now which exact coverage number we aim for. Instead, we should define that the latest commit did not decrease comparing with the previous one (achievable with configuration only using codecov)
• Other informative and 'cool' visuals that can be added to the PR are welcome
• If the tool has other useful capabilities - bring it on!

🤖 Configure a tool that detects leaked secrets

🎯 Goal: - If intentionally a developer who is using our starter leaks a secret into the code, the CI/git-hook should catch this early and fail.

🤔 Things to consider:

• It's useful to also allow local detection of leaked secrets (e.g., exposed npm script)
• There are a handful of tools for this, for example this wrapper by @lirantal. As always, evaluate all the options
• Ensure to put this not only for the root CI, but mostly for the generated code CI

turn on clearMocks in jest conf

I think we should turn on clearMocks config in jest as otherwise, you can forget to clean mocks and it will affect your next tests

WDYT?

Which is the right Monorepo tool?

Triggering a discussion about Monorepo tools

🔖 See GitHub test reports with files visual (annotations)

🎯 Goal: - When a test or two fail, it can handy to see right within the checks tab which test failed and what is the error. Otherwise, one would need to navigate away to test report or skim deep in the CI log

⚙️ More details:

• The GitHub action jest-annotations-coverage can do this for us. Maybe run also a quick check whether there are other better alternative tools
• There is no need to get a coverage report since we will use a dedicated tool for this

🤖 Request more info bot

🎯 Goal: - Having a bot working for us when someone is opening a PR without sufficient details, the bot will ask for more info and prevent wasting the maintainer's time

🤔 Things to consider:

• If you need permissions for the installation phase, just ask and it will happen quickly
• Phrase a kind message to the opener who was just mistaken
• Test it!:)