teleporthq/teleport-code-generators

A collection of code generators for modern JavaScript applications

teleport-code-generators's Issues

Ensure safe naming for component names

All names from the UIDL should be made "code friendly", i.e. valid for function / class names

ex: "My Component" - "MyComponent"

Rethink mapping logic for children elements

When we resolve a type in the UIDL with a node which also has children, there are some additional checks we perform, like trying to insert the original node children into the $children placeholder. Unfortunately, there are multiple use cases here to take into consideration and this replacements needs to be a bit more generic

• $attrs prefixed urls should also work deeper down the tree (videos are a good example)
• attributes which are passed to children should not be added in the parent (videos again)
• props and state should also be passed in a similar manner
• see #11, this may affect the use of $children and $attrs prefixes

Treat the input UIDL as an immutable data structure

There are a few points in the process of resolving the UIDL nodes where the input structure is being modified. We should avoid this, as it can lead to unexpected behavior/bugs when we work with more complex scenarios that involve multiple generators

Dynamic style - pros and cons

Not all styling libraries support dynamic styling when it's manipulated from css. How should we proceed with this from the UIDL / generators perspective? This is also related to how we handle dynamic styles in the playground

[proposal] allow for user custom eslint rules for code generation

It would be nice for the code generator to read custom formatting rules provided by the user and apply them to the generated code classes.

This could be an additional step to the code generation such as - prettier-eslint --write

for example styles in the generated code are not formatted correctly.

Project generators functions need some refactoring

There's a lot of duplication currently in the main functions of the project generators:
src/project-generators/react-basic/index.ts
src/project-generators/react-next/index.ts
src/project-generators/vue-basic/index.ts
src/project-generators/vue-nuxt/index.ts

Helper functions can be extracted for creating various types of files (ex: createJSfile, createCSSfile, etc.), or for creating folders (ex: createFolder) or for the dependencies merge then generator functions will be smaller.

React project generators should have priority for now, until #45 is solved, because Vue generators don't have the globals implementation yet

Cleanup component builder

The Builder module (https://github.com/teleporthq/teleport-code-generators/tree/master/src/component-generators/pipeline/builder) needs to be cleaned a bit. It has some functionality (namely related to slots) that was intended at first, but was not implemented all the way and was dropped

@vladnicula you should also have a look at this when it's done, since you worked most of the logic there

Replace $props and $state with proper JSON structure

In the UIDL if we want to address something from props, state, we rely on strings, hence the UIDL manipulation can seem a bit clunky (ex: $props.message, $state.isVisible)

We could talk about a JSON based structure for defining dynamic content (props, state, local vars - see #10)

A resolution on this should be made before the official release.

Deprecate createReactComponent

This is a special case of component generator which changes the style plugins at runtime. It's a good idea, but it needs a better API. Maybe some additional work later on.

Code is found here: https://github.com/teleporthq/teleport-code-generators/blob/master/src/component-generators/react/react-all.ts (bad naming...) and it is used only in the teleport-component-playground project

Describe layout component from UIDL

Currently we assume that the first node inside the root.content is describing the different routes/states of the application.

We may want to describe some sort of layout component around the main area where routes are deciding which content to show. For example, we could describe headers, footers, side navigations, etc.

Because each project flavor has specific ways of working with routes, this might be tricker to implement in one flow.

addMapping does not work for Vue Component Generator

While writing the first tests for the vue component generator, I noticed addMapping did not work. This needs another look, as it might have been a mistake, but I'm reporting it for now just to be sure.

jss version should be swiched when 10.0 is official

Currently we rely on an alpha version: https://github.com/cssinjs/jss/releases/tag/v10.0.0-alpha.10

UIDL and Generators Versioning

Couple of questions still up for discussion:

• is the schema URL enough for UIDL versioning?
• how are we connecting specific UIDL versions with specific generator versions?
• are we keeping generators in sync in terms of versioning?

A solution/resolution/decision on this is CRITICAL before the official release

Component generator standard interface

The component generators all expose an async function called generateComponent. This function runs the major parts of the component generator (resolver, assembly, builder) and returns a standard interface. This interface could be a bit better

Current Implementation

• code - string - the formatted main file (for react it's the .js file, for vue is the entire .vue file)
• externalCSS - string - any external css which the generator might compile (used only for css-modules at the moment)
• dependencies - Record<string, string> - all the external dependencies (of type: package) which are needed by the project generators for the package.json.

Proposals

• code and externalCSS can be exported as an array of outputs, with a file type or identifier next to them
• code can be exported as componentCode and externalCSS can be exported (together with other potential files) as an array of external files.
• dependencies could stay in the same way, but it could be null if no external dependencies are there. At the moment it is returning an empty object.

Revisit dependencies details

Is it enough to specify the version of the external (package) dependencies in the UIDL?
can we enforce that the same version is used through the UIDL?

Should we rely on fixed versions or on semver friendly syntax?

We should come to an agreement by the time we go ahead with the official release.

Cleanup chunk names

We have a slightly obscure at first logic for naming chunks, which is needed for connecting them at build time. This should be passed again through someone's eyes and properly documented. Also, default should be consistent across the plugins (ex: imports, imports-lib)

Also in doing this, we might consider the way in which plugins are exported

Template formatting for .vue files

The XML built template chunk of the vue file is not properly formatted. We should look into the prettier formatting plugins or find a different solution.

Refactor object concatenation in resolveNode

At the Resolver level, we use a lot of object concatenation for creating the final mapping object. The code could be optimized so that we don't concatenate empty objects if no param is given

Define JSON schema for the entire styles object

Currently the styles object is not defined inside the schema. Given that there are a lot of options and we have to take into consideration nesting with media queries and pseudo selectors.

We should discuss this before moving further, as it involves some work

Handling nested routes

At the moment, when doing project generation we only look at the first level routes inside the root component and we assume we have a flat hierarchy of routes in the application.

This might be a limitation we don't want moving further so we should at least start discussing about managing multiple levels of routing in projects generators.

The problem here is that each project flavor has particular ways of implementing routing so it will be tricky to come up with a single flow for all project generators.

Generating local variables in code

This might be useful as we work with repeat structures, but can also be a generic concept in the UIDL at some point. It needs some exploration and a proof of concept.

Ex: when generating a structure with a map iteration, the local variable that is inside the body of the function may be an object so we could address a specific field on the object.

Consider:

• XML and AST implementations may differ a lot here

error package.json name using yarn, illegal character space + uppercase

Yarn doesn't allow spaces or uppercase letters in project name. Should change the name to teleport-project maybe.

"name": "teleport project",

➜ dist git:(master) ✗ yarn install yarn install v1.9.4 error package.json: Name contains illegal characters info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

String does not match the pattern of "^(?:@[a-z0-9-~][a-z0-9-._~]*/)?[a-z0-9-~][a-z0-9-._~]*$".
The name of the package.

Project Templates - provide default package.json files for all generators

It's a longer discussion here, about projects that make sense without a template and projects that require some boilerplate code (webpack configs, babel, vue-cli, etc.)

Error handling + friendly messages

Some areas which need attention in terms of error handling and messages

• UIDL Validators (both for component and project) - definitely needs better messaging
• Component generators should output the potential errors from the UIDL validators (see #2)
• Project generators should output the potential errors from the UIDL validators (see #108)
• Uniform usage of try / catch blocks in sensitive areas

Open Questions:

• Are we using console.warn / console.error / etc ? (check other similar libs)
• See #41 if it can be connected

Component generators should validate input UIDL

First step in the generators, before any node resolve, should be a call to the UIDL validator

Validator functions are found in: /src/uid-definitions/validators

Concerns:

• validation is done using an external JSON schema validation library (ajv) which may be slightly big in terms of kb and it would not be tree-shaked if used on all component generator paths
• validation should be opt-out, via a configuration in the generator

Develop a cli for the component/project generators

Questions to answer:

• which would be the use cases for a cli?
• which library would help us to most with what we want?
• would the cli be a separate package?

Don't generate the props param if it is not used

Currently all React components are generated with (props) => in their signature. This should only be added if the component uses props

allow for custom UIDL json schema

We could think of allowing custom UIDL JSON schemas as well. It would be needed for custom integrations and/or custom code generators.

Extend the UIDL validator with business rules

UIDL validation is currently performed strictly on the JSON schema structure.

We could add a thin layer of validation for:

• prop that is being used without being defined in propDefinitions
• prop that is defined but not used in the content area
• state that is being used without being defined in stateDefinitions
• state that is defined but not used in the content area
• prop and state key are not the same? (this could work in React, but not in Vue)
• using local variables inside repeat:
  • using index without declaring useIndex in meta
  • using custom local variable name without specifying it in meta as iteratorName

Additionally for project generators:

• component is referenced but does not exist in the project
• component name is different than component key (in project UIDLs)
• external dependency version is consistent across the project
• route state key is defined inside root
• the first level children in the root component are conditionals based on route

Async generators can be made sync?

Initially we developed the component generators as async functions, but since the flow is sync, it might not be needed after all.

A resolution would be great before the official release

Generating ternary expressions

When doing conditional rendering based on a boolean state value, we can introduce ternary rendering in JSX:

<button onClick={() => setIsExpanded(!isExpanded)}>
  {isExpanded && "Collapse"}
  {!isExpanded && "Expand"}
</button>

can be re-written as

<button onClick={() => setIsExpanded(!isExpanded)}>
  {isExpanded ? "Collapse" : "Expand"}
</button>

Remove unused areas from .vue files

<style> should not be added in the vue file if it's empty. the check should also be done for the <script> in case that's possible

Perform state transition by changing styles/classes

This was already debated a bit inside the team. For now we only support state changes by rendering/not rendering elements, not be toggling classes on/off. But if we want to support animations, we should reopen the topic at some point

Consider adding optional chaining operator in generators code (not generated code)

We could consider adding the optional spread operator in the generator code. We could benefit from it a little bit as it allows us to write more or less the same code in less lines but without sacrificing readability in my opinion.

Link to tc39 proposal: https://github.com/tc39/proposal-optional-chaining

Link to typescript discussions about it: microsoft/TypeScript#16

The main target of this exploration would be to see if we can use optional-chaining now, maybe via a babel plugin that runs before typescript in the babel pipeline.

Extract external dependencies inside component generators

Currently component generators returns all the dependencies collected as the nodes are being parsed (library, local, package). But as an output, we only need the package type, because that is used when generating the package.json in the project generators.

The extractExternalDependencies function is already defined in the project generators, it can be safely moved to the component generators

Custom paths for components defined in meta

Each component could have defined in its meta object a custom path where it should sit when the project is generated. This needs to be orchestrated together with handling dependencies from one component to another, so that we know how to generate the local imports.

For example, you may want to have some specific button components in a /buttons folder and be imported from there.

-- components
-- -- buttons
-- -- -- PrimaryButton.jsx
-- -- -- DefaultButton.jsx
-- -- Grid.jsx
-- -- Card.jsx
-- pages

Not sure if this is mandatory for the official release, it might be a later optimization/enhancement

Integrate snyk

Snyk is free for open source projects. Liran Tal will be very happy :)

Refactor static and dynamic attributes and text nodes

So far this is a bit tied to using props or state, but we have to think generally for using any variable in attributes and as text nodes.

We currently address props and state in the UIDl with $props.[propKey] and $state.[stateKey] but we do not treat objects and the possibility of local variables (see #10)

Style plugins are not handling dynamic values in nested styles

Most of the style plugins are only handling the first level keys in the styles object. This means that they will not replace dynamic $props. expressions inside media queries or hover expressions. The whole dyamic styles topic should be considered because not all styling systems support that out of the box

Passing down classes as props

When you pass a class as a prop, there's a question if the root node of that class should inherit that class. Vue seems to behave in a way that this automatically happens. In React, the class needs to be added manually, but we're not supporting multiple classes on one node yet.

A resolution for this should be set before the official release

Add typings for chunks

One of the last parts of the generators which is not typed.

As a broader task: we should restrict the usage of any at all levels in the generators

Events mapping

Event keys defined in the UIDL are currently used as they are. This means that a UIDL defining events for React ("onClick") will not work on Vue (@click). We need to define the events in an abstract way and map them to the corresponding framework, similar with how we map/resolve tags

On a related note, we might need to also map additional attributes from html to specific frameworks (ex: autoplay - autoPlay, for - htmlFor)

Handling events at the top level inside components

This refers to events we might want to handle in component lifecycle hooks, like catching a scroll event.

Might be a nice addition to the UIDL structure as well as a nice capability for describing more complex components (ex: header that changes style as the user scrolls)

Bring Vue generators on par with the React one

We developed some features only for the React generators in the past month, including state management and event handlers. The UIDL has all the functionality, so we need to update the Vue generators with all the new features:

• state handling (state definition and conditional rendering)
• event handlers
• repeat structures
• children props
• revisit routing
• globals and manifest
• assets prefix
• style plugin (dynamic style)
• performance tests and optimization

Combine state keys in a single conditional render instruction

It might be useful at some point to render something based on two different state keys. Currently this could be done in theory, but in a hackish way with nested states.

Since we have the capability of describing different operations when rendering based on a state value, we could also introduce the state keys in the mix so that a single rendering instructions can be conditioned based on two different state values

React ex:

{ isVisible && isReady && <div>...</div> }

For vue, we have to implement state handling first, then figure this out as well

Handling boolean attributes in XML chunks

Boolean values are not properly transformed into html attributes

ex: disabled: true becomes disabled="true", not simply disabled

Using lerna for organizing the repo

At some point we want to split the packages on npm, so we might consider using lerna https://github.com/lerna/lerna for organizing the repo. We need to discuss this and have a proof of concept before the official release if we want to push this way

Updates:

Meeting notes from March 27th

Attendees: @Utwo, @vladnicula, @anamariaoros, @alexnm

Discussion Topics:

• monorepo - yes/no
• splitting the npm packages
• what we publish on npm for each package
• project-packer
• lerna vs yarn workspaces vs other solution
• tools for transpiling to dist

Action Items

• we keep a single monorepo for all the generator code
• we move uidl-definitions inside the core module
• monorepo will include project-packer with all it's separate packages
• current proposal for npm packages (all will be namespaced with @teleporthq
  • teleport-generator-core
  • teleport-generator-shared or teleport-generator-lib (not sure yet)
  • teleport-component-generator-[react/vue/etc,]
  • teleport-project-generator-[react/vue/next/nuxt/etc]
  • teleport-plugin-[react-base-component/vue-base-component/react-styled-jsx/etc]
  • teleport-project-packer
  • teleport-project-publisher-[zip/now/git/disk/etc]
• typings folder remains in root and is shared by all packages
• we publish everything on npm (src, dist, typings, etc.) - prefer developer experience over package size
• UIDL version remains in sync with the core package
• UIDL schema remains next to the core package for development purposes (ex: validation), but is also available remotely via a static website for integrations and JSON editors
• we will adopt a RFCs style discussion on github for all major changes to the UIDL / generators, after version 1.0

Open questions

• Do we use lerna or yarn-workspaces or both or something else?
• Do we keep certain packages in sync in terms of versions?
• How do we work with the git tag vs npm versions? Also, how does this affect the CI/CD pipeline?
• What we use for build/transpile? (targeting: esm, common.js, minified?)

References

• https://blog.logrocket.com/setting-up-a-monorepo-with-lerna-for-a-typescript-project-b6a81fe8e4f8
• https://yarnpkg.com/lang/en/docs/workspaces/
• https://github.com/reactjs/rfcs
• https://github.com/pikapkg/web

Use enum values in JSON Schema

Some of the fields that we are restricting as strings can be expressed by enums for better suggestions and better validation rules:

• globals.assets.type (link, script, style, etc.)
• globals.manifest - multiple fields (check specification)
• dependency.type (local, package, library)
• propDefinitions.type
• stateDefinitions.type

Custom metadata for each page in the UIDL

Problem

At some point we would probably want to define custom metadata (title, description, keywords) for each individual page from the playground. So we need support in the UIDL to override the default metadata and also we need the implementation in the generators.

Solution

This problem will require changes in both the UIDL and in the architecture of project/component generators.

UIDL

We should support meta tags for each stateKey inside the route stateDefinition.

"route": {
  "type": "string",
  "defaultValue": "index",
  "values": [
    {
      "value": "index",
      "meta": {
        "path": "/",
        "componentName": "Home"
        <<< meta values here >>>
      }
    },
    {
      "value": "about",
      "meta": {
        "path": "/about",
        "componentName": "About"
        <<< meta values here >>>
      }
    },
    {
      "value": "contact-us",
      "meta": {
        "path": "/here-we-are",
        "componentName": "Us"
        <<< meta values here >>>
      }
    }
  ]
}

Open Questions

• Which values do we want to support in the first phase?
• Does it make sense to support assets (eg: link, script, style) or just title/meta stuff?

Component Generators

Once we have the UIDL values, we need to pass them in the component generator that handles the pages of the application. That generator should be enhanced with a plugin that handles this part of the UIDL and modifies the AST / HAST structures to support this feature.

• For React, we can use react-helmet and render the corresponding tags inside the JSX node
• For Preact, we can use preact-helmet with a similar behavior
• Next.js has it's own next/head implementation
• For Vue, we can use vue-meta for both vue/nuxt

Open Questions

• Will we add a plugin for each new framework?
• How does SSR work with all of these implementations, is there something specific we need to configure in the project setups?

Revisit PropTypes handling

PropTypes are not too flexible at the moment

• each prop requires a default value to be displayed
• we don't have the ability to set more complex default values (ex: object, array)
• children props don't have a clear definition/standard (also check slots in vue)