Token types - optional or required

[kevinmpowell]

Token types - optional or required

[c1rrus]

It may be worth pointing out that in the current draft spec, the type property itself is optional. But, if you omit it, the type of that token will be inferred as follows:

If the type property is absent, tools MUST treat values as one of the basic JSON types and not attempt to infer any other type from the value.

So, a tool that supports this format is required to interpret things as follows:

{
  "token with type": {
    "value": "#ff0000",
    "type": "color"
    // This is a color token becuase we've set the type
  },
  "token without type": {
    "value": "#00ff00"
    // This is a string token because no type
    // property is present
  }
}

In other words, every token in the file does have an unambiguous type (which, as others have already pointed out, is important for token files to be interoperable between different kinds of tools).

However, the basic JSON types (string, number, boolean, etc.) probably aren't all that useful in the context of a design system (except perhaps numbers, which could be used for line-heights, aspect ratios, as inputs to modular scales, etc.). In practice I therefore expect most tokens will include a type property in order to declare which of the DTCG types they are (color, dimension, font name, etc.).

For manually authored token files, typing out "type": "..." lots of times may get tedious. However, I think the idea we've been discussing in the DTCG format team of allowing the type property on groups would alleviate that. This discussion thread has made me more convinced that we should add it to the spec.

The way it would work is that a token's type is determined as follows:

1. If the token has a type property, then the token is of that type.
2. Otherwise, step up the parent groups until you find the nearest one with a type property and use that.
3. Otherwise, if you reach the root group and no type property was encountered, then the type of the token is the JSON type of its value.

So folks wanting to have files that only contain one kind of token (e.g. colors.tokens.json) could easily do so by just adding "type": "color" to the top-level. Hopefully that won't be too onerous. :-)

[dbanksdesign]

Design tokens are always typed, explicitly or implicitly. Style Dictionary for example doesn't care how you type your tokens, but to transform tokens you need to tell it which transforms should apply to which tokens. The built-in transforms implicitly type tokens based on the token object structure, but that is not the only way to do it, you could explicitly add attributes to each token to type them to be targeted by specific transforms.

One thing the format module editors have not tackled fully is separate/multiple files, which would be a necessity for a feature like 'all tokens in this file are of type color'. There were some very initial conversations around having some data inherit from a token group like so:

{
  "brand": {
    "type": "color",
    // type gets applied to all tokens in this token group
    "primary": { "value": "#ff9900" },
    "secondary": { "value": "#00ff00" }
  }
}

I would still consider this explicit typing because the user is specifying the types of tokens in the token file. The case for explicit typing in the design tokens spec would be to simplify implementations for library maintainers and tool makers. Without explicit typing, the specification would need to define how libraries and tools should infer design token types based on their value, which can get very complex. We have talked about not enforcing any kind of naming or grouping structure, therefore types could not be inferred based on name or group.

Another use-case to consider is component tokens. Having a colors.json file with all colors makes sense, but if I have a button.json tokens file, some sort of typing is required for each token in that file.

The design token spec should define how tools understand the types of tokens so they can do something meaningful with them. Figma needs to know what a color is versus a padding or border width. In my mind doing this explicitly will lead to less edge cases and potential bugs/discrepancies between tools. However I do see it being more cumbersome than implicit typing, but I would much rather leave that implicit stuff to tool-makers.

I have an analogy that might help as well. I think of the token spec like the CSS spec. Many people write plain CSS just fine. Some reach to something Sass to improve the authoring experience, but it ultimately compiles to CSS. Libraries like Theo and Style Dictionary would be like Sass. Theo and Style Dictionary could output a valid and explicitly typed design token file from a syntax that infers design token types. Or conceivably Theo and Style Dictionary could understand a design token spec compliant file as well (like how SCSS is a superset of CSS).

@blackfalcon there is nothing in Style Dictionary that applies attributes to all tokens in a file or in a group, but that could be achieved in a few ways. One would be to use a custom parser and modify the file's contents as it comes in to Style Dictionary. The other would be to use JS modules as the token source and do that application with a JS function.

[nhoizey]

I would vote for optional, as I currently prefer using one file per token type, such as color.json, so repeating the same type for every token would not be ideal.

[kevinmpowell]

Reviewed by the spec editors on 2021-10-19.

Decision
Token Types are not required to be manually set, but all tokens have a default JSON type. To gain the most interoperability and features from using the token spec, setting a type on each token is recommended. Vendor tools will not be required to parse tokens to infer types, so vendor tool functionality may be limited if types are not provided.

Decision
The spec should support a type property at the file & group level so a type property does not need to be manually added to every token. Follow #72 for additional details.

Action to be taken
Next version of specification will reflect these decisions.

[blackfalcon]

Not having types would mean each tool will have to parse and type-check themselves

I disagree. Why do I have to meet a specification for a tool I may not be using? If I am using it, there would be a requirement from that tool to have the type defined. That's my only motivation to do so, otherwise, I fall in the same camp as @nhoizey.

[kevinmpowell]

Relevant to this conversation, I've created #72 to discuss file & group level properties.

[mattfelten]

I don't like requiring maintainers to do more work to keep their tokens working so my initial thought was optional.

Requiring it now will allow consuming tools more consistent and easier time implementing tokens, which to me is a big win. Not having types would mean each tool will have to parse and type-check themselves which could lead to inconsistent results tool to tool.

My vote is required.

[jina]

note, there are tools like Theo that let you set a type globally for all the tokens in a file, so it doesn't have to be on each individual token. if we allowed that to fulfill the requirement, would that help? I think types are pretty important.

[blackfalcon]

The question is, is the enforcement of the type for the betterment of the library or for the ease of integration of a 3rd party tool? Or maybe better said, is this still useable without the mandatory type? While I agree typing can be helpful, what is the driver for this?

Unless there is a direct benefit to the lib itself, making this required is inappropriate IMHO.

[jina]

@blackfalcon I'm not 100% sure what you mean by "the lib" (the token output? the framework you're importing them in?)

As aligned with our mission objectives, we want to be extensible. Design tokens are also meant to be agnostic so that they can be usable by various libs and tools. Part of our goal is to give design tools/vendors a way to integrate tokens in a predictable way that makes sense and has agreement with the community. This was a big motivator in why we established this community group in the first place. Types seem necessary to make this work.

And in terms of the "lib", if you mean your design system codebase, how are you documenting that? I find that types being required help display tokens in the appropriate way in the docs (colors vs units vs whatever else). Additionally, when using JS/TypeScript or other languages beyond CSS/Sass, I've found it can be very useful for linting/testing/validation (which hopefully you're doing!) and testing certain attributes for accessibility.

I think from a standards perspective, as long as you can define them globally so you don't have to do each individual token (which seemed to be one of your concerns in your original comment), that this shouldn't be too inconvenient to support? Not sure I'd find this "inappropriate"? :)

[blackfalcon]

The additional context is helpful. The motivator for type is to have a consistent data type for vendors. That makes sense. But that still doesn't answer the question about making this required.

I would look at how Style Dictionary works. AFAIK, the only thing that is required is value. That being said, you can generate a token without value, you just have to write a different variable to get its value, although not recommended.

I am of the school that I have a color.json file and I define types at that level. That being said, I can see the value of having a type value for other vendors to display data. But all of that seems like it should be an open flexible API option. Use it if you need it. Don't if you don't. Making things required is heavy-handed IMHO in favor of vendors vs authors.

I'd ask @dbanksdesign where he sees the tradeoffs for this.

I see your argument for linting/testing/validation based on data types, but if you are going to use that as a motivator and make it required, then there should be a list of allowed types. Without that vendors who create linting/testing/validators will have a hard time doing so IMHO. Either this body comes up with that list or there is a specification for authors to maintain that list of types.

Regardless of all of this, allowing for types to be set globally versus individually, is a GREAT IDEA! Fully support that. I do something similar to that, but I wrote the code that appends the common data to the individual. @dbanksdesign I am not aware of a feature in Style Dictionary that supports that? If not, wow, that's a GREAT IDEA!

[jina]

I brought up linting/testing/etc to address your "Unless there is a direct benefit to the lib itself, making this required is inappropriate IMHO." comment. I do see that as a direct benefit.

For what it's worth, at Salesforce, where design tokens originated, we relied on types to do transforms. For example, a hex code in a color type would convert to what we needed for Android or iOS (example: 8 digit hex for Android instead of rgba for CSS).

I admit I'm not as familiar with how Style Dictionary does this, but global types (and other attributes) were a big part of how Theo worked.

[blackfalcon]

That makes sense. Style Dictionary does not rely on a type in the data, but allows the user to define that on transform by defining the output format.

[mattfelten]

To elaborate more on the point I made above, making types required will set an expectation that any tool using the spec will have types present and to allow it to make better decisions while using the value. Whether that's transforming them to other formats, like the Style Dictionaries and Theos, displaying them in documentation, etc.

From the example above, the tool can expect #000000 to be a Color and nothing else. It doesn't have to make its own guess at what the intent of the value is.

That seems like a good precedent to set early.

[c1rrus]

Closing this issue as the draft spec has been updated accordingly by PR #87.