Generating Zod schema from TS type definitions

[colinhacks]

Generating Zod schema from TS type definitions

[flybayer]

I'm not sure if this is exactly what @danenania was asking for, but this would be super helpful!:

type Dog = {
  name: string
  neutered: boolean
}

const dogSchema = z.object<Dog>({
  name: z.string().min(3),
  neutered: z.boolean(),
});

And that would fail to compile if dogSchema did not match Dog

[derekparsons718]

@bradennapier I love your project, it seems really helpful. However, from what I can see, it doesn't address exactly what I want. It looks like your project replaces Typescript types with equivalent Zod schemas, which is awesome and much more in line with what the original author requested; but I, on the other hand, want to keep all my types as they are and create schemas that conform to them. I do not want to replace my existing type definitions with schemas, I want to create schemas that conform to my existing types. My whole point is that I want to keep my type system in typescript and only use Zod to validate that objects match up with my existing types. (I'm sure your code could be very easily adjusted to do what I want if something like an advanced tozod was available)

I realize that we can't get away from using schemas because of the whole typescript-doesn't-exist-at-runtime thing, and that isn't really what I'm asking for. I'll give an example.

Right now, if I want to validate that an object matches A, I will need to replace my existing code...

export interface A {
   readonly ID: number;
   delayEnd: number;
   userID: number;
   reason: string;
   taskID: number;
   initiationDate: number;
   days?: number;
   userName?: string;
}

...with something like this...

//hopefully this is correct, I am still new to Zod
export const aSchema = z.object({
   ID: z.number(), //Note that I've lost the functionality of `readonly` in this conversion
   delayEnd: z.number(),
   userID: z.number(),
   reason: z.string(),
   taskID: z.number(),
   initiationDate: z.number(),
   days: z.number().optional(),
   userName: z.string().optional()
});

export type A = z.infer<typeof aSchema>;

But what I really want is something more along the lines of what tozod does, so instead of replacing my type definitions with a schema I can have both. Something like this:

export interface A {
   readonly ID: number;
   delayEnd: number;
   userID: number;
   reason: string;
   taskID: number;
   initiationDate: number;
   days?: number;
   userName?: string;
}

//The use of toZod ensures that the schema matches the interface
export const aSchema: toZod<A> = z.object({
   ID: z.number(),
   delayEnd: z.number(),
   userID: z.number(),
   reason: z.string(),
   taskID: z.number(),
   initiationDate: z.number(),
   days: z.number().optional(),
   userName: z.string().optional()
});

//Note: The above code actually would work just fine with the current `tozod` library,
//but more complex examples would easily break. Hence my original comment.

This preserves my original types and has a schema that conforms to those types. This gives me the same strong typing as using infer<> would have, but it leaves Typescript in charge of defining my type system and still gives me all the benefits of runtime validation. It also preserves certain Typescript functionality that can't be modeled in Zod, like the readonly in A, which gets lost in the current system. I think this scenario gives the best of all worlds and is truly "Typescript-first". I realize that it is slightly more verbose than just having a schema, but I think the benefits are well worth it. It fits much better into the established Typescript paradigm, in my opinion. I can give more extensive reasoning if desired.

Am I the only one thinking along these lines? I'm not even 100% sure that such a system is feasible, which is why I asked earlier if this is something we can even look at pursuing.

EDIT: I've opened a separate issue for this. See blitz-js/legacy-framework#492

[RonHouben]

You can also do simply this with just TypeScript:

const zodSchema = z.object({
	id: z.string(),
	title: z.string().min(5),
})

type ZodSchemaType = typeof zodSchema._output;

[colinhacks]

I think Dane was thinking more along the lines of a Babel plugin for generating Zod code, but what you just described is a very powerful idea. Basically it's the inverse how how Zod works. Zod infers static types from a bunch of method calls and data structures. This is the opposite paradigm: "inferring" the internal runtime structure of a Zod schema from the TS type.

I actually already implemented this as a utility for my own purposes. I didn't put this into the Zod library because it needs TS 3.9+ to work. Here's a gist; don't panic when you see it 😛

https://gist.github.com/vriad/074a8509cd506fdc1f96cad27cc20c77

This gets extremely exciting when you start using it on recursive or mutually recursive types.

type User = {
  id: string;
  name: string;
  age?: number | undefined;
  active: boolean | null;
  posts: Post[];
};

type Post = {
  content: string;
  author: User;
};

export const User: toZod<User> = z.late.object(() => ({
  id: z.string().uuid(),
  name: z.string(),
  age: z.number().optional(),
  active: z.boolean().nullable(),
  posts: z.array(Post),
}));

export const Post: toZod<Post> = z.late.object(() => ({
  content: z.string(),
  author: User,
}));

const CreateUserInput = User.omit({ id: true, posts: true });
const UpdateUserInput = User.omit({ id: true }).partial();
User.shape.posts.element.shape.author;

The above uses a z.late.object method that is currently implemented but undocumented.

As far as I know this is the first time any validation library has supported a recursive object schema that still gives you access to all the methods you'd want: pick, omit, extend, partial. I've been using this to implement the API endpoint validations for a new company and it's been a dream. I'm planning to release first-party support for this in Zod 2 in a few weeks.

[bradennapier]

You could use the typescript compiler api to generate using a transformer. It's actually quite simple to do -- quick demo on vscode extension but could do it in a few ways -- a TS Compiling Transformer (via something like ttypescript) could do it as well

[mrskiro]

@flybayer btw I spun out the toZod utility from that gist into its own module. It's not in Zod core because it requires TypeScript 3.9+ (for complicated recursive type reasons).

yarn add tozod

import { toZod } from "tozod";

type Dog = {
name: string
neutered: boolean
}

const dogSchema = toZod({
name: z.string().min(3),
neutered: z.boolean(),
});

This fails in the current zod. ( "zod": "3.20.2")
We need to override the type in some way.
Here is my example.

import { z } from "zod"

type Dog = {
    name: string
    neutered: boolean
}

/*
Type 'Dog' does not satisfy the constraint 'ZodRawShape'.
  Property 'name' is incompatible with index signature.
    Type 'string' is not assignable to type 'ZodTypeAny'
*/
const badDogScheme = z.object<Dog>({
    name: z.string(),
   　　neutered: z.boolean()
})

type AnyObj = Record<PropertyKey, unknown>

type ZodObj<T extends AnyObj> = {
    [key in keyof T]: z.ZodType<T[key]>
}

const zObject = <T extends AnyObj>(arg: ZodObj<T>) =>
    z.object(arg)


const goodDogScheme = zObject<Dog>({
    name: z.string(),
  	neutered: z.boolean()
})

[fabien0102]

Hi everybody,
I finally did have some time to work on this crazy idea last week, and, I think this is ready to get some testers and feedbacks 😁

I'm glad to present the very early version of ts-to-zod, this is working with zod@next only, I let you try & play yourself and I'm waiting for your feedbacks 😃

Have fun folks! 🤘

Pro tips: Since I'm planning to use this in production, you can generate some "types integration tests" with the --tests, to make sure the generated schema are 100% compatible with the original types 😁 (Edit: Not needed anymore, the validation is part of the generation flow)

PS: Huge thanks to @bradennapier for his POC https://github.com/bradennapier/zod-web-converter this was a very nice starting point ❤️ and @colinhacks to have quickly implement the .required() (#357)

[derekparsons718]

I love this idea that @flybayer suggested, which was partially implemented in the tozod library. ❤️

I'm looking to implement Zod in my project, but it seems like I need to strip out pretty much all of my current type definitions and replace them with Zod schemas. That effectively puts Zod in charge of my type system, which I don't like; in my opinion, Typescript should be the master of typing, and Zod should be the master of validation. One way to do that would be to make the schemas conform to my existing type system in the way flybayer suggested.

tozod was a great step in the right direction, but it isn't sufficient as it is. It either needs to be built out a lot more or we need another solution. Is this something that can be pursued?

To put it a different way, Zod is advertised as "Typescript first", but right now it feels more like "Zod first with Typescript as a close second". I say that because, currently, you have to write the Zod schemas first, then use them to generate types (not doing it this way leads to messy situations). To be truly "Typescript first", the Typescript types have to be king, and the schemas should conform to the types instead of (or in addition to) the types being generated from the schemas.

Would it be helpful to open this as a separate issue? I know it is not really what the author requested and is technically off topic 😜

EDIT: I've submitted this idea as a separate issue, blitz-js/legacy-framework#492

[ctsstc]

It seems the gist is down. Was there any movement to get this baked into Zod? I know we have z.infer but it would be great to have the opposite to create a zode schema from a TS Type or at least couple it and validate it against a TS type.

[lloydjatkinson]

Is this built into Zod now or in the future? Would be great to have it!

[colinhacks]

@flybayer btw I spun out the toZod utility from that gist into its own module. It's not in Zod core because it requires TypeScript 3.9+ (for complicated recursive type reasons).

yarn add tozod

import { toZod } from "tozod";

type Dog = {
  name: string
  neutered: boolean
}

const dogSchema = toZod<Dog>({
  name: z.string().min(3),
  neutered: z.boolean(),
});

[IliyanID]

I know this is old but if you're looking for another simple type to define you could do this

export type TypeToZod<T> = {
    [K in keyof T]: T[K] extends (string | number | boolean | null | undefined)
      ? (undefined extends T[K] ? z.ZodOptional<z.ZodType<Exclude<T[K], undefined>>> : z.ZodType<T[K]>)
      : z.ZodObject<TypeToZod<T[K]>>
};

Then use it like this

import { z } from 'zod'

type A = {
    b:string
}
const properties:TypeToZod<A> = {
b: z.string()
}

const schema = z.object(properties)

If you try to assign b to anything else like z.number() or you just don't include it it will throw a typescript error

[janwirth]

What is it with neutering dogs these days?

[nachten]

Hi There,

Like the toZod implementation but what I would like to see is a bit more of magic (don't know if it's possible, therefore i'm asking)

I would like to see something like this:

type User = {
    id: string;
    name: string;
    age?: number | undefined;
    active: boolean | null;
    posts: Post[];
};

type Post = {
    content: string;
    author: User;
};

export const UserSchema: toZod<User> = z.late.object((model) => ({
    ...model,
    id: z.string().uuid(), // refinements are fine
}));

export const PostSchema: toZod<Post> = z.late.object((model) => ({
    ...model
}));

Would this be possible ?

[Roamin]

Find a online website: https://transform.tools/typescript-to-zod

[rattrayalex]

The gist is now here: https://gist.github.com/colinhacks/074a8509cd506fdc1f96cad27cc20c77 and the related npm package here: https://www.npmjs.com/package/tozod but it's fairly limited and doesn't seem much used.

[ctsstc]

@IliyanID it's nice that it binds the two together, but would still be nice if it could auto-magically generate the zod type from the TS type.

[nachten]

Should I create a feature request for this ? Or can we re-open the this ticket ?

[colinhacks]

I was more hoping for a way to read out a list of models/fields/relations from the Prisma client, instead of having to worry about pulling schema information down from the DB. Prisma's trying to make the .prisma file the single declarative source of truth for your schema but don't provide any easy way to build codegen tools on top of it...drives me up the wall.

imho it still makes sense to generate Zod schemas for your base types (AKA a one-to-one mapping with what's defined in your Prisma schema). That's the ground truth, then you generate the derived types that you need from there. Plus defining the "base types" is the part that requires duplicative static/runtime declarations (because TypeScript can't infer recursive types), so if you can codegen that, you get rid of any syncing headaches.

[jeadys]

If you don't want to install a package you could also try out this online solution: https://transform.tools/typescript-to-zod

[jonlepage]

An extension to @IliyanID comment above is this alternative approach that makes optional fields required with the z.ZodDefault<> type:

export type TypeToZod<T> = Required<{
    [K in keyof T]: T[K] extends string | number | boolean | null | undefined
        ? undefined extends T[K]
            ? z.ZodDefault<z.ZodType<Exclude<T[K], undefined>>>
            : z.ZodType<T[K]>
        : z.ZodObject<TypeToZod<T[K]>>;
}>;

export const createZodObject = <T>(obj: TypeToZod<T>) => {
    return z.object(obj);
};

This forces you to give a z.default() value to optional properties, so when you go to parse() the schema, you do not need to worry about undefined properties.

For example:

Given the type...

type Body = {
    prompt: string;
    size?: number;
};

I can create the schema...

const schema = createZodObject<Body>({
    prompt: z.string(),
    size: z.number().default(512),
});

Then do...

const { prompt, size } = schema.parse(body);

And size will be of type number and not undefined.

when trigger intellisence, this ts code freeze vscode IDE in infinity loop and crash after 2 min !

[Birowsky]

Oh, well then, I'll have to make a case for my kind of peoplez. I don't touch code for maybe weeks in a new project. I only model the system with types and try to find all their dependencies. Only after I feel completely comfortable with how the system is modeled, I start implementing it. So not having the types in advance, is not an option for me, which, I hope, makes a stronger case for why we would find use in Zod generator.

[flybayer]

@vriad beautiful!!! 😍

This will be awesome with Prisma 2 since Prisma provides types all the models in your DB.

[flybayer]

Prisma does have undocumented introspection tools via @prisma/sdk. We are using them in Blitz.

But not sure if auto generating zod from the DB is worth it, since usually your input is a bit different than what's in the DB, just like you have shown in your example above.

[maneetgoyal]

@danenania I should warn that toZod is limited in what you can express. It doesn't support unions (except the special case of .undefined and .nullable), intersections, tuples, records, enums, or literals. I could add literal and enum support if you need it (I think). In general toZod is pretty fragile since its a generic recursive alias (which also can lead to other performance issues with the TS server).

If you could provide a reproducible example of the "Go to definition" issue I can look into it. I haven't encountered that.

@vriad Do these limitations look solvable in the near future or do you think they are just technically infeasible? Trying to assess if I should be waiting on toZod.

Apart from the 3 (or more) use cases described above, I have a slightly tangential (maybe?) requirement which toZod could solve. Basically, some portions of my back-end are in Python and needs data validation. So what I've been doing is auto-converting TS definitions to JSON Schema (for doing validation with say, this library).

After looking at Zod (which looks great for JS based validation), I tried Zod Object Schema --zod.infer--> TS Types --typescript-json-schema--> JSON Schema. This chain works fine but I am getting a JSON schema output which is slightly inferior in quality.

If I define TS interfaces first, then I can end up with better JSON schema (with slightly more advanced validation). It is because typescript-json-schema uses the comments we write alongside the type interface declarations to improve the JSON schema output. With zod.infer, there is no "comment" that I can add to help typescript-json-schema, and hence the validation takes a slight hit.

Now if I have TS interfaces first, then the issue is to keep it in sync with Zod object schema over time as other folks mentioned. So I am kinda split between whether to maintaing both Zod object schema and TS interfaces or move the server side data validation part from Python to JS (and ditch JSON schema) or just be content with whatever JSON schema I am currently getting.

[colinhacks]

@maneetgoyal @danenania The design goal for toZod is to have a tool that can express database models. My goals were to express the "GraphQL grammar" (or perhaps "Prisma grammar" is a more familiar term!).

1. you can define models
2. those models can have primitive fields: strings, numbers, booleans, optionals of each, and arrays of each
3. models have relations to other models (either to-one or to-many)

Perhaps I should have named it zod-model to better indicate this. In this sense I don't consider tozod "incomplete". You can still use toZod to implement certain "core" types, then hook those core types together with intersections, etc.

Dane, I'd like to hear more about your issues with "Jump to definition" and maybe see a minimal repro.

Maneet, I think the ideal flow in your case is to define your JSON Schema types as your "ground truth" and codegen your Zod schemas. Of course, you'd have to implement the JSON-schema => Zod codegen step. If you did, I'd happy to bring it into core and maintain it moving forward. This might help: https://www.npmjs.com/package/json-schema-visitor

[maxfi]

That looks awesome @fabien0102! 🎉 Can't wait to test it out. Also hoping blitz update to zod v3 soon to be able to use this in blitz apps!

[flybayer]

Ok thanks! I wonder if there is a way you can add it into core, but then throw an error if someone tries to use this and TS version isn't 3.9+?

[colinhacks]

@danenania I should warn that toZod is limited in what you can express. It doesn't support unions (except the special case of .undefined and .nullable), intersections, tuples, records, enums, or literals. I could add literal and enum support if you need it (I think). In general toZod is pretty fragile since its a generic recursive alias (which also can lead to other performance issues with the TS server).

If you could provide a reproducible example of the "Go to definition" issue I can look into it. I haven't encountered that.

[bradennapier]

Yes, exactly. It'd be quite easy. Simply add a second tsconfig.tozod.json which only targets the toZod file (via files array in config) and set its out file to the main output directory, add to the build with a tsc targeting that tsconfig, publish.

Better yet you could simply setup a "build" system via references to do it all automatically.

(You will need to make sure the tozod isn't included in the includes pattern of the original tsconfig if it will otherwise cause compilation errors ofc)

[flybayer]

@maxfi you can update to Zod v3 with Blitz. Blitz doesn't bundle zod, so you own the version in your package json :)

[maxfi]

Thanks @flybayer. The problem is if I update to [email protected] it breaks query/mutation resolvers. zod@beta is working though. 👍

[RivasCVA]

An extension to @IliyanID comment above is this alternative approach that makes optional fields required with the z.ZodDefault<> type:

export type TypeToZod<T> = Required<{
    [K in keyof T]: T[K] extends string | number | boolean | null | undefined
        ? undefined extends T[K]
            ? z.ZodDefault<z.ZodType<Exclude<T[K], undefined>>>
            : z.ZodType<T[K]>
        : z.ZodObject<TypeToZod<T[K]>>;
}>;

export const createZodObject = <T>(obj: TypeToZod<T>) => {
    return z.object(obj);
};

This forces you to give a z.default() value to optional properties, so when you go to parse() the schema, you do not need to worry about undefined properties.

For example:

Given the type...

type Body = {
    prompt: string;
    size?: number;
};

I can create the schema...

const schema = createZodObject<Body>({
    prompt: z.string(),
    size: z.number().default(512),
});

Then do...

const { prompt, size } = schema.parse(body);

And size will be of type number and not undefined.

[Birowsky]

Hah! Just a few hours ago? Was just about to plead for the same feature!

But Dane, since we already have the TS types, why would we need the inferred types again?

[danenania]

@Birowsky I don't want to duplicate zod and ts definitions, so am replacing all ts types with zod schemas, then using infer to also get the ts type.

[colinhacks]

Unfortunately I have no idea how to do this sort of thing. Very open to a PR from anyone with experience with this sort of thing.

[colinhacks]

@flybayer Yep, exactly :) I actually have some code lying around that auto-generates Zod schemas from a Prisma 2 client — I'll try to throw that into a separate npm module at some point. I wish the Prisma client provided a nice way of introspecting the schema but currently they don't. So I really had to dive into the guts of the client to make it work.

[flybayer]

Are you in the prisma slack? I'm sure they'd love to hear what you need on the codegen front. And/or open an issue on Github.

[danenania]

Interesting discussion here! We initially redefined most of our TS types as zod schemas, then used z.infer to extract TS types. This works pretty well, but the extra layers of inference seem to sometimes be causing compiler performance issues with complex types, and can also break some IDE features like Go to definition. So we're now looking at refactoring to use the toZod approach where we'll have the TS type and schema defined side by side. I originally didn't want to do this because I worried about types and schemas getting out of sync, but toZod seems to deal with that nicely.

[colinhacks]

@flybayer I'm not aware of a way to do that. If toZod is in core, then TS compilation will fail for any project <[email protected]. Don't think there's a way to disable the type checker based on version. :/

[bradennapier]

Export toZod separately and have them import directly? At least could just be a choice then rather than a whole lib.

Prob a decent bit more can be done as well.

[colinhacks]

Unfortunately most users consume zod with import * as z from "zod" so there's no way to export it from index.ts without everyone accidentally importing it and messing up their builds.

It would be nice to enable something like import { toZod } from 'zod/tozod'; but I'm not sure how to do that. @bradennapier Is that what you mean by "export separately"?

[danenania]

Thanks for the heads up @vriad on toZod's limitations. In that case, we'll hold off on that refactoring since we have lots of complex types.

This is getting pretty far from the original issue, but based on @bradennapier's demonstration that it isn't too hard to translate between TS and zod definitions, I'm starting to think the ultimate workflow here might be for zod to automatically generate schemas for all defined types (probably based on included files in tsconfig), and then allow for extending the auto-generated schemas to add more validations if needed. I'm sure this opens up some other cans of worms, but would make using zod a complete no-brainer in any TS project imo, since currently the only real drawback is needing to define types in zod's DSL instead of in plain TS.

[bradennapier]

I love this idea that @flybayer suggested, which was partially implemented in the tozod library. ❤️

I'm looking to implement Zod in my project, but it seems like I need to strip out pretty much all of my current type definitions and replace them with Zod schemas. That effectively puts Zod in charge of my type system, which I don't like; in my opinion, Typescript should be the master of typing, and Zod should be the master of validation. One way to do that would be to make the schemas conform to my existing type system in the way flybayer suggested.

tozod was a great step in the right direction, but it isn't sufficient as it is. It either needs to be built out a lot more or we need another solution. Is this something that can be pursued?

To put it a different way, Zod is advertised as "Typescript first", but right now it feels more like "Zod first with Typescript as a close second". I say that because, currently, you have to write the Zod schemas first, then use them to generate types (not doing it this way leads to messy situations). To be truly "Typescript first", the Typescript types have to be king, and the schemas should conform to the types instead of (or in addition to) the types being generated from the schemas.

Would it be helpful to open this as a separate issue? I know it is not really what the author requested and is technically off topic 😜

I mean that is pretty much what the project i had started does precisely, @derekparsons718 - more info at #96 . In the end you are going to need it to be converted to the zod because it needs to be runtime level, not type system level in order to implement the featureset, so the best we can do is to make it at least easier to convert your previous types to the appropraite schema when possible.

[derekparsons718]

Since it seems like there are at least a few other people interested in the possibility of typechecking schemas against existing types, I've opened a new issue to discuss it: blitz-js/legacy-framework#492

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ldelia]

It works like a charm! Thanks!

[ahmad-ali14]

Well, this is a super complex problem unless all of your types are set in schema files somehow. I believe the typescript API may be helpful in reading source files and then generating the Zod schema(s); but, it is no easy task.

anyway, would be a very nice feature.

[ctsstc]

This sounds like it would require code generation, which some people are fine with and others may hate.

Folks would need to determine how the generation works. Like does it utilize a glob pattern and generate any type it finds into a zod schema? Would it utilize decorators which would then currently limit it to Typescript? Does it come with a watcher tool that regenerates on changes?

[Harm-Nullix]

Might want to add Date in the mix?

[K in keyof T]: T[K] extends Date | boolean | number | string | null | undefined

An extension to @IliyanID comment above is this alternative approach that makes optional fields required with the z.ZodDefault<> type:

export type TypeToZod<T> = Required<{
    [K in keyof T]: T[K] extends string | number | boolean | null | undefined
        ? undefined extends T[K]
            ? z.ZodDefault<z.ZodType<Exclude<T[K], undefined>>>
            : z.ZodType<T[K]>
        : z.ZodObject<TypeToZod<T[K]>>;
}>;

export const createZodObject = <T>(obj: TypeToZod<T>) => {
    return z.object(obj);
};

This forces you to give a z.default() value to optional properties, so when you go to parse() the schema, you do not need to worry about undefined properties.

For example:

Given the type...

type Body = {
    prompt: string;
    size?: number;
};

I can create the schema...

const schema = createZodObject<Body>({
    prompt: z.string(),
    size: z.number().default(512),
});

Then do...

const { prompt, size } = schema.parse(body);

And size will be of type number and not undefined.