Comments (15)
@vriad just my 2cents as the yup author, distinctly split transforming and validating. mushing them together like Joi (and so yup) makes it really hard to type correctly and hard to reason about. It introduces oddities like having chaining order matter a lot when it shouldn't for the output type. Some of this is actively making better TS support in yup harder, and while the API is nice, I do wish I made different choices :P and now it's hard to make sweeping changes.
Also Zod is very cool π i'm gonna steal borrow some ideas
from zod.
Thanks for taking the time to write this up. I'm definitely dragging my feet on implementing something like this since it increases the complexity so significantly.
I'm still figuring out whether I want to go in this direction. That decision is primarily determine by whether I can find a way of doing this that I like. Muddying up the schema declaration API with a bunch of pairwise mapping functions (i.e. stringToNumber
) sounds like a mess. I haven't come up with a generalizable solution that I like yet, but I'll leave this issue open so others can chime in with ideas/proposals.
from zod.
It seems to me the main discussion here is that currently zod
handles the validation part of the input processing, but does not provide utilities to handle the sanitization part, which I believe it should since both steps seem inseparable.
For example:
// any => number
const fooSchema = z.number().transform((v: any) => Number(v));
const foo1 = fooSchema.parse('0'); // 0
const foo2 = fooSchema.parse('asd'); // NaN -> throw
// any => boolean
const barSchema = z.number().transform((v: any) => (typeof v !== 'undefined' && v !== null && v !== '' && v !== 'false') || v === 'true');
const bar1 = barSchema.parse(''); // false
const bar2 = barSchema.parse('false'); // false
const bar3 = barSchema.parse('undefined'); // false
const bar4 = barSchema.parse(null); // false
const bar5 = barSchema.parse('1'); // true
// any => string with a specific format
const transformPhone = (v: string): string => { ... } // '1234567890' => '123-456-7890
const refinePhone = (v: string): boolean => { ... } // if string is not 'ddd-ddd-dddd' return false
const phoneSchema = z.string().refine(refinePhone).transform(transformPhone);
const p1 = phoneSchema.parse(1234567890); // '123-456-7890'
const p2 = phoneSchema.parse('(123) 456-7890'); // '123-456-7890'
const p3 = phoneSchema.parse(' (123) 456- 7890'); // '123-456-7890'
// string => string
const createUserInputSchema = z.object({
email: z.string().email(),
// Makes much more sense to be done here, instead of doing it before passing the object to the parse function
password: z.string().refine(checkPasswordStrength).transform(hashPassword),
});
// any => object with specific structure/values
const teamSchema = z.object({
name: z.string(),
cityId: z.number(),
}).transform((v: any) => defaultsDeep(v, {
name: randomTeamName(),
cityId: 1,
}));
This also opens the question about default
utility that sets a default value if the parse/check fails.
With the transform function it can be as easy as:
const fooSchema = z.number().transform((v: any) => Number(v) || 42);
or if there is a default(v: T)
function similar to yup
const fooSchema2 = z.number().default(42);
For simple values the default utility seems trivial, but for complex nested objects it gets a lot more complex.
const orderSchema = z.object({
userId: z.number(),
status: z.string(),
createdAt: z.date(),
}).default({
status: 'Processing',
createdAt: new Date(),
});
The default
function could also support a (v => v2) function that lets you access the input object, for API simplicity this could be also done with the transform
function as in the above teamSchema
example:
const orderSchema2 = z.object({
userId: z.number(),
billingAddressId: z.number(),
shippingAddressId: z.number(),
}).default(v => ({
billingAddressId: v.shippingAddressId,
}));
from zod.
I've been following along on this, and FWIW I think that #42 (comment) is a great option.
It adds a hook into the API which does not transform type information at all, it simply gives the user a hook to transform the raw type into something more useful. We have a library solving a similar problem at https://github.com/sevenwestmedia-labs/typescript-object-validator and it does this with arrays.
Our use case is when using a xml -> json library, it doesn't know if an element is part of a list, or a single child.
This could easily be resolved with
z
.array(z.object({
id: z.string(),
name: z.string(),
})
.transform(val => Array.isArray(val) ? val : [val])
I still would want zod to validate the items etc, I simply want to apply a simple transform. Numbers are just z.number().transform((v: any) => Number(v))
etc.
from zod.
@vriad got it. FWIW: I enjoy zod
so far and this feature would make it even easier to switch.
Idea: maybe mapping should be totally separate from validating. Then zod
could expose helpers to create mappers easily. For example:
const looseString = z.mapper(
z.union([z.string(), z.number()]), // inputs
z.string(), // output
(i) => { // here 'i' can be already inferred as string or number
if (i instanceof String) {
return parseInt(i)
}
return i
}
)
However, I am still not sure how my original problem with object mapping could be modeled this way π
from zod.
+1 for transformations and casting
Yup has a very simple and nice implementation of mixed.cast()
and mixed.transform()
functions:
https://github.com/jquense/yup#mixedtransformcurrentvalue-any-originalvalue-any--any-schema
as well as some in-build transformers like string.trim()
, string.lowercase()
, number.round()
, etc.
from zod.
I'm using Zod as a way to share API schema between frontend and backend.
I solved the need for a transformer by using superjson's serialize/deserialize functions instead of standard JSON.stringify(). It emits metadata so that a Date() is serialized and deserialized back to a Date() itself and not a string.
from zod.
I'd be happy providing an API that lets people make transformations - as long as those transformations don't change the data type. Though I suspect it would get very confusing for people if I introduced a .transform(...)
method that only allowed "intra-type" transformations. Perhaps ".clean" would be more intuitive? Open to ideas on this.
It gets messy when you want to have the input type of .parse
be different from the return type. Zod isn't architected to handle that currently. io-ts
is far better if you need transformations, since it models everything as a "codec" with separate input and output types.
from zod.
@jquense Thanks for chiming in on this! You've summed up my concerns beautifully, which makes sense - you've been struggling with these things for a lot longer than I have. I have some ideas for how to avoid some of these gotchas; I'd love to run them by you sometime. Gonna follow up on Twitter π€
from zod.
Let me dump here few of use cases that I feel like zod
should support. Btw. @vriad I borrowed the API that we discussed in private ;), I hope you don't mind.
- Strictly typed inputs
This is quite useful when working with backend code and input coming from transforms that can loose some type information (like query strings). It's io-ts
's decode
.
const stringToNumber = z.codec(z.string(), z.number(), x => parseFloat(x)));
const qsData = {
userId: stringToNumber(),
name: z.string(),
// ...etc
}
- Whole object transformations
This would be useful while verifying configs from process.env
.
const envSchema = z.codecMap({
serverHost: { value: z.string(), key: 'SERVER_HOST' },
publicS3: { value: z.string(), key: 'PUBLIC_S3_URL' }
privateBackendURL: { z.string(), key: 'PRIVATE_BACKEND_URL' }
})
z.codecMap
could be a built-in helper written using z.codec
under the hood.
- Strictly typed inputs AND outputs.
Let's stick with the API example, imagine that we want to transform the output of the function as well. For example, we work with numbers but for some reason they are always output them as a strings, then it would be useful to have a function to do a reverse and reuse already existing codecs. With io-ts
it's: encode
.
- This should work with opaque types.
Opaque types are types representing some subset of a bigger type (ex. JWT are specific strings or cryptocurrency addresses are specific strings). You can read about them more here: https://github.com/krzkaczor/ts-essentials#Opaque-types
This might be out of scope for this issue but let me give you a great example when opaque types with codecs really shine.
Some time ago I worked on an RPC server implementing ethereum rpc-json
standard. Not to bore you with the details, we ended up having the protocol spec coded in code as huge map describing rpc call name, inputs and outputs: https://github.com/ethereum-ts/deth/blob/master/packages/node/src/rpc/schema.ts#L10
Based on this we had generated type describing all possible rpc calls: https://github.com/ethereum-ts/deth/blob/master/packages/node/src/services/rpcExecutor.ts#L44
What's great here is that: hash
, address
, quantity
are just codecs derived from make functions producing opaque types.
What all of this means is that for example: we got ethereum address over the wire (which can be written in few different ways), we normalized it automatically leveraging codec decode
to an opaque type. So even though it was just a string in the runtime, we treated it as type Address
so it's impossible to pass any string when address is expected. And then finally when we returned it from rpc call it was again transformed back using codec encode
to some (possibly other) form expected by the standard to be sent back to the user.
I hope what I described here makes sense :D I guess points 1 and 2 are most important and rest of it it's just me babbling about how powerful codecs and opaque types are π
EDIT:
When I think more about it, 3 is not that important as it can be easily be done by writing additional codecs transforming stuff "in the other way around". And I just realized that if only codecs would enable passing type arguments instead of zods types they would support opaque types as well...
from zod.
Seems like this could be really useful when it comes to more complex objects. For a specific use-case I'm thinking of, take GraphQL unions that are generated from inputs. Union types only exist on outputs, so it's required to use some workarounds to have varied inputs.
The following is a common solution for this problem illustrated in TypeScript:
type Animal = {
lifespan: number;
};
type Mineral = {
elements: Array<string>;
};
type Vegetable = {
calories: number;
};
type Answer = Animal | Mineral | Vegetable;
type TwentyQuestions = {
answer: Answer;
};
// This is equivalent to TwentyQuestions shown above, but structured differently.
// It is also possible to represent invalid values with this structure.
type TwentyQuestionsInput = {
animal?: Animal;
mineral?: Mineral;
vegetable?: Vegetable;
};
The pattern here is to make each variant of the union a unique field on the input type which is then validated in the API logic to ensure that only one is provided and persisted.
Concretely for zod, this would look like a union on one side and an object on the other side with some additional logic (with zod refine) to ensure that only one key is provided.
There are some patterns in GraphQL and in other systems that require transformations like this. Creating complex structures for both validation/parsing and mapping would be fairly redundant, so it would be ideal to include this type of logic in a library like zod.
A couple related thoughts:
- What actually gets validated? If the parser maps A -> B, does the validation logic operate on A or B?
- What do you do if you want to have mappings from both A -> B as well as A -> C? Thereβs likely a lot of common logic and structure there. What is the best way of sharing it?
- Should there be library logic for composing transformations? That is, if you have logic for A -> B and B -> C, should there be something like a z.compose(a, b) that creates an A -> C mapping from the separate mappers? How would that work?
- What about reversible operations? How would you easily create both A -> B and B -> A?
from zod.
Just created an RFC for this issue at #100
from zod.
@colinhacks I think this issue can be closed, given that z.transform
is implemented already
from zod.
Isn't this a little bit too global? Why does it have to be a compiler flag?
from zod.
Ah found it with strictObject
from zod.
Related Issues (20)
- infer literal in discriminatedUnion fails typecheck but not parse HOT 1
- Add a .precision(number) option to numbers which checks the number of digits after the decimal point
- zod safeparse not working with nextjs server action useFormState hok HOT 2
- Descriptions from root schema carry over to schemas created with a chained `.array()`
- How make field optional but have minimal character HOT 3
- Is it possible to create a zod enum from a string literal type HOT 1
- Importing `infer` type makes all props `unknown`? HOT 2
- Zod Preprocess returning unknown type HOT 1
- object.props "required" but z.infer<T> returns "optional" type? HOT 2
- Required key inferred as optional in mutually recursive tree structure HOT 1
- Strip checks and effects
- Zod refine is not working HOT 1
- Type Inference not working HOT 1
- add `actual` for `invalid_string` error in `.regex()` HOT 2
- Add a Real Example of File Upload Validation to Documentation HOT 1
- Recursive types losing types and get `any`
- Feature Request: Autoscroll to Input Field on Validation Error HOT 1
- Documentation example for recursive types gives type error
- Does anyone know how to conditionally validate form fields? HOT 3
- [Bug ?] Zod union ignoring zod object when fields are z.optional() HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zod.