Proposal preview: "Universal versioning"

[microsoft]

Proposal preview: "Universal versioning"

[cleemullins]

If I can summarize what I'm reading, to see if I understand it:

1. When new changes are introduced, they should be back-ported to earlier versions.
2. Clients on the earlier versions ("/v1.2/") now have access to the same API's that are in "/v6.9".

This approach works for additive APIs - that is, API's that done make breaking changes to behavior. Services such as Azure and O365 spend huge amounts of effots to make sure API's don't require breaking changes, but sometimes there is no choice. I don't see how your proposal can work for changes that are themselves breaking.

Services frequently use their logging infrastructure to determine call rates on older API versions. It's not unusual to get an email/phone call saying "You app is the only user of API-X on Version Y which is 4 years old. We would really like to deprecate this API. Can you please upgrade? ". That way API can be deprecated, freeing the API Service to make any needed changes.

Cheers,
Chris

[sharwell]

When new changes are introduced, they should be back-ported to earlier versions.

A better way to interpret this is the documentation accommodates for prior versions. For example, you could say that a call has been available since version 1.2, and implementations prior to 1.2 will return a 501 when you call the method. The client is then responsible for ensuring the client behaves properly if a 501 is ever received from this call.

Clients on the earlier versions ("/v1.2/") now have access to the same API's that are in "/v6.9".

In most forms where I've considered this, all URIs used the same version (e.g. /v1.0). In another form, the server could simply maintain redirects such that /vY is handled by /vLATEST for any Y <= LATEST.

Services such as Azure and O365 spend huge amounts of effort to make sure API's don't require breaking changes, but sometimes there is no choice.

In my experience, the overwhelming majority of breaking changes occurred simply because no one stopped to ask if there was a non-breaking change which is just as straightforward. I'd be interested in hearing about scenarios where a breaking change was needed for some reason.

[cleemullins]

A common example is error codes. The adding of a new error code to an API is often a breaking change to clients. Yet more detailed error codes are often something clients really want added between API versions.

This means front ends are stuck mapping between old/new error codes according to API version, which is just rough and hard to maintain.

[sharwell]

@cleemullins There are multiple ways to support new error codes (and more specifically, the need to add new error codes in the future) without needing to make breaking changes. Two possibilities include:

1. Make sure the HTTP status code (number), and optionally a reference error code if the response body includes one, does not change over time for a given input. Provide an additional field in the response body that includes more details about the error in human-readable form, and indicate that the value of this field may change over time.
2. Allow sub-specialization of error codes. For example, if error code InvalidArg exists and you get an enhancement request for indicating the user omitted a required argument, you could define MissingRequiredArg as a specialization of InvalidArg. To derive this on the client without breaking support for old clients, you could either provide an error codes API which defines derived errors, or return the result in a form with enough information to derive this (e.g. InvalidArg:MissingRequiredArg).

[cleemullins]

What you describe is pretty much exactly what the docs call out doing. :)

With that said, over time, it dos accumulate. It's is baggage and legacy and slows down development, makes testing harder, complicates documentation, and so on.

There isn't an ideal answer that I've seen yet.

[johngossman]

@sharwell You guidance is pretty much what we tell our teams: never break anything if possible. Old versions of the API still work until there is no more traffic on them. Being explicit about the version number allows for cases when things really do semantically change too far to any longer be supported for us to run the APIs side-by-side without creating an entirely new endpoint.

For a little deeper dive, we distinguish between things we know can break clients and things that must break clients. For example, we've found that simply adding a new, optional field to a JSON payload, which clients can and should ignore, commonly results in live site incidents because (for example) clients used a type-strict serializer and throw when they can't map the field. For these we tell services to increment the version number and still maintain the old one.

Somewhat harder are changes that will break applications who are not expecting it. For example, if we change the auth schema for an API obviously any old clients are going to be dead in the water.

Most subtle and difficult are semantic changes. We introduced a new kind of append blob that did not updating or deleting existing blocks as all older types did. This had the potential to break all generic storage explorers. In that case we had to work with all the explorer applications to make sure they were updated.