AIP-180

Backwards compatibility

APIs are fundamentally contracts with users, and users often write code against APIs that is then launched into a production service with the expectation that it continues to work (unless the API has a stability level that indicates otherwise). Therefore, it is important to understand what constitutes a backwards compatible change and what constitutes a backwards incompatible change.

Guidance

Existing client code must not be broken by a service updating to a new minor or patch release. Old clients must be able to work against newer servers (with the same major version number).

Important: It is not always clear whether a change is compatible or not. The guidance here should be treated as indicative, rather than as a comprehensive list of every possible change.

There are three distinct types of compatibility to consider:

  1. Source compatibility: Code written against a previous version must compile against a newer version, and successfully run with a newer version of the client library.
  2. Wire compatibility: Code written against a previous version must be able to communicate correctly with a newer server. In other words, not only are inputs and outputs compatible, but the serialization and deserialization expectations continue to match.
  3. Semantic compatibility: Code written against a previous version must continue to receive what most reasonable developers would expect. (This can be tricky in practice, however, and sometimes determining what users will expect can involve a judgment call.)

Note: In general, the specific guidance here assumes use of protocol buffers and JSON as transport formats. Other transport formats may have slightly different rules.

Adding components

In general, new components (interfaces, methods, messages, fields, enums, or enum values) may be added to existing APIs in the same major version.

However, keep the following guidelines in mind when doing this:

  • Code written against the previous surface (and thus is unaware of the new components) must continue to be treated the same way as before.
    • New required fields must not be added to existing request messages or resources.
    • Any field being populated by clients must have a default behavior matching the behavior before the field was introduced.
    • Any field previously populated by the server must continue to be populated, even if it introduces redundancy.
  • For enum values specifically, be aware that it is possible that user code does not handle new values gracefully.
    • Enum values may be freely added to enums which are only used in request messages.
    • Enums that are used in response messages or resources and which are expected to receive new values should document this. Enum values still may be added in this situation; however, appropriate caution should be used.

Note: It is possible when adding a component closely related to an existing component (for example, string foo_value when string foo already exists) to enter a situation where generated code will conflict. Service owners should be aware of subtleties in the tooling they or their users are likely to use (and tool authors should endeavor to avoid such subtleties if possible).

Removing or renaming components

Existing components (interfaces, methods, messages, fields, enums, or enum values) must not be removed from existing APIs in the same major version. Removing a component is a backwards incompatible change.

Important: Renaming a component is semantically equivalent to "remove and add". In cases where these sorts of changes are desirable, a service may add the new component, but must not remove the existing one. In situations where this can allow users to specify conflicting values for the same semantic idea, the behavior must be clearly specified.

Moving components between files

Existing components must not be moved between files.

Moving a component from one proto file to another within the same package is wire compatible, however, the code generated for languages like C++ or Python will result in breaking change since import and #include will no longer point to the correct code location.

Moving into oneofs

Existing fields must not be moved into or out of a oneof. This is a backwards-incompatible change in the Go protobuf stubs.

Changing the type of fields

Existing fields and messages must not have their type changed, even if the new type is wire-compatible, because type changes alter generated code in a breaking way.

Changing resource names

A resource must not change its name.

Unlike most breaking changes, this affects major versions as well: in order for a client to expect to use v2.0 access a resource that was created in v1.0 or vice versa, the same resource name must be used in both versions.

More subtly, the set of valid resource names should not change either, for the following reasons:

  • If resource name formats become more restrictive, a request that would previously have succeeded will now fail.
  • If resource name formats become less restrictive than previously documented, then code making assumptions based on the previous documentation could break. Users are very likely to store resource names elsewhere, in ways that may be sensitive to the set of permitted characters and the length of the name. Alternatively, users might perform their own resource name validation to follow the documentation.
    • For example, Amazon gave customers a lot of warning and had a migration period when they started allowing longer EC2 resource IDs.

Semantic changes

Code will often depend on API behavior and semantics, even when such behavior is not explicitly supported or documented. Therefore, APIs must not change visible behavior or semantics in ways that are likely to break reasonable user code, as such changes will be seen as breaking by those users.

Note: This does involve some level of judgment; it is not always clear whether a proposed change is likely to break users, and an expansive reading of this guidance could ostensibly prevent any change (which is not the intent).

Further reading

  • For compatibility around pagination, see AIP-158.
  • For compatibility around long-running operations, see AIP-151.
  • For understanding stability levels and expectations, see AIP-181.
  • For compatibility with client library resource name parsing, see AIP-4231
  • For compatibility with client library method signatures, see AIP-4232

Changelog

  • 2022-08-11: Added "Moving components between files" section.
  • 2022-06-01: Added more links to other AIPs with compatibility concerns
  • 2019-12-16: Clarified that moving existing fields into oneofs is breaking.