Skip to content

Latest commit

 

History

History
150 lines (111 loc) · 11.3 KB

DEVELOPER_DOCUMENTATION.md

File metadata and controls

150 lines (111 loc) · 11.3 KB
Raw

Developer Documentation

File structure

Domain and database design

Domain

Domain classes are found in lib/pact_broker/domain. Many of these classes are Sequel models, as the difference between the Sequel model and the functionality required for the domain logic is similar enough to share the class. Some classes separate the domain and database logic, as the concerns are too different. Where there is a separate database model, this will be kept in a module with the pluralized name of the model eg. PactBroker::Webhooks. Unfortunately, this sometimes makes it difficult to tell in the calling code whether you have a domain or a database model. I haven't worked out a clean way to handle this yet.

Domain terminology

  • pacticipant - an application that participates in a pact. A very bad pun which I deeply regret.
  • pact - this term is confusing and overloaded. It generally means a pact publication in the code.
  • pact publication - the resource that gets created when a PUT request is sent to the Pact Broker to /pacts/provider/PROVIDER/consumer/CONSUMER/version/VERSION.
  • pact version - the JSON contents of the pact publication. One pact version may belong to many pact publications. That is, if a pact publication with exactly the same contents is published twice, then a new pact publication resource will be created with an incremented revision number, but it will reuse the existing pact version.
  • pacticipant version - a resource that represents a version of the application
  • integration - the relationship between a consumer and a provider
  • pseudo branch - A time ordered list of pacts that are related to a particular tag. The most recent pact for each pseudo branch is a "head" pact.
  • matrix - the table that shows the cartesian join of pact versions/verifications, and hence shows which consumer versions and provider versions have been tested together.

Tables

  • pact_versions - the JSON content of each UNIQUE pact document is stored in this table. The same content is likely to be published over and over again by the CI builds, so deduplicating the content saves us a lot of disk space. Once created, a row is never modified. Uniqueness is just done on string equality - no special pact logic. This means that pacts with randomly generated values or orders (most of pact-jvm pacts!) will get a new version record every time they publish.

  • pact_publications - this table holds references to the:

    • provider (in the pacticipants table)
    • consumer version (in the versions table),
    • pact content (in the pact_version_contents table)
    • and a revision number

A row exists for every PUT or PATCH request made to create or update a given pact resource. Once created, a row is never modified. When a pact resource (defined by the provider, consumer and consumer version number) is modified via HTTP, a new pact_revision row is created with an incremented revision_number. The revision_number begins at 1 for each new consumer_version.

  • versions - this table consists of:

    • a reference to the pacticipant that owns the version (the consumer)
    • the version number (eg. 1.0.2)
    • the version order - an integer calculated by the code when the row is created that allows us to sort versions in the database without it needing to understand how to order semantic version strings. The versions are ordered within the context of their owning pacticipant.

Currently only consumer versions are stored, as these are created when a pact resource is created. There is potential to create provider versions when we implement verifications.

  • pacticipants - this table consists of:

    • a name

  • tags - this table consists of:

    • a name
    • a reference to the pacticipant version

Note that a consumer version is tagged, rather than a pact_version. This means that when a given version is marked as the "prod" one, all the pacts for that version are considered the "prod" pacts, rather than having to tag them individually.

Views

  • all_pact_publications - A denormalised view the one-to-one attributes of a pact_publication, including:

    • provider name and provider id
    • consumer name and consumer id
    • consumer version number and consumer version order
    • revision_number

  • latest_pact_publications_by_consumer_versions - This view has the same columns as all_pact_publications, but it only contains the latest revision of the pact for each provider/consumer/version. It maps to what a user would consider the "pact" resource ie. /pacts/provider/PROVIDER/consumer/CONSUMER/version/VERSION. Previous revisions are not currently exposed via the API.

The AllPactPublications Sequel model in the code is what is used when querying data for displaying in a response, rather than the normalised separate PactPublication and PactVersion models.

  • latest_pact_publications - This view has the same columns as all_pact_publications, but it only contains the latest revision of the pact for the latest consumer version for each consumer/provider pair. It is what a user would consider the "latest pact", and maps to the resource at /pacts/provider/PROVIDER/consumer/CONSUMER/latest

  • latest_tagged_pact_publications - This view has the same columns as all_pact_publications, plus a tag_name column. It is used to return the pact for the latest tagged version of a consumer.

  • latest_verifications_for_pact_versions - The most recent verification for each pact version.

  • matrix - The matrix of every pact publication and verification. Includes every pact revision (eg. publishing to the same consumer version twice, or using PATCH) and every verification (including 'overwritten' ones. eg. when the same provider build runs twice.)

Dependencies

= head_matrix view
  -> latest_pact_publications view
    -> latest_pact_publications_by_consumer_versions view
      -> latest_pact_publication_ids_by_consumer_versions
      -> all_pact_publications
        -> versions, pacticipants, pact_publications, pact_versions
  -> latest_verifications_for_pact_versions
    -> latest_verification_ids_for_pact_versions
    -> versions
  -> latest_tagged_pact_consumer_version_orders
  -> latest_pact_publications_by_consumer_versions

= head_pact_publications
  -> latest_pact_publications
    -> latest_pact_publication_ids_for_consumer_versions
  -> latest_tagged_pact_publications
    -> latest_pact_publications_by_consumer_versions (optimised for pp_ids)
    -> latest_tagged_pact_consumer_version_orders (optimised for pp_ids)

Database modeling approach

In the beginning, I made a lot of Sequel models based on views that pulled in the different tables of data together (eg denormalising consumer, provider, pact publication and pact version in to all_pact_publications). This made the Ruby code quite simple, but it was not very performant. As time has progressed, I have moved more and more of the "data joining" code into the Ruby to optimise the queries. That's why there are a lot of "aggregated data" views that are not being used by the code any more.

Useful to know stuff

  • The supported database types are Postgres (recommended), MySQL (sigh) and Sqlite (just for testing, not recommended for production). Check the travis.yml file for the supported database versions.
  • Any migration that uses the "order" column has to be defined using the Sequel DSL rather than pure SQL, because the word "order" is a key word, and it has to be escaped correctly and differently on each database (Postgres, MySQL, Sqlite).

Adding a resource

  • In spec/features add a new high level spec that executes the endpoint you're going to write. Don't worry if you're not sure exactly what it's going to look like yet - you can come back and change it as you go. Have a look at the other specs in the directory for the type of assertions that should be made. Basic rule of thumb is to check the http status code, and do a light touch of assertions on the body.
  • Create a new directory for the classes that relate to your new resource. eg For a "Foo" resource, create lib/pact_broker/foos
  • Create a new migration in db/migrations that creates the underlying database table.
  • Create a new database model for the resource that extends from Sequel::Model. eg lib/pact_broker/foos/foo.rb
  • Create a decorator in spec/lib/pact_broker/api/decorators/ that will map to and from the representation that will be used in the HTTP request and response.
    • Write a spec for the decorator.
  • You may need to create a contract to validate the request. This is kind of broken while I upgrade to the latest dry-validation library. See Beth for more details.
  • Add the HTTP resource in lib/pact_broker/api/resources/. It should extend from BaseResource.
    • Write a spec for the resource, stubbing out the behaviour you expect from your service.
  • Add the route to lib/pact_broker/api.rb
  • Create a service that has the methods that you need for the resource. eg. lib/pact_broker/foos/service.rb
    • Add the new service to lib/pact_broker/services.rb
    • Write a spec for the service, stubbing out the behaviour you expect from your repository.
  • Create a repository eg. lib/pact_broker/foos/repository.rb.
    • Add the new repository to lib/pact_broker/repositories.rb.
    • Write a spec for the repository.
  • Go back and make the original feature spec pass.
  • Profit.