Skip to content

Review App Pattern - 10x any team's development process

Creating a "Review App" pattern allows every developer to have a "QA environment" for their branch. But setting this up is not as straightforward as it sounds. The secret to the Review App pattern is to maintain two pipelines: code-based and state-based.

Code pushes up, State pulls down.

The History

Problem solving is dependent on feedback – especially in innovative, abstract realms like software development.

Whether they are web, app, IOT, embedded or game devs – developers need feedback.

Just like painters look at the canvas, and musicians listen to their takes - developers need to see their code running, and experience its inputs and responses.

For web developers, the traditional solution to this issue has been "QA environments".

Back in 2006, I landed my first web dev job. The team paid for 3 big linux servers. They were named "dev", "staging" and "production". A part of my job was to SSH in, and configure the resources to look similar (ah, the halycon days of yum package management).

Every developer would take turns "pushing" (literally: ftp copying) their work up the pipeline. Whats worse, they often made changes and fixes in dev or staging that then needed to be manually propagated elsewhere. This enormous bottleneck required exhausting coordination and communication.

Each deployment was fraught with risk. Every release had "scheduled downtime" – while we would manually copy every file and setting that worked on staging to production, test it while the server was in "maintenance" mode, and only then put it back online. This produced so much anxiety in every one of us – each release was like a rocket launch !

Then in 2010, I encountered git branch-based workflows, also called "feature branches. At the time I was the co-founder and Technical Director of a small creative agency. We would work on 2-4 client projects at once, so I was desperate for a way to lower the risk of context switching. Once I read the devops manifesto: The 12 factor App, I saw the future, and never wanted to go back. The context was stored with the codebase! We now had a framework for establishing repeatable, reproducible, deployment workflows.

We had evolved past the "pray it works" deployment flow, but were still stuck on a dev > staging > production pipeline with a single path and it's bottlenecks.

Why?

The Database State Problem

Well, it came down to "state". State was difficult to manage in more than one pipeline. Even though we had the ability to push code, versioned database migrations, package dependencies, etc up through the branches, that wasn't the full story.

An typical app or website is also stateful and highly dependent on its database. While we were happily pushing code & config up the pipeline - the databases were often setup or reset only once per month (or less).

The database "state" is crucial to having your app be in a test-ready position. Its the Yin to the Yang of your Django backend, gulp/grunt/webpack configuration, and postgres and redis connections. It's the "state" that makes your app work.

Inevitably you'll need a particular superadmin user, a customer account in a certain state, the right feature-flags enabled and even a bunch of static data (how many customer databases have I seen a list of US states in?) to have a truly qa-ready environment.

Because dev teams, or non-technical stakeholders often configure these things within the application, the only trustworthy environments remained shared environments – the dev & qa servers.

The promise of code + configuration moving up a pipeline is simply not enough for most situations.

Meeting in the Aisle - Review Apps

The answer: combine the "bottom up" code deployment with "top down" state propgation.

Push the code & configuration up from the repo.

Pull the latest state down from database.

This is the "Review App" pattern that can unlock a dev team.

I've begun implementing this pattern in my consulting work with as many clients as possible and the developer experience is fantastic.

Lately, platforms like Pantheon and Vercel have made this pattern easier than ever.

Its not without its additional work. A few things to watch out for:

  • Sensitive data & privacy: There are very few scenarios where you want customer data to be pulled into every developer's review environment. Flag the tables that store sensitive data and redact or remove them from pulls
  • Seeds & fixtures: This pattern works best when a developer can pull critical data, but also seed an initial set of users / models (or a custom set) for their particular use case
  • Dependencies, Backups & Automation: You must be able to regenerate a working app in a matter of minutes. If you can't do this, you'll need to do the prework necessary to copy or create an environment, database server, networking etc. Docker makes this a possibility in ways that weren't before.
  • Be prepared for more merges: With this new pattern, you'll need faster review cycles and more frequent merges. This is a good thing. But if your developers are used to letting PRs sit for days, you'll end up in a bad place while developers just keep pushing ahead on tickets due to remaining unblocked

The results:

In teams where I've implemented this pattern, we've gone from two-three week release cycles to multiple per week. The bottleneck is no longer the deployment pipeline, but the review process: "ie. How quickly can you review features in a QA environment and approve a merge?"

If you're still using dev > staging > production environments, it's time to adopt the Review App pattern. It's a change that unlocks a new level of productivity for your development team.