Everlywell Gains 5x Deployment Velocity with Support from Tonic

The Everlywell Challenge

The Danger of Growing Too Fast

‍Orders for Everlywell’s home-based lab testing kits spiked when the pandemic made telehealth mandatory in 2020. The company went from 80 employees at the start of 2020 to 1000+ a year later. 

Even with all that new talent in house, their twin challenges were scaling up to meet demand and testing out new features that would put them out ahead of the competition. They were handling around 1 TB of data before the company underwent exponential growth. Sandbox environments were taking up to 30 minutes to spin up, which severely cut into the amount of testing they could schedule in a day.

“If we didn’t have Tonic, we would have to have a DBA to do all that along with a team of 2 or 3 people. It’s hard to even gauge what that would look like because our tables are changing constantly. They would need to run all these custom DBs with fake data. That’s the thing. We need all that fake data so we can create the features people need.”

To keep up with customer demand, Everlywell had to instantly scale up their DevOps team while running more tests on the increasingly complex integrations connecting legacy RDBMS to consumer mobile platforms. All the while, developers were responsible for safely handling highly sensitive, HIPAA-compliant data. 

‍

‍

Developers primarily worked in Ruby on Rails connecting to Postgres with regular calls to AWS. Their CI/CD pipeline went from Dev → Preview/Sandbox → QA → Staging → Production with a tech stack including: 

• Git, 
• Jenkins, 
• AWS CodeBuild, 
• S3 Artifacts Server, 
• AWS Secrets Manager, 
• Launch Darkly, 
• DataDog, 
• and Cloud Watch

‍

‍

‍

The development and QA teams had a lot to keep straight and there were plenty of ways for PII and PHI to leak out into insecure environments. 

“We needed to make copies of many different tables so that we could actually use them to run tests securely. We have all these various user workflows so QA tests became very difficult - setting it up, automating it, etc. - because our database changes weekly, sometimes bi-weekly. We had to make sure our tests were based on the latest data available.”

The Everlywell Solution

Tonic Accelerates Synthetic Data Generation

Before Tonic entered the picture at the beginning of 2020, Everlywell developers were spending too much time generating sandbox environments that were similar enough to production to be valuable. Everlywell developers would have to manually spin up a database, then manually create whatever data they need. All that work would take up to an hour. With Tonic APIs and AWS scripts, everything is included in an automated process that now takes minutes. 

Kowalczyk walked through the process step-by-step, “We wanted to automate the process of getting the latest snapshot of our production database into Tonic. What we do now is delete the old snapshot then create a new RDS snapshot based on the latest data. We use the Tonic API to generate mimicked data and then run a script to confirm and validate that it is practical and has the users we expect. If not, we just add the users. We create a PG dump and push that to an S3 bucket. Based off of that S3 bucket, we create a new database container. When we startup that container in our sandbox environments, that database starts right away.” 

With 50 to 100 sandbox environments running concurrently at any given time, spinning up new functional environments to test features was a top priority. 

As an online-only startup in a field full of healthcare giants, Everlywell called on Tonic to help them outmaneuver the competition by ramping up time-to-market on more and better features. Constant feature development and testing were necessary to make absolutely certain that features were fully market-ready on release. The loss of consumer confidence that follows from a single buggy feature could be a deadly mistake.

The Everlywell Results

Greater Speed, Less Headaches, Lower Production Costs

There are several ways that Tonic has super-charged the feature development cycle at Everlywell, touching every team from development to QA to DevOps.

First of all, new developers are spending less time on the learning curve. “Since we started using Tonic, I’ve seen several developers push new features to production their first week on the job,” Kowalczyk said. 

With their explosive employee growth rate, there is little time for training and mistakes. Everlywell’s development team went from 15 to over 150 in a matter of months. Automation with Tonic had to bridge the knowledge gap, handling the job of 1 - 3 full time database engineers. Those add up to significant cost reductions, especially for a startup.

Beyond the cost savings aspect, the time required to spin up new environments to test features dropped from half an hour or more to less than 5 minutes. That meant that releases to production accelerated from about once a day to 3-5 times a day on average. This new accelerated feature development and testing capability put Everlywell on a new playing field in terms of meeting consumer demand for online healthcare. 

It’s a new world out there and Everlywell is on the forefront of changing the healthcare industry. Tonic’s expertise in generating safe, secure, accurately mimicked data and automating the back end has become an engine of their success.