Empowering developers to deploy their own data stores with Terraform and puppet

Tomas Doran

Main - Tuesday 2/2/2016 11:00- D.AUD

Traditional infrastructures involve sysadmins setting up machines and significant lead time. In an agile environment this is often a bottleneck to rapidly iteratation and experimenationt. With the advent of cloud computing, it’s easy to get more machines however they still need configuration and management. Whilst you can empower developers to launch their own instances, they seldom want the ongoing maintenance. At Yelp, we’ve taken a hybrid approach to this problem - in common cases such as Elasticsearch or Cassandra clusters, we already have operational best practices for deploying and managing multiple clusters in multiple regions, and recently we’ve taken this further, allowing developers to entirely self-service provision / scale their own clusters. Using puppet with per-module hiera data allows cluster(s) to be configured from YAML files. Jenkins re-uses this to generate Terraform configuration for clusters, and the operations team is notified about pending changes, allowing easy review and deployment.

About Tomas Doran

Tomas works on infrastructure automation at large scale for Yelp. He speaks regularly at technical conferences on a number of topics between testing, development, architecture, automation and systems administration. Tom came to the dark side of systems & devops after being a professional perl developer for many years. He’s an avid open source contributor and lives in London with his fiancé and far too many cats and dogs.