Engineering for Failure

Chris Grigor, Kerim Satirli

Ansible - Monday, February 1 - 14:40- Ansible room

"Config management tooling makes it easy to install software and set-up services. Testing, on the other hand, is still a lot harder than it should be. In a way, we engineer for success, hardly testing our hypothesis. After all: with quick iterations, you can easily re-run a failing playbook. We recently deployed a a new tool that required a different approach: instead of trusting that Ansible would be able to do what we need it to, we engineered all plays for failure. Taking into account that any of the moving parts could fail, we built the project with failure in mind. In this talk, we look at the steps we have taken to make our playbooks more stable, how we handle alerting and what we have learned from the whole process."

About Chris Grigor, Kerim Satirli

At IceMobile, Kerim is part of a team of automation engineers and helps the company by stewarding the introduction and expansion of DevOps like techniques and practices. In the past, Kerim built web applications for advertising companies, worked on a content management system that was used by the top museums of the Netherlands and helped make music and video more accessible for a number of start-ups.

From installing IP-scramblers in Cold War bunkers to deploying software on Windows in Johannesburg, Chris has seen a lot during his tenure as an integration architect. Now, at IceMobile, Chris wrangles a multi-faceted team of engineers while fighting the good fight of not becoming an armchair general. Ansible is his current weapon of choice.

Slides: Speakerdeck
Twitter: @grigorc and @ksatirli