I help software teams ship software faster
Do you wait forever to see your features in production?
I helped a multinational retail and ecommerce company move a critical infrastructure project forward faster with less risk.
Like many companies they had a critical piece of software, something that was used worldwide. Any outage in this product in either of two stages they needed to maintain would impact users worldwide, keeping them from accessing the only control panel available.
After a short, but productive discovery session, we identified the following issues:
- Preparing for a release took hours
- Even with hours spent it was easy to make mistakes that was difficult to detect
- Consistent automated testing was non existent, though tests had already been written
- There was no safe way to rollback
Because the team didn’t have a strong foundation in operational principles, we ultimately decided that it would be best for me to do it for the team and then train them afterwards on the new processes and tools, using the new procedures to teach the thought processes and mental frameworks needed so they could work independently in the future.
In the processes I was able to remove dangerous steps. Steps that if even a small mistake was made would cause data loss of an entire server, causing the entire application to be down for over an hour, tying up the time of at least two engineers across two separate teams.
Previously the process was only vaguely documented and scattered across and followed across several different documents, making it easy to miss steps. Now it’s a familiar, push button interface that any team member can use safely to have a new release reach production, a process that could take hours, sometimes even days, is now minutes.
I also helped them take advantage of code they’d already written. The team had been writing tests, but they were rarely seeing the light of day. They were required to be run manually and there was no way to track what passed and what didn’t over time. Understandably, only some team members remembered to occasionally run this check allowing many bugs to slip through. This would cause hours of debugging and rollback efforts and an increase in support burden.
As this involved the control panel for internal customers to configure infrastructure that could change both routing and content for worldwide sites, an inability to access it could cause catastrophic loss. Outages had previously been estimated in the millions of dollars.
This software was used to add resiliency to the network, routing around trouble, or even correcting other mistakes.
This allows the company:
- to maneuver quickly
- respond to incidents especially during high traffic/peak events
- route around other software errors without having to make a separate code change, saving time
Are you ready for automation?
Do you have the foundations in place to add automation to your development and release process?
I lay out 10 core things that you need to have in place in order to help speed up your release cycle as well as provide guidance on how to do them.
I’ve helped teams and individuals at companies like:
Amazon, Zappos, WalmartLabs, and VMWare.
Complementary Process Diagnostic
With 30 minutes spent looking over your existing pipeline or process, we can diagnose the root cause(s) of slow released or bugs. After 30 minutes on the phone or Skype with me, you will:
- Understand why your existing process may be allowing bugs to reach production and slow you down
- Learn ways you can increase the speed that features reach customers
- Have a list of ideas for improving your release process
We’ll have to move fast, of course, to get through all this in 30 minutes but don’t worry, I’ll keep us on track. To receive your meeting agenda and scheduling instructions, enter your email address below and click the button.