The Timeline of Infrastructure Engineering: Past, Present, and Future
The role of infrastructure engineer has undergone a significant evolution over the past twenty years. As technology has advanced and the way businesses operate has changed, so too has the role of the infrastructure engineer. Today's companies are striving to do more with less. In this blog post, we will take a look back at the past twenty years and examine the changes that have occurred in the infrastructure engineer role, and discuss the debate of how infrastructure engineers must continue to change to remain relevant in the future. Additionally, we will also explore the concept of "Cattle vs Pets" management style of infrastructure resources.
In the early 2000s, the role of infrastructure engineer was primarily focused on maintaining and managing on-premises infrastructure. This included tasks such as configuring and maintaining servers, troubleshooting network issues, and ensuring the availability and performance of applications and services. These tasks were typically performed in on-premises data centers, which were the primary location for storing and processing data.
As cloud computing became more prevalent in the late 2000s and early 2010s, the role of infrastructure engineer began to shift. The emergence of cloud-based infrastructure presented new challenges and opportunities for infrastructure engineers. They were now responsible for managing and deploying cloud-based infrastructure, in addition to maintaining and managing on-premises infrastructure. This included tasks such as configuring and deploying virtual machines, managing and scaling cloud-based storage and databases, and ensuring the security and compliance of cloud-based infrastructure.
In recent years, the role has continued to evolve to include new technologies such as containers and Kubernetes. These technologies allow for the deployment and management of infrastructure at scale, and have become increasingly important as businesses have moved towards a more agile and DevOps-oriented approach to infrastructure management. Today's SREs are yesterdays systems engineers - adopting software approaches and methodologies to apply to infrastructure and platform creation. Additionally, the role has also involved more emphasis on security, monitoring, and incident response. This is due to the increasing number of cyber threats and the importance of ensuring the security and compliance of infrastructure. Remember everyone's favorite security phrase... "Security is everyone's job ;)".
The use of automation and configuration management tools such as Ansible, Terraform and Puppet, have also become increasingly common in the role of the infrastructure engineer. These tools allow infrastructure engineers to automate repetitive tasks, manage and deploy infrastructure at scale, and ensure consistency and reliability across different environments. I've likened Terraform to a spice dispenser where I've described AWS, Azure, and GCP as salt, pepper and paprika. They're all just spices that we can shake out with Terraform.
One of the most important changes in the infrastructure engineering role is the shift from "Pets" to "Cattle" management style of resources. I've recently read and heard a lot on what many believe an SRE or platform engineer should be, especially with the recent big tech layoffs. There's a lot of debate about how infrastructure operations should evolve if they are to be deemed worthy of the title of DevOps, SRE, or platform. My take on this, plainly, is to adopt software practices and automate everything possible. In the past, infrastructure engineers often treated servers and other resources like pets, giving them unique names, and carefully managing them. With the advent of cloud computing, virtualization, and automation, resources have become more disposable, and it's more efficient to treat them like cattle, where you have many similar resources and can easily replace them if needed. This approach allows for better scalability, reliability, and disaster recovery. Fun fact, my first infrastructure support position had a server closet and the "servers" were named after big cats with no naming convention or differentiation by role - just Cheetah, Panther, Bobcat, etc... This implementation is understandable in small operations but certainly not scalable.
While the role of infrastructure engineer has evolved to meet the changing needs of businesses, there is ongoing debate about how infrastructure engineers must continue to change to remain valuable in the future. Some argue that infrastructure engineers must continue to expand their knowledge and skills to include new technologies such as artificial intelligence and machine learning, in order to keep pace with the rapidly changing technological landscape. Others argue that infrastructure engineers should focus on developing more advanced skills in areas such as automation and configuration management, in order to improve the efficiency and effectiveness of infrastructure management.
There's no doubt that the role of infrastructure engineer will continue to evolve and change in the future. The increasing importance of cloud computing, the adoption of new technologies such as containers and Kubernetes, and the need for automation and configuration management tools, will all play a role in shaping the role of the infrastructure engineer in the future. It's important for infrastructure engineers to stay informed about the latest technologies and trends, and to continuously develop their skills to meet the changing needs of businesses. This includes not only staying up-to-date with new technologies and tools, but also understanding the "Cattle vs Pets" management style of infrastructure resources and how it can benefit the organization. By staying adaptable, infrastructure engineers can ensure that they remain valuable assets to their organizations in the years to come.
One final note. In today's economic climate and career uncertainty, the best investment that anyone can make is in themselves. Continue to adapt, listen, learn, absorb, review and reflect. If you're an infrastructure engineer looking to move into DevOps/Platform Engineering/SRE, I strongly recommend taking a look at roadmap.sh/devops and familiarizing yourself with a CI/CD tool, an infrastructure-as-code tool, and start working toward learning a development language.
