Photo by Pascal Müller on Unsplash
Top 11 Time Wastes as a DevOps Engineer
12 min read
The daily work of a DevOps can be like a puzzle in the sense that the idea is to assemble several pieces in a logical order to create a structure understood by everyone. The different assembly steps are usually the same for any puzzle and therefore require learning to gain efficiency. Efficient time management is probably what every DevOps engineer seeks to advance in his career. This is the skill that the top performing engineers have improved over time and iteration. Time is what everyone needs to learn and test something new to become an expert in the area.
In a world where everything is changing rapidly, productivity and speed through efficient time management is probably what every company is looking for today. This article lists the known time losses that every DevOps engineer should avoid in order to free up time and spend it more effectively.
"God was able to create the world in seven days because God didn't have to worry about legacy” – Enzo Torresi.
Not Reducing The Technical Debt
The main goal of most DevOps engineers is to identify the technical debt and plan the migration to a new version of the architecture with better processes, newer tools, new ways of working to improve the velocity and the quality of each deployment in production.
Achieving this is never easy. It requires a good understanding of the existing architecture and enough knowledge on new trends to properly define the migration plan. Unfortunately, most of the time, this is not completed in the time allotted to the migration project, which will result in the creation of legacy infrastructure that will require, the next time, reverse engineering to modernize the last elements.
Reducing technical debt is important to move quickly in production, retain top engineering talent (nobody likes to work on unmaintained stuff with no documentation/resources) and avoid rewriting an app from scratch.
Reducing technical debt usually means not working on new features but the trend illustrated by Martin Fowler in the next graph shows that reducing technical debt used to improve feature delivery and quality over time.
Poor design, bugs and outdated technologies are critical points for any development cycle, it can quickly become extremely difficult to work with a complex/buggy code base. Spending time on technical debt reduction is probably the duty of any Devops engineer.
Adding Too Many Features
The easiest way to waste time as a DevOps Engineer, adding to many features.
This can be seen in different ways, the first is probably the lack of communication between teams that lead to emergency management to quickly deploy something into production because customers are waiting for it. Everyone in his career had a misunderstanding between the sales team and the engineering team, where sales people started selling an unfinished feature. The result is usually to do everything possible to move forward as quickly as possible in production and to solve the subsequent potential problems where everything would have been properly managed the first time.
A second aspect that could lead to a waste of time is during the DevOps migration, when it is time to decide on the new potential architecture and the use of new tools. Remember that adding a new tool requires this tool to be maintained (deployed, updated, monitored, etc.). In this case, the objective is to improve how to deploy an application in production without adding technical debt. Adding a new feature to an infrastructure or application must be studied and scheduled properly to be used effectively in production by internal (teammates) or external customers.
Writing The Same Automation Code Again And Again
One of the major principles introduced by DevOps is to deploy quality code quickly in production while requiring the minimum effort. Achieving this goal, as a team, is not that simple. Working as a team requires defining and following best practices to avoid Writing Everything Twice (WET).
WET is probably the opposite of the DevOps culture. Today, developers need to be mindful of the various conventions to properly develop code, the first time. Using the same formatting and the same protocols ensure the code written by one developer can be read and understood by another one. That is probably the key to the success of any automation project.
Don’t Repeat Yourself (DRY) is the opposite of WET. DRY is probably more in tune with the DevOps culture in a way that people need to think about the integrity and quality of the code by minimizing, for example, the duplication of the code. Duplication can cause difficulties in reading as well as debugging, which often results in a waste of time.
Today there are many best practices that have been defined for each programmatic language, each automation tool and each script language. These best practices are crucial to effectively develop any piece of code in a way that your teammates will be able to read, understand and debug easily. Minimize redundancies in code and processes to make your life easier.
The main goal of DevOps is to break down the walls between developers and operators. Unfortunately, even in a world full of different modes of communication, some problems persist and the lack of communication between teams is humongous.
Being transparent with your teammates is probably the best practice for everyone. Transparency reduces risks and promotes team cohesion, which improves the working environment and productivity.
Each communication has its own channel to obtain a quick response, use a direct chat, in case of emergency, call someone, to give an overview of the progress of the tasks, use a board (scrum or kanban), to give an overview of the objectives of the team, use a road map, etc.
Keep in mind that communication, used correctly, is probably the best weapon against wasting time, so, break the walls!
Poor Project Management
It’s potentially not a DevOps engineer’s job to define and manage projects, but it’s important to know how to manage your time correctly, transparently, identify changes in time spent on tasks and identify time wasted on non-critical tasks.
Defining, as a team, each week or two weeks, the work that needs to be done to comply with the deadlines of a project is a best practice. Combined with the daily stand-up to quickly explain what has been done, what needs to be done to close a task and which blocker(s) some face, is a good approach to improving productivity.
Regarding the previous topic, a project management tool can be an excellent tool to easily communicate information to anyone else (team members, other teams like executive team, etc).
This must of course be done properly to not become a waste of time. The time spent on project management must be significantly less than other tasks in order to gain productivity, visibility on future projects and transparency with teammates.
Observability is a critical aspect of DevOps. Monitoring systems allow DevOps engineers to visualize and understand the state of a system using a predefined set of metrics, traces, and logs. By monitoring the behavior of the various components of a system, DevOps engineers can be the first on the scene to detect failures when they occur. Everyone agrees with this statement, any downtime has a cost, it can be human, time, money, etc., it has a cost for someone.
Working with an ineffective observability platform can result in a lot of time wasted in different ways, the most common is probably to be blind to what happened to something on the infrastructure that leads to multiple people/teams spending time identifying where the problem occurred and why instead of applying an alert solution.
Observability also has many benefits. The collection of DevOps implementation metrics can lead to interesting reports over time, such as improvements resulting from refactoring of CI/DC pipelines, the number of deployments per day compared to the previous year, etc. These parameters are important in justifying that the methodology used to improve processes works and is effective.
Observability has many impacts on the daily work of the whole company, if it is not effective for your use, identify problems and take measures to improve it. As mentioned earlier, this part of the infrastructure is essential to save you time.
Delaying Automation Testing
DevOps has improved our work in different aspects with the same goal, improving the speed and quality of new releases and deployments. The beauty of DevOps is that it has introduced best practices, followed by developers for years, to operational teams.
For example, the unification of automation code in frameworks (such as Pulumi, Terraform and Ansible) has allowed operators to use it to release infrastructure, automate deployments and tests, the same way a developer would manage any project.
Like many projects, testing IaC projects is usually forgotten or delayed, but automating the test is certainly a good practice to accelerate production in a more reliable and efficient way. Unlike an application software package that can be quickly brought back to a previous version, IaC can have more impacts on an infrastructure and the rollback used to be more complicated or even impossible without recreating resources. Hence, testing the IaC before pushing a change in production can reduce the potential risks of any deployment.
Every automation tool today has at least one test tool that can be quickly added to any pipeline to test any change. All you have to do is think about it and not delay the implementation to reduce the risk and potential future time loss.
Delaying Security Implementation
The dilemma of every SRE and Dev(Sec)Ops engineer, security used to be the first thing to be delayed on each project for many reasons (time constraint, budget, and many others). But in fact, delaying the implementation of security principles can have a huge impact on the future of an infrastructure, application, database, Infrastructure as Code project, etc. Nobody wants to rebuild an entire infrastructure because a VPC has not been configured properly and it cannot be updated without impacts.
DevOps introduced a new mindset for developers and operators: “Security is everyone’s responsibility.” This principle is obviously the main objective of the DevSecOps methodology, automating the integration of security at each phase of the software development lifecycle to distribute and share the security decision at the speed of agility.
Thinking about the security aspect as soon as possible in any project avoids potential bigger future projects. The purpose of a code (IaC, software, pipelines, etc) is not to be redeveloped every year from scratch. Ignoring security for a long time used to have this bad effect and it can be easily avoided just by defining and potentially enforcing basic security rules that any software lifecycle must respect.
By thinking about security as soon as possible in any project, we avoid future larger projects. The purpose of a code (IaC, software, pipelines, etc.) should not be reworked every year from scratch. Ignoring security for a long time used to have this bad effect and it can be easily avoided just by defining and potentially enforcing the basic security rules that any software lifecycle must comply with.
Not the easiest to implement and apply, but certainly an aspect of DevOps that must be introduced into any project as soon as possible.
Not Measuring DevOps Performance From The Beginning
This is a big mistake that most DevOps engineers face in their careers. Combined with a poor observability, not measuring DevOps performance from the start could become a waste of time without knowing it.
Measurements are values that can be graphed over time to identify bad behaviours but also improvements. DevOps processes should not be ignored in this collection of metrics. DevOps Research and Assessment (aka DORA) is a good example. DORA, through surveys, is able to measure the effectiveness of a DevOps implementation in a company compared to other similar infrastructures.
The main objective of their method is to analyze the velocity and the quality of a DevOps architecture. Improvements need to be made gradually and measured to see them. The number of days to push a commit to production, the number of deployments per day, the number of rollbacks, etc, are metrics that must be collected even before changing something in an existing CI/CD process. What a DevOps engineer wants to avoid is to design a new platform that is ultimately no more efficient than the previous one.
Relying On Outdated Knowledge
This is probably the most important aspect of our career and potentially the most complicated to achieve as it requires a good knowledge of oneself and obviously time and effort.
As mentioned earlier, in our world, things change rapidly. That means, we have to keep our knowledge up to date so we’re not obsolete. Relying on outdated knowledge could be a waste of time. New trends appear every year and require special attention to familiarize yourself with these new concepts and to get a perspective on things that, in some contexts, could affect your career in the future.
Don’t think that what you learned during your study will be applicable throughout your career. DevOps has improved a ton of things in our daily work. Now, people and businesses are thinking about the next step as the NoOps methodology, LowCode strategy, etc. This does not mean that all your knowledge needs to be formatted, you just need to keep an eye and an open mind to new ways of working.
Not Raising Your Voice
Everyone has an opinion about something and should not be afraid to expose their thoughts. Of course, it is easy to say, but it is not easy to do, because you need a certain amount of trust and knowledge to explain your point of view.
This fear of speaking occurs very often when there is a change of context such as a new job or a promotion. New employees, for example, may feel the internal conflict of knowing that there are significant technical debts that need to be settled, but they are afraid to express that concern.
But keep in mind that new employees need to be listened to because of their fresh eyes on the existing technical debt. It should be mandatory for newcomers to express their concerns and push back product management or engineering management to remind them of existing issues and possibly update priorities.
Expressing an opinion constructively is, in some situations, less a waste of time than saying nothing and doing a task knowing that it is not the best approach. Next? The waste of time is probably an unavoidable step in every career. Trying to avoid it is probably impossible, the best we can do is to reduce it by knowing them.
For more information on some mention in this post, refer to the documentation below:
- Is High Quality Software Worth the Cost?
- 8 Keys to Failproof App Modernization
- You Can’t Keep Building on a Broken System: Why managing technical debt is so important
- DORA Metrics to Measure DevOps Performance
About The Authors
Hicham Bouissoumer — Site Reliability Engineer (SRE) — DevOps
Nicolas Giron — Site Reliability Engineer (SRE) — DevOps