DevOps in Corporations Myth or Reality

18 Apr 2023     9 min read

DevOps is one of those Buzz words present on the Tech scene for some time. Funny enough during all this time the meaning of it varies a lot, depending on its usage, who is using it, and in which context.

People described whole organisations as DevOps organisations. People described teams as DevOps teams and at the same time, people described certain roles of some people as DevOps engineers.

Some order in the chaos of buzzwords

By the look of it, DevOps means everything and nothing. But don't worry there is some order in all of this.

In most cases when people point out that their org is a DevOps organisation, either they have no clue what they are talking about and just want to make their organisation sound and look modern, or it means that they embraced good "DevOps" practices of having CI/CD tools in place and either DevOps teams or people with the role of DevOps in their org.

In its pure form, DevOps teams should mean that all team members are equally capable in knowledge and access rights to develop code, but also take care of other infra/cloud/network/etc needed stuff for development and running in production.

In most cases, it means that there are people in the team who have DevOps roles who are responsible for all nonpure development stuff. In old days, these people were called system administrators or Ops (short for operations) or Run engineers. Usage of name DevOps engineers is used to highlight that they do all their work through automation and code of some sort (Infra as Code for example) compared to old school admins/sys ops/etc

Need for DevOps in modern Organisation

Let us look at why is there even the need to have something called DevOps in the first place.

First, we need to take a look at the team working on developing a new or existing application that is solving some business need, or adding value. For the team to deliver the end product and value they need to satisfy two parts

• Application Development needs
• Runtime needs (environment, infra, and support around it)

Application Development is done by engineers (backend, frontend, mobile UX/Ui), team structure varies from org to org and depends on what type of application the team delivers. These engineers usually are following engineering practices and develop code in programming languages.

Runtime needs are all the things that are needed for code that engineers wrote to start working in a real production environment and also to be transferred there. Here a lot of tools and other systems come in place, like a version control system for code, a ticket system for development, CI/CD tools and pipelines, and different environments (Test, Acceptance, and Production) which are in the Cloud or Data Center, etc.

What is the same for all organisations and teams, is that whole Runtime needs (stuff) can be easily split into two parts

• Common
• Workload specific

The common part is reusable across the whole organisation. Here we find things like a version control system, ticketing system, basic setup of CI/CD tools and pipelines, setup around artifactory and all other common stuff like static code analysis and quality gates, and other stuff that are the same for all workloads.

Workload-specific stuff consists of utilizing common components and enriching them with specific configurations used in that particular workload. It consists also additionally, of any runtime or infra or similar stuff that is unique for that workload and there is no possibility at the moment for it to be reused. Here comes also any type of environment variables and similar things.

For any team to be able to deliver the end product, work needs to happen in all of these three boxes

1. App development
2. Workload-specific Runtime
3. Common Runtime

Ideal setup and challenge of real world

In an ideal world and setup, we would love to have team members being able to do the work in all three boxes. Since in this way anyone would be able to pick up any work, and this would allow to focus resources depending on the need at that point in time.

In reality, a good part of engineers is not able to pick up work in all three boxes and execute it to high standards. What I see over and over again, in different companies and with different engineers, is that most people are good with doing work in one of those boxes, while not being proficient in the other. This is also one of the reasons why for years there was a split between Software engineers doing the work in Box 1 (Dev) and Runtime/System Administrators doing the work in Boxes 2 and 3 (Ops).

The original idea behind DevOps is to break the wall between Dev (Box 1) and Ops (Box 2 and 3) and optimise work more efficiently. In pure form, it would mean anyone picking any type of work, and if engineers in an organisation are on the level that they can pick work in all boxes, things will move extremely fast and with good quality.

The question remains how to approach this problem in the case when not all of the engineers in the organisation are Rock Stars, and in most cases, they will not be.

Solutions

What I see over and over again is that usually, companies approach this problem in one of two ways

Option 1

Create a Center of Excellence (COE) for Common Runtime needs. This can be either a team or a department consisting of multiple teams, depending on the size of org. Here all common needs are taken care of, templates are being built and anything that can and should be reused across the whole organization is engineered here.

Organize Application Development teams as DevOps teams. Roles present in the team, are depending on the type of work that that team is doing. There can be one or multiple Backend, Frontend, Mobile, UX/UI and other engineers in the team. In teams, there are also DevOps Engineers (or Ops engineers, role names vary from company to company), whose task is to do the work on Workload Specific Runtime needs.

One of the things that can easily happen is that there isn't enough work for a DevOps person in one team, in which case DevOps engineer would split their time between multiple teams. In my experience working in more than two teams is usually very difficult and contra-productive.

Additionally, there is an expectation that in case something that was developed in one specific team to solve their need, evolves and grows into something that is needed in other teams also, would move from the Application Development team into CoE for Runtime Common Needs. This would allow good reuse of solutions across the organisation.

For this to be true proper collaboration model should be put in place between the CoE of Runtime Common and the DevOps part of the organisation

Option 2

Similar to Option 1 create a Center of Excellence for Common Runtime needs across the organisation.

In the case of Option 2 CoE of Runtime would also fulfil needs of Workload-specific needs of Application Teams. In this case, Application teams would again consist of different roles needed to deliver the end product, however, they would not have a dedicated DevOps engineer. For all needs around workload-specific runtime, they need to contact CoE for Runtime for execution.

Option 1 vs Option 2

Time to Value

• Option 1 allows for shorter lead time for delivery to value for teams, areas, domains, and projects, optimizes for time to market
• Option 2 can have a longer lead time for delivering value on team-specific needs compared to Option 1, due to one backlog of items for both Box 3 and Box 2 type of work

Flexibility

• Option 1 allows that in case of high priority, swarming of additional resources to it while making sure that there is enough skeleton crew present on other workloads
• Option 2 In case of high priority in one workload, all other workloads would suffer a lack of capacity
• Option 1 allows the ability to customize the approach per the need of the team, area, domain, project
• Option 2 doesn't allow an easy way to customize the approach per workload, area, domain, project

Resources

• Option 2 would allow operation execution with a smaller number of resources without the need for moving resources around compared to Option 1, the quality of service might suffer

Evolution of Runtime solutions

• Option 2 allows Easy insight into request needs around the Runtime environment that would lead to possible easier optimisations
• Option 1 can reach full potential and ROI only if there is strong and good collaboration between DevOps and CoE Runtime parts of the organization, to make sure of good handover when the capability is moving from Box 2 to Box 3
• Option 2 Since the same team do work in both Boxes 2 and 3, there is no handover needed once capability moves from Box 2 to Box 3

Common mistake

One common mistake that I see over and over again is that CoE for Runtime Needs is positioned in part of the organisation and under people who don't understand Engineering needs or what is needed to deliver modern software. To give you an example classic Stereotype would be an old-school Network or Administrator type of manager and mentality.

In both Option 1 and Option 2, CoE Runtime Needs need to be the foundation on which all Application teams are going to build and enrich the experience. If the person responsible for CoE doesn't understand the task at hand or their customers, then it is destined to fail, to lead to frustrations and reorgs that will aim to solve the problem that shouldn't be even present in the first place.

In any job, for the team to be successful, the correct people need to be in the right positions, and they need to be provided with the right tools and access rights to do a good job. CoE for Runtime need to be positioned under the person who has the right engineering and Application Development mindset, knowledge and experience, otherwise, it will be a sub-par solution.

Unfortunately, a large number of companies make this mistake, and position CoE for Runtime under Old School Infra and Operations which still look at the world through the lens of the 1960s. DevOps came to the tech scene exactly due to this problem, and because a lot of people are doing the same old stuff and using a different name is a big part of why there is a big confusion about what DevOps is and how it should be