Are We Expecting Too Much?

…is the big question that Heiðar Eldberg Eiríksson, team lead at APRÓ, raised during a presentation at the UT conference.

We asked Heiðar to sit down and dig into the presentation to tell us more about DevOps, burnout in technology, and why solutions should always be about people.

"Efficiency is the foundation of software release processes. However, it often happens that the pursuit of efficiency turns into its opposite—and can even lead to burnout among staff," says Heiðar, pointing to recent research[1][2] showing that IT professionals are often under significant stress, especially following the Covid-19 pandemic.

In his presentation, Heiðar discussed how to address this problem. This included ideas from Richard Seroter about "Shift Down," and Heiðar also introduced his own approach, which he calls "Shift Out." This is an approach that APRÓ specialists have developed and are now implementing with both good results and satisfaction. But let's start from the beginning.

Where does the stress come from? And are we expecting too much?

"In recent decades, 'agile' has been dominant in how we organize software projects," says Heiðar, adding that "…this methodology sometimes goes by other names, like Scrum or DevOps, but really it's largely coming from the same roots."

According to Heiðar, the main goal is to move more and more tasks to the left in the software development value chain. This is meant to enable each team to handle all parts of the value chain and remove obstacles in software release.

The flowing waterfall

To explain this, Heiðar describes the workflow that's common in many IT companies today: 

"Let's imagine a traditional and linear workflow in software development, often called waterfall. Suppose there's a specific problem or need that isn't being met, and from that comes requirements for what a solution might look like.
Next comes analysis, where the potential solution is evaluated for its merits—for example, what kind of users the solution would serve and whether there's a market for it, and so on. Usually, product management and/or sales teams handle these aspects.

From there, developers take over and design the system. The requirements are translated into system design. Then development of the solution begins, followed by testing by testers to verify that the system meets the stated requirements.
Finally, we reach release and general use—assuming all went well. Then the system is handed over to system administrators for operations and maintenance."

Shared goals—different responsibilities

This arrangement can work, but the reality is often different, as Heiðar points out:
He mentions that product managers and sales teams are usually in a position where they try to speed up delivery as much as possible, since their main goal is to create value for the customer in as short a time as possible. 

Heiðar continues by noting that developers who take the next step also want to create value for the customer, but they also need to make sure things are done right.
"We need to be careful with system design and data structure so we don't run into problems later." 

Testers try hard to make sure the product is as free of defects as possible, while system administrators may be primarily focused on uptime and security:

"Everyone shares the same goal—to deliver good software as well and quickly as possible. But because these are separate teams with separate responsibilities, tension can develop between teams," says Heiðar.

Heiðar describes the friction that often arises: "Developers sometimes need to hold back product management from moving faster, but then also need to push testers who are holding back development in certain ways." 

He says it's natural that testers want to be more careful in an effort to prevent serious bugs from reaching users.

Once testers finish, they align with developers and product management to want to get this out quickly. Then it's system administrators who stand in the way of release with their change management process, trying to minimize risk from the release in terms of security and system uptime.

Image: UT Conference

A persistent root cause

"A heavy change management process encourages more simultaneous changes, which increases the likelihood of release problems, which then leads to stricter management," says Heiðar.

He notes that more changes means more testing, which lengthens testing time, which pushes developers to make more simultaneous changes, and so on, in a vicious cycle. 

"In DevOps theory, this tension is called The Core Chronic Conflict, and the friction that grows between teams when responses to problems become slower is called The Downward Spiral."

Various agile methodologies have emerged to eliminate this tension by moving responsibility for tasks as far left as possible in the value chain. 

Heiðar explains that the most common approach is to put developers and testers together in one team, moving responsibility for testing to the left. Then there's one team that designs, writes code, and tests.

He also says it's common to use something called "Continuous Integration" to increase feedback cycles and enable teams to make more frequent smaller changes rather than fewer, larger ones.

The bottleneck shifts

What remains, however, is the bottleneck between the development team and system administrators, and that's where DevOps methodology comes in.
By bringing responsibility for release, operations, and maintenance into the team, development teams are incentivized to solve common problems that come up in operations through automation and with better and better feedback loops.

By focusing on three core elements of DevOps methodology (Flow, Feedback, and Experimentation and Learning), teams are motivated to maximize efficiency in discovering and fixing problems, which reduces waste and rework, improves product quality and security, and increases uptime—because even when problems occur, the time to find the problem, respond to it, and solve it has been minimized.

"Of course, this is a radical simplification of a comprehensive methodology that's best understood by reading The DevOps Handbook," but Heiðar also directs readers to its roots in Lean, Toyota Production Systems, and others.

The benefits of DevOps

The benefits of working according to this methodology are fairly clear-cut, and research[3] shows that companies:

• Release software faster and are therefore quicker to market and able to respond to market changes.

• Find defects sooner, which minimizes negative impact and reduces costs.

• See increased reliability and automation, which similarly increases efficiency and limits rework and waste.

• Improve product quality because problems are discovered and solved sooner and follow-up is better because responsibility for maintenance doesn't shift between teams.

The annual State of DevOps Report from PuppetLabs and Google's DORA team (DevOps Research and Assessment) shows similar results year after year, according to Heiðar, where "companies working according to DevOps methodology are significantly more likely (at least 1.35 to 2 times more likely) to achieve their goals, whether those goals are about:

• Profitability

• Productivity

• Market share 

• Number of customers 

• Product or service sales

• Operational efficiency 

• Customer satisfaction

• Quality of product or service delivery 

• Achievement of organizational/institutional goals

  
  

Heiðar notes that this applies equally to for-profit companies and nonprofit organizations, including educational institutions and public agencies[4].

Careful implementation and right priorities reduce burnout risk.

Other statistics have emerged, says Heiðar, that point to the importance of how such changes are made: "How responsibilities and tasks are moved to the left in the value chain. And we can't forget the risk that comes with overreaching in this area, whether it's called Agile, DevOps, or something else."

Heiðar quotes Richard Seroter, Chief Evangelist at Google, on the subject, and Seroter says, among other things:

"Certainly, shifting left—embedding security and quality reviews into the development process earlier—is completely reasonable. But over time, more types of tasks that haven't always been part of a programmer's job description have shifted left in the name of creating 'full stack' developers."

Seroter goes on a bit of a rant to make his point:

"There are about nine real 'full stack' developers on Earth. Almost no one writes React frontends, sets up Kubernetes, configures RabbitMQ clusters, provisions space on a SAN, and racks switches in data centers.
Today, developers are expected to know web frameworks, design patterns, testing methods, build systems, many types of databases, caching, automation infrastructure, container orchestration systems, L4-L7 networking concepts, SaaS APIs, monitoring systems, a range of public cloud solutions, and yes, maybe some AI. I was flipping through Indeed.com, and it's striking to see what we're asking of new and experienced developers.

That's too much."[5]

Where does responsibility lie?

According to Upwork Research Institute research[1], it emerges that:

• 71% of full-time employees show signs of burnout

• 96% of Managers want to use AI more, while 47% of employees don't know how to do it

• 77% say that AI tools have increased their workload

  
  

Upwork Research Institute results thus point to the problem being in work methodology and its implementation—and so-called top-down performance metrics.

In Heiðar's view, management metrics focused on performance and progress are at odds with Agile and DevOps philosophy, which emphasize team autonomy in measuring, evaluating, and taking responsibility for their own performance.

Another interesting study that Heiðar references is by Tulili, Capiluppi, and Rastogi, "Burnout in software engineering:

A systematic mapping study," which tries to draw lessons from a total of 92 studies on burnout in the IT sector worldwide going back to 1990, and it shows that over 80% of IT professionals have experienced or are experiencing burnout after Covid-19.[2]

In their research, Tulili et al. examine how burnout in the IT sector has been studied so far, whether lessons can be drawn from it, and whether improvements can be made in analysis, including with the help of AI or machine learning.

"One of the most remarkable things I found from the research was their attempt to synthesize lessons from all these studies into some kind of causal diagram of burnout, where we see certain patterns worth noting," says Heiðar, and highlights a few points.

"We as individuals bear responsibility for ourselves. Possibly the biggest risk factors for burnout are our own personality traits, as burnout is often called the disease of the conscientious. So we need to be aware of and in connection with ourselves." 

Heiðar continues by saying that if people experience signs of burnout due to stress, he encourages them to seek help.

But managers also have responsibility, says Heiðar. Work demands and workload must be realistic, but it's also been shown that how managers communicate with their staff matters a lot.

Heiðar says it matters a lot that staff feel they're getting sufficient and correct information from management, whether it's about the work, the company's status, or something else—that staff feel they're hearing the truth.

Furthermore, implementation of so-called agility practices matters, and Heiðar notes that it's important that roles are clear and says: 

Burnout is costly

The total cost of burnout is really largely unmeasurable, says Heiðar. That is, if about 80% of IT staff experience burnout, we can conclude that their performance has decreased by some amount X, but that's very difficult to measure.

However, we can use VIRK activities as a kind of unit of measurement, as the benefit of the activity is calculated at 19 billion per year, and clients from IT along with administration, financial services, and general office work top the list as VIRK's largest client group.[6]

What can we do?

Heiðar again quotes Seroter, who has come up with an idea he calls "shift down" as a possible solution to this problem. Seroter has explained the solution this way:

"As an industry, we need to help. First, instead of telling programmers (and their managers) to shift everything left, we need to encourage them to shift down by making full use of the technology available to them and moving more work down onto the platforms they're already using. Compress your technology stack. Don't force people to need to know so much to do their job. Offer platform abstractions."[5]

Here Seroter is referring, Heiðar explains, to an evolution in DevOps methodology called Platform Engineering. This is about wrapping up the complexity of implementation, integration, and operations and making it easier to utilize certain capabilities within the software development value chain.

This trend of building platform engineering teams can be seen more widely in recent months, Heiðar points out, but according to Gartner, about 80% of companies plan to have a dedicated Platform Engineering team by 2026[7] following the model of companies like Google.

And Seroter continues:

"At Google, we provide platforms for development, testing, building, release, deployment, hosting, alerts, and more. Special teams from Google support these crucial platforms so our product developers can focus on what they need to do without having to know about or manage a 'full stack' of infrastructure. Every organization should do the same."[5]

Although he agrees with the philosophy, Heiðar notes that this isn't realistic except for larger companies like Google, and he says:

"It's not realistic for a small development team with maybe 4-6 developers to build an entire separate platform team to support themselves, let alone multiple teams for different parts of the platform."

However, Heiðar encourages specialists and managers in Iceland not to give up on the philosophy.

Heiðar's idea is to acquire a platform that meets the needs for the development environment and value chain at any given time, to achieve results.

APRÓ has recently developed such a platform for the Icelandic market called APRÓ Lighthouse. This way smaller companies and organizations can get access to extensive experience and good tools without significantly increasing pressure on their staff or major additional expenses in development costs.

APRÓ Lighthouse

"APRÓ Lighthouse is developed in collaboration with Icelandic companies and organizations and is largely about collecting and formalizing what we know has worked well over time," says Heiðar, also pointing out that significant economies of scale can be achieved by participating—across companies and organizations. 

Lighthouse offers the opportunity to use and develop a platform together for the needs of Icelandic companies and organizations instead of each working in their own corner to solve basically the same problems.

Heiðar says that APRÓ Lighthouse is a way for companies to outsource complexity and much of the pressure to parties that have solving these problems as their core business.

"Meanwhile, our customer development teams can focus on creating value for their organization without compromising their ability for development speed, monitoring, uptime, security, or anything else. That's our core business."

Heiðar also points out that they at APRÓ see growing demand in the business environment to be able to use AI safely, but at the same time we see that it brings increased pressure and complexity for development teams. That's where the idea for the data and AI Lighthouse came from.

"With APRÓ's approach, companies can safely use the possibilities that lie in AI and modern data infrastructure without making compromises in security."

He explains that AI models are run in a secure environment in a cost-effective way, "which allows our customers to enrich AI, even with their most sensitive data, while being secure in the knowledge that nothing is going through systems of foreign companies."

APRÓ's AI runs AWS Bedrock services, designed so that the infrastructure is separate from other APRÓ or AWS customers and with encryption that meets the strictest security and quality standards.

More want to be part of the solution

Modern technology calls for modern solutions and work methods that create both job satisfaction and pleasure as well as increased efficiency and value—something that cannot be compromised on, says Heiðar. 

"Our approaches are constantly evolving, but DevOps and Shift Out have worked very well for us at APRÓ, and we believe in this solution." 

He adds that the more companies and organizations that become part of APRÓ Lighthouse, the better the solution becomes—so the solution remains sustainable in that it improves with increased participation.

 All of this leads to both better methods but also better well-being for those involved in the work, and Heiðar ends with these closing thoughts:

"Our solutions will never be better than the people themselves doing the work, and the key is finding a way that maximizes both." 

Want to know more about APRÓ Lighthouse? Click here

References and Further Reading

[1] Monahan, Kelly, and Gabby Burlacu. "From Burnout to Balance: AI-Enhanced Work Models for the Future." Upwork, July 23, 2024. https://www.upwork.com/research/ai-enhanced-work-models.  

[2] Tien Rahayu Tulili, et al., "Burnout in software engineering: A systematic mapping study", Information and Software Technology, Vol. 155, (2023)

[3] Rubén Grande et al., "Is it worth adopting DevOps practices in Global Software Engineering? Possible challenges and benefits."  Computer Standards & Interfaces, Vol. 87 (2024),

[4] Dr. Nicole Forsgren, Jez Humble, Gene Kim, Nigel Kersten, et al. "State of DevOps Report" 2013-2024 PuppetLabs, DORA,

[5] Seroter, Richard. "Richard Seroter on Shifting down vs. Shifting Left | Google Cloud Blog." Google, June 8, 2023.

[6] Stefánsdóttir, Berglind, and Guðrún Rakel Eiríksdóttir. "Burnout in Work." VIRK Vocational Rehabilitation Fund, May 23, 2022.  

[7] David Sandilands, Margarret Lee. "2024 State of DevOps Report: The Evolution of Platform Engineering, Puppet by Perforce,