Creating Value in AI Powered DevOps

While artificial intelligence (AI) and machine learning (ML) are emerging technologies, and we know they can help an organization analyze large data sets and gain actionable insights. But does AI-infused processes make a difference in organizations using DevOps? The answer was explored in a recent survey by Tricentis. Working in partnership with Techstrong . GroupTricentis surveyed more than 2,600 DevOps practitioners worldwide about AI-powered DevOps. The results indicated a bright future in DevOps.

In this survey, the majority of respondents stated that they are already getting value from AI-powered DevOps. A third of survey respondents find AI-powered DevOps very useful today and nearly half find them extremely useful. Although the technology is still relatively new, a 2021 report by IT infrastructure automation company Puppet Labs found that 83% of organizations are implementing DevOps. This rapid adoption speaks volumes about the promise of technology.

DevOps Experience 2022

Interestingly, nearly two-thirds of respondents indicated that testing is the area within DevOps where AI will have the greatest impact. Testing is a major weakness for DevOps organizations because the computed DevOps environment requires complex test scenarios and large amounts of data, and many organizations struggle to scale their automation to the level that DevOps requires. Without automation, manual testing can significantly delay releases. In AI-infused DevOps, AI can help speed test authorship, help testers understand where the risks are and fix broken tests.

The greatest potential of AI is functional testing

We usually think of software testing as either functional, where the software allows users to perform tasks correctly, or non-functional, where the application is safe, fast, and stable. Respondents were asked which test disciplines most benefit from AI, and 65% of them indicated functional testing. Unit testing and UI testing were mostly chosen subcategories.

The complexity of the functional testing process is where AI shines. Functional testing is open-ended and many changes are required to ensure proper testing coverage. AI intervenes and manages these permutations, analyzing large amounts of data and providing insights. In addition, AI can help discuss production signals that are not detected in the test environment.

During the survey, respondents specifically mentioned the promise of AI in terms of UI testing. User interface testing is probably the most time-intensive manual test in most organizations. With so many variations on existing user experience paths, AI can help simulate the actions of a real end user, adapting to application nuances or differences in experience. AI and machine learning can also enhance your building process, increasing the efficiency of repetitive tasks. The ability to quickly identify and solve problems can improve the effectiveness of your automation.

While survey respondents see the most potential in the test, they acknowledge that AI-powered DevOps can tackle other business challenges. More than 40% claim that these processes can reduce the skills gap and enable junior employees to perform complex tasks, improve customer experience, reduce costs, foster innovation and increase efficiency among developers. Nearly 40% also believe that AI-powered DevOps provides leadership with the insights they need for continuous improvement.

Automation is the key to scaling up development processes

It is worth noting that among the DevOps organizations that see benefits from AI and machine learning, most are mature organizations; They use DevOps workflow pipelines, tool chains, automation, and cloud technologies, and have automated more than half of their testing. Only 21% of practitioners who dip their toes into DevOps have achieved this level of automation. Even in mature organizations, testing remains a major bottleneck; Only 40% of them automate more than half of their tests.

The key to scaling DevOps practices is to develop test automation that can keep up with the increasing volume and speed of releases. But the road to DevOps isn’t always easy. Practitioners cited obstacles such as lack of technical skills (44%), insufficient budget (25%) and choice of tools (19%). Organizations that want to implement DevOps need to rely on external suppliers and internal resources to meet these challenges and scale effectively.

Looking to adapt more DevOps practices?

So where does an organization that wants to implement DevOps start? With the big questions. It is necessary to identify the specific business problem that you wish to solve. What profit are you looking for? In which program life cycle do you expect to find benefit? AI and machine learning can add business value, but they don’t solve every problem throughout the lifecycle.

After the organization sets its DevOps goals, its focus should shift to automation. To reap the benefits of AI-powered testing, an organization must automate frequent functional tests, especially unit testing and regression. Most organizations begin by implementing continuous integration and continuous delivery. Once the organization takes these steps, it will start accumulating a large cache of results and artifacts generated from running test cases, which AI can mine to increase test stability and identify recurring issues. The more you automate, the more you’ll get out of it and the faster your delivery.

Organizations just getting started with DevOps must understand the difference between AI and ML. Most people refer to ML as AI, but it is not interchangeable, and most organizations haven’t come forward to use AI. Real AI is powerful at scale, while ML techniques are narrow and solve a domain specific problem for which the model has been trained.

When evaluating potential sellers, there are a number of key questions to ask, including:

• What specific AI technology does the seller use? Is it a ready-made library or something they built themselves?

• How do they use artificial intelligence? What algorithms are running and what is your approach?

• How do they train their models? Different types of patterns appear in the datasets depending on the industry, the frequency of updating the data, and the application. The dataset the vendor is training on should look like your production data, or the model may take you in the wrong direction.

• How do they make sure that bias does not play a role? If they train models in a different field or sector, for example, you may be out of step with that vendor.

• How do they deal with failure? How do they monitor AI misclassification and misapplication in a way that allows you to trust the data?

• Will your data be used to train and improve your learning models? If so, does that raise any privacy concerns? Expectations about data privacy should be explicit and leave no questions asked.

• Are there other clients in your sector with a similar business profile who are getting the benefits you are looking for? Can the seller provide examples?

What does AI-saturated DevOps look like in practice?

Consider a financial services organization that has more than 20 apps to manage. With so many applications, important UI testing is necessary and the tests should be stable. One of the biggest challenges of UI testing is locating different web elements; Most people start with manual text-based automation, but creating and maintaining it can be difficult and time-consuming. Using AI and smart locators to locate these items can significantly save time and reduce user interface testing maintenance.

Survey respondents were asked how they are currently using AI to increase their testing processes. 37 percent say they speed up the creation of their automated test cases, while 44 percent focus their tests on the most risky areas. Nearly half of them are able to reduce test case maintenance by self-healing and 43% use AI to identify the root cause of failed tests. Thirty-one percent claim AI provides insights into testing process improvements and 34% use AI to help select tests to run based on in-app changes.

Where will AI-powered DevOps go next?

From now on, AI will become an integral part of testing processes. Great progress is being made with ready-made sellers, and their products are more easily accessible than they were before. As tools continue to mature, these products will reduce the need for specialized skills. Internal initiatives such as data lakes or data warehouses will continue, but there will be less need to build large data science teams to take advantage of the benefits of AI.

Leave a Comment