In this chapter, we introduce a new AI-powered performance optimization approach that automates performance testing, identifies optimal configurations based on custom-defined goals, and applies them to target systems.
Any company offering digital services today needs to ensure that the highest level of performance and availability is delivered to end users. However, despite investments made to improve the quality of code and the time spent in load testing and performance tuning, results are often neither adequate nor cost-effective.
The world of performance optimization is changing in response to software becoming more agile and technology becoming more complex. Traditional performance tuning approaches are not effective and do not scale anymore, so significant optimization opportunities are left on the table.
How best to optimize the performance of modern applications sitting atop of so many technology layers? How to find the best configuration when there is an exponentially growing number of settings at each layer, with counterintuitive interplay for each specific application — to the point of heavily affecting the overall performance even when vendor defaults or best practices are adopted?
The answers to these questions are addressed by Akamas, a new AI-powered performance optimization approach to automate performance testing, identify optimal configurations based on custom-defined goals, and get them applied to the target systems. This approach enables human experts to understand the dynamics of even the most complex systems, make decisions about which configuration to best apply, and ensure that systems are continuously kept optimized in a closed-loop fashion.
In this chapter, we describe some real-world use cases where the joint use of Akamas and Tricentis NeoLoad testing capabilities help solve this optimization problem, by making reference to a sample containerized Java application.
In a world where the ability to deliver new digital services and updated capabilities at business speed is critical, the topic of performance optimization is even more important today than ever[1].
As a matter of fact, tuning configuration parameters can deeply affect performance of the application. A striking real-world example was reported in[2] about a critical Java-based application of one of the largest insurance companies in Europe. In this case, it was enough to just reconfigure the Java Virtual Machine (JVM) to get a 75% reduction of CPU usage of the application servers, with the system still able to support the same traffic load.
Unfortunately, performance optimization is getting harder and harder these days. There are several fundamental reasons, including:
As a consequence, infrastructures are often overprovisioned (which may also imply higher licensing costs), yet simultaneously under-optimized. And when they are not optimized, more infrastructure may actually impede good application performance and a better user experience. Moreover, a manual, trial-and-error process typically has a huge operational impact in terms of hours spent by expert performance engineers — and this may prevent organizations from being more agile in their delivery processes.
The good news is that there is a better approach to performance optimization.
[1] Sogeti, “State of Performance Engineering”, 2020
[2] Stefano Doni, “Lessons from Capacity Planning a Java Enterprise Application: How to Keep Capacity Predictions on Target and Cut CPU Usage by 5x”, CMG 201
[3] Stefano Doni, “How AI optimization will debunk 4 long-standing Java tuning myths”, CMG 2021
The Akamas methodology and solution were developed by veteran practitioners who had first-hand experience with the performance tuning, capacity planning, and performance optimization challenges described above. They realized that just trying to automate the process of running tests to verify whether a given configuration was providing better results (with respect to defined goals) was not enough. The need to explore thousands of configurations makes a brute-force approach simply unfeasible.
Akamas’ key ingredient is patented AI/ML techniques that make it possible to smartly explore the space of configurations, in a way that can rapidly converge to an optimal configuration. With Akamas, every optimization study is made by several experiments where a specific configuration of multiple parameters is tested: in each experiment, a configuration is applied and the results of the system under load are scored in order to identify the next most promising configuration to be tested in the next experiment, until an optimal result is reached.
Akamas AI is complemented by so-called “Optimization Packs.” Each Optimization Pack codes relevant parameters for optimizing a specific technology, as well as their internal relationships and key performance metrics. Akamas ships more than 40 Optimization Packs for key technologies such as Java, MongoDB, Kubernetes, AWS, Spark, Oracle, and more. Custom Optimization Packs can be easily created (no development required) based on the specific domain-knowledge of performance engineers.
The following figure describes the Akamas experimental approach, where full process of applying a configuration, loading the system, and measuring the results to score the experiment with respect to the defined goal is fully automated by Akamas workflows so that an entire optimization study of dozens or hundreds of experiments can be executed without any human intervention. What used to take weeks can be accomplished in hours — with the additional capability of exploring counterintuitive configurations that may provide higher service performance and resilience.
Figure: Akamas high-level architecture
The Akamas approach is completely technology-agnostic, as it does not depend on the specific technologies of the system to be optimized. Indeed, it has been successfully applied to both Java and .NET applications, both traditional/monolithic and modern/microservice applications on Kubernetes-orchestrated Docker containers, and on a variety of components such as application servers, databases, and middleware.
As illustrated by the previous figure, Akamas does not require any agents to be installed, as both performance and business KPIs can be collected from any monitoring tool that is already in place. Akamas provides out-of-the-box integrations with Tricentis NeoLoad, Micro Focus LoadRunner, Dynatrace, Prometheus, as well as influxDB repositories or simple CSV files.
An important Akamas feature is the ability to customize the scoring function with respect to any specific goal. For example, a goal could be maximizing utilization of a specific resource (e.g., memory heap) or minimizing service latency or minimizing cloud cost. In addition to goals, it is also possible to specify constraints that any optimal configuration needs to match.
For example, the optimization goal may be about reducing the resource usage while making sure that throughput is not negatively affected. This can be specified either as an absolute value or with respect to the baseline. In general, any formula on one or more technical and business metrics can be used to define a goal or a constraint so as to reflect any specific SLOs or other technical or business requirements. In order to compare the situation before and after Akamas, a very first experiment is typically run without changing the system. This is called the “baseline configuration,” as it represents the reference before any optimization is applied. The following figure illustrates how Akamas experimental approach works.
Figure: Example of Akamas experiments converging to an optimal configuration
The following figure summarizes the Akamas approach, where performance engineers only need to focus on defining their load testing scenarios and optimization goals and constraints and Akamas takes care of finding the optimal configuration automatically.
Figure: Akamas approach to performance optimization
The main outcome of an optimization study is the optimized configuration of your system. This enables better decision-making — for example, you can compare configurations and possibly also identify a better configuration (even if not optimal) that can be immediately applied (e.g., without having to restart servers). Akamas also makes it possible to apply optimal configurations automatically, to be sure that your application is always optimized. This can be done both in the context of a traditional change management process and in the case of a modern CI/CD pipeline, also thanks to Akamas API and CLI interfaces.
As a final note, it is important to clarify how Akamas approach compares to modeling approaches adopted by other solutions. In general, predefined or inference rules have been successfully applied to support root-cause analysis and other operational use cases. However, the complexity of real-world environments is very hard to be modeled in a way that is effective for application performance purposes, not just to improve the underlying infrastructure performance.
Akamas provides a native integration with Tricentis NeoLoad that allows you to easily plug Akamas into your existing environments and reuse your load testing scenarios for AI-driven tuning. Akamas integration with NeoLoad can be bi-directional: any definition of services and systems and NeoLoad configuration itself could be inherited thanks to the Akamas API and CLI interface, which allows Akamas optimization studies and workflows to be automatically defined.
As an example of how Akamas and NeoLoad can work together, let’s take the example of Konakart, an e-commerce application based on Java and Docker.
As displayed in the following figure, the several technologies and Optimization Packs were considered in the optimization study, and the integration to several telemetry providers in addition to test results and metrics provided by NeoLoad were used.
Figure: Example of Akamas and NeoLoad integration
The Akamas workflow in the following figure is structured as follows: i) first some initial steps are executed to reset the system under analysis; ii) the JVM configuration to be tested is then applied; iii) the Konakart service is started and a wait time is introduced to ensure the system is up and running; iii) finally, the load testing tasks are triggered, with the appropriate values for the NeoLoad parameters as provided by the Akamas workflow for each specific experiment.
Figure: Akamas UI showing a workflow with NeoLoad integration
The following figure shows how an Akamas study after just a few hours identified an optimal configuration that reduced the infrastructure footprint by 41% and cut average response time by 54%.
Figure: Akamas UI showing experiments to find an optimal configuration
In addition to helping to deliver higher application performance at lower costs, Akamas can also be used to improve service resiliency under high-volume and failure scenarios that can be challenging. Akamas’ goal-oriented optimization makes it possible to identify configurations that make your service more resilient and minimize unplanned downtime.
Figure: Akamas UI showing the improved level of resilience provided by the optimal solution
The new Akamas autonomous performance optimization approach allows performance engineers to find best configurations in a fraction of the time using AI and automation. Performance improves and costs are reduced, in terms of infrastructure, licensing, and cloud expenses. This approach also helps performance experts to improve their operational efficiency and gain a better understanding of the system dynamics and the underlying technologies services rely on. Finally, Akamas helps organizations become more agile in delivering quality services at business speed while also ensuring resilience even at maximum throughput and under failures.
The Akamas approach is typically used in a pre-production/testing environment, as this makes it possible to test configurations against multiple workloads and against potential failure scenarios (e.g., according to a chaos engineering approach). In particular, this requires the ability to have tests that are representative of real or potential production workloads. In this regard, Tricentis Neoload is able to provide testers with synthetic performance elements that mirror real-world traffic[4].
As more and more customers adopt an Agile and DevOps approach, Akamas is being increasingly integrated into a continuous performance optimization process and within the CI/CD toolchain. In the future Akamas is expected to be directly applied to production environments as part of an ongoing performance optimization process.
Stefano Doni
Co-founder & CTO at Akamas
Stefano is obsessed with performance optimization and drives Akamas vision for Autonomous Performance Optimization powered by AI.
Stefano has more than 15 years of experience in Performance Engineering, working on critical projects for major international enterprises and leading Moviri Research & Development team before co-founding Akamas.
He is a regular speaker at international conferences and in 2015 he won the Computer Measurement Group Best Paper Award for his contributes to capacity planning and performance optimization of Java applications.
Akamas is the AI-powered autonomous optimization solution that enables enterprises and online businesses to deliver unprecedented levels of service performance and resilience at minimum cost. Built by veterans in performance engineering and data science, Akamas exploits advanced machine learning techniques to optimize hundreds of interdependent service configuration parameters while matching both technical and business goals. Akamas customers include leading enterprise organizations in financial and online services. Akamas is a company of Moviri and counts Dynatrace, Neotys, Micro Focus, and Splunk among its partners. Headquartered in Milan, Akamas has offices in Boston, Los Angeles, and Singapore.
Visit us at www.akamas.io
