Why “shift left” is becoming a strategic imperative for scaling AI and agentic systems

In a blog post by Arun Sahu on data governance being crucial when it comes to AI the concept of shifting left was brought up. We will focus a bit more on this topic and why we argue that it will be increasingly more important to succeed with AI and Agents, but also simply succeeding with data. When this article is being written, we are living in an era where AI and Generative AI is no longer a futuristic concept but a strategic necessity. From performing roles autonomously to predicting demands, AI is reshaping how businesses operate. Yet, as organizations race to adopt these technologies, many are hitting a wall: their data pipelines keep needing fixing, data quality and trust in analytics is low, and AI solutions are either underperforming or more commonly, won’t scale or becomes only an isolated success. Findings also point towards many organizations still struggle to move towards truly deriving value from data (Data powered enterprises 2024). With increasingly mature solutions when it comes to data, DevOps and connectivity as we have with Microsoft and Microsoft Azure, the window of opportunity to start shifting left is here.

The root cause? Data that isn’t ready for AI (or analytics!)

The shift left imperative

“Shift Left” is a paradigm borrowed from software engineering that emphasizes moving critical practices—like testing, validation, and governance—closer to the source of data creation. In the context of AI and analytics, this means embedding data contracts, observability, and compliance at the point where data is generated, not downstream when problems are harder and costlier to fix.
This approach is more than a technical tweak—it’s a foundational shift in how organizations treat data. And we argue it is becoming essential for three key reasons:

1 – Fixing broken pipelines is costly and erodes trust

Most large enterprises today are stuck in a reactive loop—constantly fixing broken data pipelines and dealing with low quality or limited or non-existing data catalogues. This not only consumes valuable engineering hours but also undermines trust in AI and analytics. When data quality is inconsistent, models become unreliable, insights are questioned, and adoption stalls.

The “1-10-100 rule” illustrates this well: fixing a data issue at the source might cost $1, but the same issue costs $10 to fix midstream and $100 once it reaches production. Shift Left flips this equation by catching issues early, reducing rework, and enabling faster, more reliable AI deployment.

We get that shifting left all the way might not always be easy, and that shift left as much as possible might be closer to reality. However, we do see value in shifting left all the way to source, simply because if you have that conversation and design and set a solution, you start to define your (testable) system boundaries and potentially start establishing the foundations for a common data model meaning common language in analytics and for AI across your organization. Conversations with source teams also build a stronger culture and understanding, even if this sometimes can be painful, especially in the beginning. An example where we see this happening is together with Husqvarna and Microsoft, where we digitize factories.¹

2 – Scaling agentic AI requires systems engineering discipline

As we move toward more autonomous, agentic AI systems, the need for robust systems engineering becomes non-negotiable. These agents interact with data in real time, make decisions, and even trigger actions. Without strict boundaries and testable interfaces, the risk of unintended behavior—or even harm—skyrockets.

Shift Left introduces the concept of data contracts—formal, code-based, versioned agreements between data producers and consumers. These contracts ensure that data is structured, validated, and governed at the source, much like APIs in software. This is critical for agentic AI, where modules must operate within defined boundaries to prevent cascading failures or rogue behavior.

3 – Streaming and batch data both need governance

As previously written about, governance will still be important. Agentic systems often rely on streaming data to handle real-time transactions—think fraud detection, dynamic pricing, or autonomous operations. But batch data remains vital for training models, running analytics, and generating reports. Shift Left applies to both and you will need both a “technical” governance as well as “organizational” governance.

Sogeti’s practical approach to shifting left

We argue that implementing Shift Left doesn’t require a massive transformational project. Why? Because we think the start is as simple as a conversation with that first data source producing team; preferably with one of those sources that really matter and deliver value to your organization. On top of this, we can exemplify this from one of our clients, Husqvarna, where the team is conversing, defining and building a common data model, definitions and working with source producers to standardize protocols used and how interfaces should be (for example at least versioned changes if changes are made). There are also several examples of shifting left from Netflix, Gable, devops.com as well as writings in academia making us believe this is an older concept becoming increasingly relevant in the space of data and AI.

An example of how you could go about shifting left:

First of all, focus on value and impact, select one or a few data sources that deliver value.

When you know where to look, start with data contracts: Define schemas and expectations between teams. Use tools that enforce these contracts at ingestion or even at source, before ingestion even happens.

Embed Observability: Monitor data quality and schema drift in real time. Alert teams (or autonomous governance agents) when anomalies occur.

Automate Compliance: Tag sensitive data (e.g., PII) at the source and enforce access controls.

Integrate with CI/CD: Treat data pipelines like code. Use GitOps for versioning and rollback.

Work with users and consuming systems of the shifted left data to ensure you get it right, users and system owners are happy and they get that golden data value for real!

Train Teams: Empower domain teams with the tools and knowledge to own their data quality. Make them understand and maybe even somehow get a piece of that value their data deliver!

By doing this in a step wise, full vertical manner you can:

Lastly, we believe there is value in the craftmanship of software engineering and architecture. Thus, we are now bringing our expertise in the full-stack domain over to the data domain. Not as newbies, but rather with the notion of 1 + 1 = 3 – If we join sound software engineering practices with data engineering practices we get believe we by design build a shift-left minded workforce ready to take on the world of autonomous AI Agents at scale!

Addressing the critics

Some argue that Shift Left increases complexity and burdens developers. Others claim it’s “dead” in the age of AI tools that generate code faster than teams can validate it. These critiques highlight real challenges—but they miss the point.

Shift Left isn’t about dumping more work on developers. It’s about building testable, versioned and robust system boundaries that make AI as well as decision making safe, scalable, and trustworthy. In fact, without Shift Left, the very AI tools critics rely on will be fed poor data, leading to poor outcomes, or add speed to building more of the same downstream leading to increased complexity or simply sustaining an unsound practice longer.

Yes, there can be an upfront investment. But the long-term payoff—in agility, trust, and reduced risk—is undeniable. And we argue that it should be doable stepwise; and doing it such a way would significantly lower the upfront investment, and the cost and risk ending up as low compared to the alternative.

Shift left is needed for AI and Data to scale and really deliver

In the age of Generative AI and autonomous agents, data governance is no longer a back-office function—it’s a strategic enabler. Shift Left ensures that data is ready for AI from the moment it’s created. It reduces costs, accelerates adoption, and lays the foundation for AI systems that are not just powerful, but also safe and scalable. By leveraging modern technology stacks such as Microsoft Visual Studio and DevOps tools and infrastructure together with Microsoft Azure capabilities in the data, integration and governance spaces for example, we see a window of opportunity to open where it is possible to stepwise start to build data contracts and system boundaries that are testable, versionable and robust.

If your organization is serious about AI, it’s time to shift left.