Data and Gen AI: the story so far.

Why are we doing this—and what value do we expect to see?

One of the most important questions I ask at the beginning of any Gen AI conversation is simply: Why? Why are we doing this? What are we hoping to achieve?

It’s easy to get swept up in the momentum. Gen AI is visible, exciting, and yes—trendy. Sometimes that curiosity alone is enough to get a project moving. And to be fair, exploration can be a valid reason to engage with the technology. If the goal is to learn—genuinely to test, evaluate, and understand—that’s fine. But we have to be honest about it. Can your organization afford to explore without a clear path to value? Is there clarity on what that value might be?

Because data isn’t an experimental playground. It’s one of the most valuable resources your organization has. We’ve all heard how it’s been referred to as the new oil, or the new gold. But remember, it’s also ubiquitous. It powers nearly every system you operate. It connects to every function, every decision, every customer interaction.

So when we ask “why now?” or “why Gen AI?”, we’re really asking: what problem are we solving? What’s the desired outcome?

Is it cost reduction? That might mean less manual reporting or fewer hours spent wrangling spreadsheets. Is it insight discovery? Then we’re looking at pattern recognition, market signals, better recommendations. Maybe we’re trying to reach new customer segments, or open up new services. Maybe the goal is to reduce operational friction—not only by cutting data costs, but by understanding what we’re storing, why we’re storing it, and whether it’s still relevant or usable.

And then there’s compliance. Almost every dataset in a large organization includes fragments of personal information. You’re not just securing it; you’re responsible for it. So if you’re going to use that data to power Gen AI systems, the why needs to include a plan for governance.

In my view, this moment offers a real opportunity. We’ve reached a point where many organizations have amassed vast amounts of data—and are now asking, sometimes for the first time: Do we even need all of it anymore? If Gen AI is the spark that prompts that question, then it’s already useful.

But the real value comes when we pause and really think about the data going into the AI.

Do we really need this data anymore?

There are some organizations whose systems span multiple generations of technology, including a few machines that are the last of their kind still in operation anywhere in the world. That kind of history isn’t unusual in government institutions, state banks, or long-established financial services firms. Walk through the financial districts around Europe, and you’ll still see people physically carrying paper records between offices.

And to be clear: it works. These processes were designed for a reason. They’ve stood the test of time.

The datasets we inherited from earlier eras are filled with assumptions—about what’s worth capturing, what matters, and how people or events should be categorized. I’ve seen systems where every record includes a mandatory gender field with two options: male or female. No flexibility, no nuance. At one time, that may have seemed sufficient. But now? Does it still serve a purpose? Or is it something we’re just carrying forward because no one stopped to question it?

Even if the answer is “we might need it,” that decision comes with cost. Every field you store must be secured, maintained, audited, and eventually retired. And if it’s wrong or outdated, you now have a new problem—possibly a compliance issue, possibly a reputational one, certainly a logistical one.

Think about legacy accumulation. Do we still need to know what someone bought in the year 2000? Maybe—if you’re a historian or working on 100-year economic modeling. But if you’re a consumer brand responding to changing patterns every quarter, probably not. Think about music formats: twenty years ago, CDs were the dominant distribution method. Ten years before that, it was tape. Before that, LPs. What we store changes with what we need.

When we begin preparing for Gen AI, we’re forced to look more closely at our data estate: what’s powering our systems, what’s worth keeping, and what needs to be retired. If we treat it seriously, this can be a one-in-a-generation chance to put our data house in order. That means aligning on what’s valuable, identifying what’s redundant, and cleaning up what no longer serves the organization or the customer.

What risks are we taking by keeping too much data—and by trusting too easily?

People often assume that the more data you have, the better your AI will be. But that’s not always true. In fact, more data can mean more risk.

Just storing data has a cost—especially cold storage. And beyond the financial cost, there’s a security concern. The more places your data lives, the more surface area you expose. Even accessing just part of it could unravel more than intended. Add to that the challenge of auditing or testing new systems. Excess data can slow you down just as much as it supports you.

Consider the ethical risks of inherent bias. A lot of the large language models in use today—though it’s beginning to change—have been trained primarily on white, Anglo-Saxon, Western data. That bias fundamentally shapes how the models interpret the world, and who gets seen or understood within them.

One example that comes to mind is a large insurance company that’s been around for a couple of hundred years. A few years back, they decided to address bias in their hiring process—not by using the most advanced AI, but by applying a very simple one. They stripped names, removed university affiliations, and submitted all candidates in a standard format based on core skills and experience. They saw a measurable increase in diversity—across ethnicity, gender, and more. The organization not only won awards for it, but I believe it became better for it. The hiring pool more accurately reflected the customer base. The people building products and making decisions had broader perspectives. And the company could stand behind its practices with confidence.

That’s the kind of outcome we want to see with Gen AI—but we won’t get there without intention. We need to ask: What assumptions have we baked into our models? Whose data are we building on? Are we reproducing old systems of exclusion under a new name?

Bias is a complicated word. It often carries negative weight, but not all bias is harmful—or even avoidable. We all carry preferences and patterns shaped by experience, context, and even biology. They aren’t necessarily wrong. But they do influence how we interpret data, how we make decisions, and how we design systems meant to do both on our behalf.

Some biases are visible—linked to gender, race, or social status. Others are more hidden, in subtle reactions or long-standing defaults we don’t even realize we’re reinforcing. If we don’t examine those assumptions, they become embedded in the models we build and the insights we trust. It’s not about eliminating bias entirely. It’s about being willing to see it—and be challenged by it.

The days of AI being confined to ivory towers—or only to the data science team—are over. These systems will increasingly shape who gets hired, who gets access, who gets heard. And so the burden of thoughtful, representative design has never been higher.

Should we build or buy—and what are we responsible for either way?

Once your data is in a decent place—secure, compliant, reasonably well-organized—you reach the next decision point: Do we build something ourselves, or do we buy a solution off the shelf?

At first glance, buying often looks like the easier option. And in many cases, it might be. The landscape is expanding so quickly—open-source options, commercial APIs, cloud-native tools. But even when the tool is easy to access, it doesn’t mean the implications are simple.

If your data ends up on infrastructure shared with other clients, or if it’s being accessed or trained on elsewhere, you could be exposing your organization to major risk—commercial, legal, reputational.

And if something goes wrong—if you lose the data or break the trust of those you serve—you may be dealing with the fallout for years. That could include court cases, compliance investigations, and the kind of brand damage that’s incredibly difficult to undo.

Sometimes, especially in niche industries or specialized domains, there may not be a model out there that meets your needs. In those cases, building might not be optional—it’s the only viable route.

But it can be overwhelming: there are tens of thousands of tools and frameworks now. If it’s not one of the big, obvious choices, then where do you even start? And once you choose something, what are the implications for data portability, future compatibility, or integration with your existing architecture?

Whichever path you choose—buying, building, or both—you’ll need a plan for governance, risk, and accountability. It’s not just about the tech itself. It’s about how decisions get made, and who makes them.

What will it take to keep this going—and are we ready for what’s coming?

At some point, all Gen AI work becomes operational. The model gets deployed. The dashboard goes live. The API starts pulling answers. And that’s when the real questions begin.

Because with Gen AI, it’s not just the data that evolves. The model itself changes—sometimes subtly, sometimes significantly. Especially if you’re working with a SaaS platform or using external infrastructure, the version you work with today may behave differently tomorrow. You might ask the same question twice and get two different answers, depending on the timing and underlying changes in the model.

This kind of operational readiness needs to be part of the earliest planning conversations. Because every Gen AI investment carries not just cost and time, but also reputational risk. If you’re making decisions based on a model, you need confidence it’s still doing what you expect it to do.

That means knowing how this system fits into your day-to-day operations. Who monitors it? Where does it live? What happens when something breaks? These questions need to be part of the plan from the start.

Which brings us to ownership. You can have advisory committees and working groups, but there must be one person empowered to say yes or no. Everyone may have a stake—but someone needs to be accountable. That person collaborates with legal, procurement, technical, and business teams, but ultimately carries accountability. Not to centralize control, but to ensure alignment, continuity, and trust.

And very few organizations can do that entirely on their own. Once ownership is defined internally, the next challenge is finding the right support externally. You’ll need partners—people who can help you ask better questions, spot the gaps, and understand the difference between what worked for someone else and what will work for you. Two companies might look identical on the surface: same industry, similar use case, same tech stack. But the underlying assumptions, culture, team structures, regulatory obligations—they all shape what’s possible.

And then there’s the larger picture—the direction of regulation and public policy. With the EU AI Act now live, and similar efforts underway in the UK and elsewhere, we’re entering a new chapter in governance. Data privacy is just one part of the equation. The next questions will focus on AI itself: How will it be regulated? What will governments expect in terms of oversight, auditing, and accountability? Will knowledge be taxed as a commodity? As the cost of building and running AI systems continues to rise, planning ahead becomes even more important:

Every question you ask now shapes what becomes possible next.

Modernizing your data estate for Gen AI readiness is a complex challenge—but also a unique opportunity. Every organization will approach it differently, based on its context, constraints, and ambition.

Gen AI will keep evolving—and so will your business. The systems you build today will need to be challenged tomorrow. Success won’t come from having the perfect answer at the outset. It will come from asking the right questions early—and continuing to ask them as the landscape shifts.

Start with what matters most. Build forward from there. Stay curious.

Explore our modern data guide:
Get practical strategies for preparing your data—and your teams—for Gen AI adoption with Microsoft Fabric.