Digital Product Talks #39: From AI-built prototype to production

On the most recent episode of Digital Product Talks, Daniel asked Caroline Dollinger, Product Owner for data analytics in production at BMW, an interesting (if provocative) question on this topic: once you’ve connected all the data, do you just drop a chat window somewhere like everybody else? Or do you put it in the interfaces where the context makes sense?

It’s more than a yes or no. As someone who’s on the other side, being asked to build it, Caroline had quite an extensive contribution to the topic.

Why a chat window isn’t the AI interface most people need

Chat interfaces are where AI-supported solutions tend to start on the front end, but Caroline believes AI should move further into the background. It should assemble what the user actually needs there, instead of being another boring chat window.

Sure, in a chat window, you can just ask the AI, and it tells you everything, except the question has to be good, and you have to give the AI boundaries first: where to look, what to leave out, please don’t use the last 20 years, use the last 10 months, etc.

We’ve written before why this is not the most user-friendly solution. Prompt-based interfaces alone are insufficient, as they keep everything inside the chat history. Plus, those who are better at articulating themselves will have an easier time using these kind of interfaces – but what about everyone else?

Caroline’s comparison is to early Google: search vaguely and you got nothing back, but the better your search got, the better the answers. If you ask an AI what the colour of the sky is, it’ll say blue (or purple, because somebody once wrote that the sky goes purple at sunset). But if you ask how many iX3 are scheduled in production over the next three months, that’s a direct question with concrete data behind it. So you’ll get a proper answer.

What your AI prototype has actually proved

Putting chat interfaces aside, Caroline sees clear value in using AI to experiment, and encourages people to try things while keeping in mind what they want to achieve and where AI can help. 

When Daniel described someone with no coding background building a tool with AI, getting it to about 70 or 80%, and then hitting the wall where they can’t get any further, Caroline replied that from her position at BMW, that’s a daily discussion. A prototype like that can provide a good starting point, and there is a lot of potential there. But as she explains, any kind of application has to fit into existing systems and meet security requirements. 

She describes proof of concept as a way to explore a problem and a possible solution. AI can shorten the time it takes to discover whether that solution works, but it can also reveal that something is technically possible but offers less value than expected. At that point, you can try another iteration or move on to a different approach. When something works and delivers value, the next step is to put it on a stable footing and ensure it meets applicable requirements.

She also raises questions about wider use. Could a solution work in other BMW plants? Does it address a problem shared across locations, or is it specific to one place? Can it be standardised so others can use it too?

Delving into product discovery to reduce the risk of what you’re building is something we already touched on in the previous podcast episode. Especially now, with AI, you can build something faster than before, but what if you’re building the wrong thing? You have to know what problem the prototype addresses, who it helps, and what evidence would justify taking it further.

Why compliance and security can stop an AI-built app

Caroline describes helping people understand what their application still needs to meet security requirements and fit the wider application landscape. If she could get around compliance, security and authorisation, she says, she’d have her 70% done in a week too.

That’s the pattern she sees. Somebody gets to those 70 or 80% fast, it’s genuinely cool, and then, in her words, the “evil IT” turns up and says no, because certain rules aren’t met. Her team’s job at that point isn’t to kill it, it’s to tell people what they still have to do on top of what they’ve already built, so the thing is safe to use and fits the overall application landscape.

AI agents are applications too, and they need to be secure. A new tool could introduce a security gap or write data back into another system in a way that disrupts it. In a manufacturing environment, she warns, that could lead to a production outage. Fixing that costs money nobody budgeted for.

So these processes usually exist for a reason!

The developer skills AI-generated code still needs

Caroline describes a change in how developers receive work. Someone who previously brought an idea or a requirement can now arrive with an implemented version and ask for it to be checked.

She expects that shift to demand a strong understanding of how to build dependable systems, how applications interact and how they might be rolled out across regions. It also means understanding the project requirements and checking whether the AI’s output matches the problem being solved.

She encourages people building AI into digital products to try it, but to think carefully about what they want to achieve and the environment their solution will operate in.

And that’s a good summary of the whole discussion.

Want to hear the full conversation with Caroline Dollinger? Listen to the episode on the Digital Product Talks podcast on Spotify or Apple Podcasts.

‍

TL;DR

In a rush? Worry not, we've prepared a quick summary for you.

Does an AI prototype have to be a chat window? No. Caroline Dollinger, Product Owner for data analytics in production at BMW, says AI should move further into the background: simplify the process there, assemble what the user needs, and let the prompt come out of that.

Why do AI-built prototypes fail to make it into production? They stall because they still have to fit the existing systems and meet security, compliance and authorisation requirements.

What does an AI prototype actually prove? Whether the approach works and brings value. It can also show that something is technically possible but worth less than expected, in which case you iterate or move on.

What skills do developers need to review AI-generated code? Understanding how to build dependable systems, how applications interact and how they’d be rolled out across regions, plus knowing the requirements well enough to tell whether the output matches the problem.