I hear it more and more often in conversations with managers, product owners and architects: “We want something with AI.”
Often the underlying need is straightforward: prepare a report sooner, stop retyping data or complete a case file. My first question is where the work gets stuck. Are the rules unclear, is information missing, or does interpreting it take too much time? The answer helps distinguish a process problem from a task that might benefit from automation or AI.
The confusion begins with the word AI
“AI” has become a catch-all term. It can mean, for example:
- a model that interprets information, such as text or documents
- help with developing software
- an agent that carries out actions
If you call all of that “AI”, you quickly end up in an unhelpful debate. It starts to seem as if you have to be for or against AI, while the real question is which problem you are solving and which approach suits it best.
AI in the application or in its development
Two different choices often get mixed together. AI can be part of the application itself, interpreting documents or drafting recommendations. In that case, the model’s quality and limitations remain relevant whenever the application is used.
Alternatively, a team can use AI to help write code, propose tests or prepare documentation. The resulting software may contain no AI at all. The main questions then concern the quality of the development work and how the team checks it. Either approach can be useful. Any time saving needs to be measured in the actual work.
Measurement matters. In a 2025 study, experienced open-source developers took longer with AI assistance in the setting studied, despite expecting to save time. In February 2026, METR reported indications that the effect had changed, alongside selection problems that made the size of the effect difficult to estimate reliably. Neither result provides a fixed productivity multiplier for another team.
Why AI gets used where automation would do
There are a few recognisable reasons. The AI label can make it easier to secure a budget. Data problems, such as missing definitions or scattered sources, remain even when a model summarises the result neatly. And a persuasive text about the figures is not the same as an analysis you can check.
Choose an approach for each part of the task
Use rules and calculations where the required behaviour can be specified clearly. Examples include calculating totals, checking mandatory fields and routing a request against agreed conditions.
Consider AI where interpretation is needed, such as working with text, documents or patterns in data. Test it on representative examples and establish which errors occur and whether someone can detect them in time.
A combined approach may be appropriate. A model proposes information or a response; software performs verifiable calculations and permitted actions; a person reviews exceptions and consequential decisions. Divide the work according to the task and its risks. Whether the solution is called AI makes no difference.
Three examples
Reporting. A monthly report needs figures that can be traced to agreed sources and definitions. AI can help describe variances or suggest questions about the numbers. It should not silently fill in missing amounts or explanations. Keep the calculations, source data and assumptions visible.
Data analysis. For segments, trends and key figures, conventional analytics is usually the foundation. AI can help formulate hypotheses more quickly and put insights into words. You then test those hypotheses with the usual analysis.
Process improvement. If the aim is fewer handovers, less waiting time and less rework, standardisation, a clear workflow, integrations and clear responsibilities often go a long way. Whether AI adds anything depends on the task, the available data and the cost of an error.
Start with one process
Take one process and separate the work: what can follow fixed rules, where is interpretation needed, and which judgements should remain with a person? That gives you something specific to test without turning the entire process into an AI project.
Want to assess one of your own processes? Get in touch.
Sources
- METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (10 July 2025)
- METR: We are Changing our Developer Productivity Experiment Design (24 February 2026)
- Further reading, from a supplier: KNIME: Traditional Automation vs. Generative AI vs. Agentic AI: When to Use Which