In demo after demo in the past year, I’ve been shown by technology vendors how AI is so easy to build and use. Ask a virtual assistant to pull up the relevant data, select the right models, then tell the machine to show you which customers you should pitch to to reverse a quarter of slow sales. Better yet, get AI to generate the sales pitch and watch the revenue roll in.
To be fair, the message has changed of late. The hype has hit the roadblock called reality and while the technology is advancing still, it is doing so without most of their business users coming to terms with actual adoption.
Implementation is hard, as many are finding. It’s easy to set up a proof of concept with tens or hundreds of documents, or run an AI agent with well curated data that is harmonised, pointed out Philipp Herzig, chief technology officer for software giant SAP.
Try doing this on a large scale, with hundreds of thousands of documents and different user needs across different countries, and suddenly the challenge is a huge engineering problem, he told reporters today at the company’s TechEd developer event in Berlin.
One key issue is trust – for good reason, people still think that AI is just not fit for purpose in many cases. Rightly, most AI agents should not be let loose to act autonomously if they are already such poor advisors.
Hardly a day passes without AI messing things up, even in places you don’t expect. In Singapore, two lawyers were were found citing “entirely fictitious” cases hallucinated by AI today, the second such case in a month.
Other high-profile lapses are making people realise just how bad AI can be – last month, Deloitte had to refund the Australian government for a A$440,000 report that the consultancy firm had filled with errors generated by AI hallucinations.
By now, many of the issues causing such problems are clear. A lack of AI literacy is one – when people are told they no longer need experts, they forgo the rigour of careful checks. Other implementation problems are obvious too, such as poor data quality, siloed data, and AI not fitting into existing business processes.
The good news is that the technology is improving. At TechEd, for example, SAP showed not just demos of how to get AI agents up and running but also how the AI comes up with an output. For example, you can check how it shows you the credit risk of a supplier compared to others, so there is a lot more transparency involved.
This translates directly to trust, something sorely missing now. SAP’s Herzig said the company’s products are built with humans in the loop. Only when users have actually improved their work with AI will they trust the AI to act more autonomously, he noted, but that day is not here yet.
This is a view echoed increasingly by many who see trust as a big gap to close. Jason Hardy, chief technology officer for AI at data storage vendor Hitachi Vantara told me earlier last month: “While requirements will vary by industry, every organisation should start with narrow, low-risk pilots and keep humans in the loop to build trust and adoption.”
Firms rushing to catch up often take shortcuts with poor data or immature solutions, which leads to setbacks, he noted, adding that as much as 90 to 95 per cent of projects fail to move beyond pilot trials.
So, while some CEOs are happily handing down AI mandates and cutting staff to show their board and shareholders they are on the AI bandwagon, their short-term gains (especially in share price) could later come back to haunt them.
Some of them already backtracking and hiring people back. However, this should not be to simply clean up the AI work slop but also to help make the big decisions that AI is not ready to. Crucially, to take responsibility as well.
That’s because doing business with other humans involves complex interactions. An AI can predict, with a certain amount of accuracy, what a supplier or customer may do next quarter but the outcome is not predetermined. This isn’t like using a calculator to get answers that are always right.
What, for example, would you do if you were presented with two sets of possible outcomes by your favourite AI assistant? Say, if you added an external data source, such as news reports picked up by a public large language model, to your own internal data and received a different suggestion?
Which prediction should you choose? Ultimately, you are choosing because you are responsible for making the decision. AI is helping you with all the analysis but your job is dependent on you choosing the right AI suggestion.
This is why self driving cars still have a steering wheel for drivers to take over, when needed. And pilots are still needed even now when planes can fly themselves, Someone has to be responsible for that autonomy given to AI.
Similarly, if businesses want AI to succeed, they have to get humans in the loop. Build trust over time by making the AI work better with people who will take responsibility for the outcomes. Only then can AI be more autonomous, delivering the results that are expected.
