After jumping onto the AI bandwagon in the past year or so, many businesses in Asia-Pacific are finding out about the less exciting aspects of the technology that have to be addressed to fully tap its potential.
Managing data effectively and making it easy to access are two key challenges that are now coming into focus, as businesses seek to scale up from their early, experimental efforts, according to experts.
Data management is one of the biggest challenges in advancing AI development, said Matthew Hardman, chief technology officer for Asia-Pacific at Hitachi Vantara.
“While an organisation’s data enables them to uniquely differentiate their AI experience with customers, they need to make their data accessible but also protect it to maintain competitive advantage,” he added.
Having an intelligent data infrastructure is also key to AI success, according to research firm IDC.
Organisations that are the most mature in their use of AI – called AI Masters – will have data infrastructure that offers easy access to the data without cumbersome preparation or preprocessing, it adds.
“Infrastructure decisions made during the design and planning process of AI Initiatives must factor in architecture flexibility,” said Ritu Jyoti, a group vice president for worldwide artificial intelligence and automation research practice, at IDC.
The easy access to distributed and varied data—both structured and unstructured data sets with varying characteristics—is crucial given the dynamic nature of data inputs to AI and generative AI workstreams.
“This requires a flexible, unified approach to storage, a common control plane, and management tools that make it seamless for data scientists and developers to consume data with MLOps [machine learning operations] integrations,” said Jyoti.
AI Masters are likely to have instant availability of their structured and unstructured data, and can seamlessly integrate their organisation’s private data with AI cloud services.
In contrast, up to 20 per cent of AI initiatives are likely to fail without an intelligent data infrastructure, according to an IDC survey.
Organisations also need to navigate through the hype surrounding AI and recognise the practical use cases relevant to their operations, said Hartman.
“Many organisations are in a honeymoon phase, feeling that AI can solve every problem they might have, but they need to recognise that like any emergent technology or trend, value comes from the experiences or failures they encounter,” he noted.
“Over time, organisations will have a better understanding of the use cases they can address with AI,” he added.
Ensuring data quality
With data being a key part of AI, there are, understandably, concerns regarding data quality, biases, and intellectual property rights.
Some organisations lack sufficient data to train AI models, and synthetic data – information that is artificially manufactured instead of data produced by real-world events – has emerged as a key way to address data scarcity.
However, Hardman stressed the necessity of balancing synthetic and real-world data to ensure the accuracy and integrity of AI models, highlighting the ethical and legal implications inherent in synthetic data generation.
Besides that, speed will be crucial in a race to make the most out of AI in future. Businesses that can swiftly ingest and derive insights from their data are expected to be more compeitive.
However, this presents another issue. As an organisation’s datasets grow larger and more complex, they become more difficult to move—a concept known as data gravity.
In turn, if a large mass of data is being generated or aggregated in a single location, it makes sense to move the application to where the data is being generated. This means a move of the data might be necessary.
Hardman emphasised the importance of processing data in close proximity to its source for rapid decision-making, highlighting the significance of proximity to data sources in enhancing operational agility.
“Competitive advantage will be based on the speed you can plan, and proximity to the data source is key to this speed,” he noted.
“If you thought about it like oil, you wouldn’t move the oil to where the pumps are,” he added. “In fact, you would move the pumps to where the oil is.”
