In association with Elastic
By now, businesses and governments know how hard it is to turn great AI concepts into working solutions that tackle everyday problems, from boosting productivity to bolstering one’s cyber defences.
One common issue from early trials in Asia-Pacific and elsewhere is the need for good-quality data and the ability to fully utilise it. Without this, all AI’s promises remain promises.
In an Elastic study of 3,200 technology leaders last year, almost all of them – 99 per cent – said generative AI (GenAI) would drive transformational change in their organisation. Think of increased employee productivity and more engaging customer experiences.
As a pioneering trailblazer, Singapore ranks high among the 10 countries surveyed, with 63 per cent of respondents saying they are implementing GenAI.
Yet, across the world, a primary concern is data quality. GenAI models rely on the data that feeds them. Organisations must have sufficient quality data to train the models, and many do not.
Nearly 75 per cent of those surveyed reported that viewing data across all environments is a key difficulty for their organisation. This slows data-based insights and prevents organisations from using their data wisely — or in GenAI models.
This is where Elastic’s Search AI Platform comes in. Search AI combines the precision of search and the intelligence of AI, helping organisations retrieve the data they need at the right time.
Pairing search with GenAI can result in high-quality search results that are accurate, current, relevant, and derived from real-time data.
The Elastic platform also ensures results and information are presented with business context, in simple language for users and customers. This way, organisations can make sense of their data and ultimately make better-informed decisions.
They can avoid the confusion and missteps caused by common GenAI problems such as hallucinations. Instead of simply making things up, as many chatbots often do, a GenAI solution powered by the Elastic Search AI platform gives more accurate answers with context and relevance.
Already, search-powered AI has made a real difference by helping IT teams go beyond the hype and deliver clear results in both the public and private sectors.
One user of the Elastic Search AI platform is a statutory board which drives Singapore’s smart nation and government transformation efforts. Last year, it created a common observability tool for government agencies to proactively monitor and resolve issues, thus making its IT infrastructure more resilient.
To do this, Elastic provides structured telemetry data that enables government-wide observability capabilities across agencies.
Elastic was also chosen because its solution was easy to integrate with the government’s unique verification and authentication methods. Perhaps more importantly, the Elastic Search AI platform makes better use of data to deliver results. This is especially important today, as many organisations emerge from initial AI trials to find that they need to use data more effectively to boost their efforts.
This is often done using retrieval augmented generation (RAG) to retrieve information from private or public databases to supplement what a large language model (LLM) delivers. By fusing the intelligence of an LLM with trusted data from private data sources in real-time, organisations can add accuracy, context and relevance to the responses powered by an LLM.
This technique complements the LLMs that generate responses based on huge amounts of general data they have been trained on with an organisation’s proprietary data, enabling the LLMs to provide more accurate and context-relevant responses.
For example, with RAG, a customer seeking help with an online purchase can receive more relevant instructions and assistance if the LLM is augmented with context, such as the date of the purchase and the customer’s location.
While many organisations look into fine-tuning or training their own LLMs with proprietary data, RAG presents a faster and less costly way to deliver better GenAI responses.
Indeed, LG CNS, a digital transformation company in Korea, has done this. Partnering with Elastic, it has used RAG to transform its global knowledge management services, enhancing search for GenAI.
By revamping its search capabilities with Elastic, LG CNS has enabled its various applications to better understand user intent and deliver search results. In addition to RAG, LG CNS also uses Elastic’s vector search capabilities.
Vector search leverages machine learning (ML) to capture the meaning and context of unstructured data, including text and images, transforming it into a numeric representation, yielding faster and more relevant results.
Instead of traditional keyword-based search, its employees can now make use of more sophisticated context-based search combined with AI. The result: A 95 per cent relevance rate with Elastic in place.
Today, LG CNS’s KeyLook AI algorithm uses GenAI models to encode corporate data and index it within Elastic. When a user enters a question on a search field, KeyLook AI identifies related documents and delivers them to an LLM.
The LLM then delivers answers to users by compiling user-friendly, easy-to-understand answers. Through this, LG CNS can also preserve the data governance of its proprietary corporate information.
As LG CNS and the Singapore statutory board have shown, search-powered AI enables organisations to deliver real results and drive more AI endeavours over time.
While it’s true many organisations face issues with data quality today, it is a problem that can be overcome with the right tools to provide better responses from AI.
After years of collecting data in a digital economy, many organisations will find that the problem is not a lack of data but rather finding a way to combine all the useful data with AI to deliver the best business outcomes.
With tools for RAG systems and strong vector database capabilities, Elastic is adding a vital search element into many of today’s AI efforts. They are already bringing the desired success that has eluded many early AI trials in the past 18 months.
Find out how your organisation can make the promise of Search AI a reality at the ElasticON conference in Singapore on March 4, 2025. Sign up here!
