How Lucidworks and Generative AI Enhance Knowledge Management

Generative AI has all of us at Lucidworks excited about the potential intersections that the technology has with the Lucidworks platform. We’ve already discussed the various ways that the Lucidworks platform can integrate with generative AI from a commerce perspective, but what about from the perspective of employee experience?

VP of Sales Engineering Brian Land walks through LucidHealth and how generative AI engages the employee experience through powering knowledge management systems. Watch the demo and read below for a brief overview:

A demonstration with clinical trials

In the previous demo video, we gave viewers a tour of a chatbot integrated with Lucidworks and how users can ask natural language questions to the system and be given meaningful results. In this demo, we apply the same approach in the context of employees engaging with knowledge management systems.

The integration of Lucidworks and generative AI has the power to allow knowledge workers to navigate extremely layered and complex data. To illustrate this, we focused on healthcare knowledge workers. First, we indexed clinicaltrials.gov – a government database containing thousands of clinical trials documents that are typically extremely long and hard to understand.

We took this clinical trial data and applied it to LucidHealth – powered by the Lucidworks platform. Of course, it all starts with the search box.

This is powered by the Lucidworks platform: the search box, typeahead, facets, and results. That said, this demo is 100% created with Google’s large language model PaLM. In our example, if we search “prostate cancer,” you’ll receive numerous clinical trial documents on the index side of the house instead of query side – saving time and costs.

Clicking on a document takes the user to this page, where a brief overview of the condition is provided as well as standard data points that one would get from Luciworks. This is a bit standard, but for our demo we’ve broken up the trial where certain elements like “description” and “participation criteria” tend to become extremely long.

This is where PaLM comes in: we’ve asked Google to take the extensive information provided in those parts of the clinical trials and distill it down to a single paragraph:

Beyond that, we also asked PaLM to build a study glossary from scratch, which was not available at all before:

Where the real power comes in

These ways of summarizing information into timesaving chunks is just one element of the power that generative AI integrating with Lucidworks provides. The bigger benefit becomes apparent when we get into asking natural language questions about the specific study in the Copilot – the dedicated space where users can ask more nuanced questions of the document or data set.

For example, when we ask what drugs were used in this study, the chatbot is going to scan the clinical trial document that is open to pull out the most relevant information to the user. This is called grounding the truth. That’s because it is not looking across large swaths of unchecked data, but instead being kept grounded to a specific document in this case or set of data so the results are constrained to a very specific data set. Another example, one could ask the copilot about side effects, the results of which are pulled from the same document instead of tasking the knowledge worker with scanning dense reading material:

The benefit to knowledge workers in this scenario is to significantly reduce costs and time on making critical decisions quickly.

Lucidworks grounds LLM responses in the truth

The key message to take away from this demo is that our Lucidworks platform is capable of taking the vast potential of LLMs and ensuring that the data they pull from and the results they share are grounded in truth. In the above demo, Lucidworks is grounding results to one instance, but the platform is capable of working with multiple layers of data and still remaining grounded in truth.

All elements of how Lucidworks supports knowledge management platforms for employee experience (such as access control, document control, verifications, etc.) all supplement conversational search, making for a seamless employee experience. On the index side, Lucidworks supports LLMs in a number of ways that build the necessary guardrails for keeping the technology grounded in truth.

Interested in learning more about what’s possible when generative AI is integrated into Lucidworks? Remember to watch Brian’s previous demo as well as take a look at our generative AI field guide for brands interested in how they should be thinking about applications for the technology.