Lucidworks Smart Answers and Google DialogFlow for Chatbots

Conversational user interfaces are growing in popularity, but the limitations of chatbots can leave users frustrated. At this year’s Activate, team members from Google and Lucidworks demonstrated the capabilities of joining DialogFlow and Smart Answers to build a smarter virtual assistant.

They explained how conversational solutions such as AI-powered speech-to-text, powerful dialog flow design, and AI-powered semantic search found in DialogFlow and Smart Answers can be integrated to build an impactful self-service experience.

Watch the full video here.

Conversational AI Trends

A recent report from Gartner found that the number of new chat conversations has increased by 10 times in the last year, an increase that’s partly attributed to COVID-19 brick and mortar closures. The same report also echoed what we’ve been hearing from our own customers at Lucidworks: much of the engagement with conversational UI in the past year is centered around support questions. Steven Mierop, Lucidworks Sales Engineer, also notes that many prospective customers are considering conversational UI and chatbots as revenue drivers.

The bar for conversational applications is high, but the challenges are still great for many brands. Some common limitations in traditional chatbots include:

• Static, rules-driven workflows that are time-intensive to upkeep and unable to adapt to customers’ changing needs in real-time
• Limited domain knowledge that’s a far cry from connecting with a real person in-store or over the phone
• Non-contextual responses that fail to recognize and understand a user’s history and intent
• Difficult to integrate and scale with backend systems
• Inability to return the correct answers leaving users frustrated and endangering customer loyalty and employee productivity

Lucidworks Smart Answers Framework

A month into the pandemic, Lucidworks released Smart Answers to help customers address the growing crush of online requests. And while the functionality of the interface itself is critical, where (and how) the system is gathering information and answers is an important part of effective conversational UI. For example, answers could be found in community forums, PDFs, FAQs, or blog posts. Smart Answers is able to connect to all of these different data sources and ingest them into a search index.

After data sources are connected, NLP capabilities and other deep learning models need to be applied to extract the right answer for the user at the right time. With Smart Answers’ advanced entity extraction capabilities, the chatbot can understand key concepts that the customer is looking for to deliver the most relevant response whether it’s a form, a tracking number, or a product description. Over time the system continues to learn from customers by harnessing signals and applying those insights in real-time to increase response relevancy.

How to Deploy Lucidworks Smart Answers

Mierop explains that Smart Answers is conversational middleware that doesn’t require you to be a data scientist. Lucidworks can ship Smart Answers with supervised, unsupervised, or a pre-trained model that you can use as soon as you ingest data. Once the model is created it’s automatically uploaded to Lucidworks machine learning service, making it immediately available in the index and query pipelines. After that, documents and question/answer pairs are encoded into deep vector representations — so anything coming through the pipelines is indexed and used to help train and tune the model further.

One thing Smart Answers doesn’t ship with is a framework. That’s where Google DialogFlow comes in, a framework with a friendly set of APIs to connect to.

How DialogFlow Works

DialogFlow is an integrated conversational platform from Google. According to Amit Kumar, Google DialogFlow Solution Consultant, Cloud AI, relies on some of the same services that other Google products use, including speech-to-text, text-to-speech, knowledge service for supporting knowledge bases like FAQs, plus NLP capabilities and sentiment.

Kumar shares that DialogFlow allows companies to build chatbots faster with pre-built templates that can be easily configured and tuned, engage more efficiently with users across all channels including chat and voice, and maximize reach by being built once and deployed everywhere with one-click integrations across multiple channels.

See more key highlights here:

Watch the full video here to see Kumar’s demo of DialogFlow, including how to tune intent and entity recognition and explore how users are interacting with your virtual agent or chatbot.

DialogFlow Provides Framework, Smart Answers Brings Semantic Understanding

DialogFlow comes with text-to-speech and speech-to-text translation that can be used on a variety of interfaces, including your cell phone. Mierop walked the audience through a demo of DialogFlow and Smart Answers in action using his own voice. He gives a couple examples of asking semantically the same question but uses different words. The interface passes the question through Fusion on the back-end and is able to deliver the same, correct answer in both instances even though he used words that aren’t necessarily in the search index. Additionally, Mierop demonstrates how to send positive feedback to Fusion as another way for the model to learn what the most relevant responses are to a given query.

Combining a powerful dialog front-end and scalable AI-based information retrieval back-end results in a more efficient chatbot that better serves users. Interested in learning more about the integration? Contact us today by filling out the form below.