How NLP and Deep Learning Make Question Answering Systems Work

So far our second season of Lucidworks has looked at NLP vs NLU, Learning to Rank, neural IR search. and query intent classification.  With our penultimate episode of season two we are taking a look at the ins and outs of question answering systems. Watch now:

Search engines, chatbots, and smart devices are getting better at going beyond the “10 blue links” and giving us an exact answer to our question. This technology is officially known as a question answering system.

There are two types of QA systems, open and closed. A system that tries to answer any question you could possibly ask is called an open system or open domain system – think of Google or Alexa (“What is Prince’s birth name?”). And then there are closed (or closed domain) systems that are built for a certain subject or function or domain of knowledge or company (“I need to change my flight.”). Other closed systems includes a company’s FAQs system, knowledge base search, or chatbot.

But users don’t always ask clearly phrased questions the first time. When this happens, the computer needs to be a little smarter and try to infer the user’s intent using deep learning and neural IR search or even ask clarifying questions to better find an answer to the questions posed.

You can binge-watch all of season two or jump on back to watch season one.

Subscribe to the Lucid Thoughts channel and be sure to leave your questions and comments on each video.