The human brain is a marvelous machine that scientists, philosophers, and mathematicians have sought to emulate for centuries. If a human can do it—with its emotional flaws and such—couldn’t a machine do it better? If the money being shoveled into artificial intelligence (AI) startups is any indication, the answer is yes.
Gartner declared AI the top trend for 2018. And you would be hard-pressed to go to any event and not see AI getting top billing (our Activate conference included). When you throw in Machine Learning (ML), Deep Learning (DL), Natural Language Processing (NLP)—well it’s possible to assume that AI and its subsets are top of everyone’s mind.
Yet despite its ubiquity, organizations still struggle to implement AI. According to Adobe, although 31% of enterprises are expecting to add AI over the coming year, only 15% are using it today. Even so, Gartner says that by 2020, 50% of organizations will lack sufficient AI and data literacy skills to achieve business value.
If the promise is so phenomenal—why is AI such a second class citizen? We asked four AI experts to share their thoughts on what is holding organizations back. Of course data and legacy infrastructure are two of the challenges, but surprisingly, so is talent. Because while AI is allegedly supposed to replace humans—humans are still very much part of the, ahem, equation.
Data, ROI, & Scalability
I actually see AI being used quite readily in smaller tech companies. It’s usually part of their core stack as the people starting these companies are usually the AI experts or AI experts who pair with domain experts. What I have consistently witnessed is that more established companies have a much harder time adopting AI—especially AI in the context of intelligent search, machine learning, and natural language processing. A few reasons why I think adoption is slow:
1. Lack of good internal infrastructure
The basis for developing good ML and NLP models is DATA. Whether its labeled or unlabeled data or search logs, organizations need to readily have a good data store in place for data scientists to explore and build models. Creating a highly accessible data store is a heavy investment and requires lots of data engineering time to make things happen.
While a data store is one part of the story, a model deployment strategy is another. How do you get from raw data in your data store to making predictions of fraud activity on your website? This is yet another investment where companies need to ensure that there is a clear engineering path for getting your models from prototype to production.
2. Confusion at the management level on applicability of AI
What AI or machine learning can REALLY do is be kind of a black box to most product managers and decision makers. Unless you are working with it day in and day out, the concepts can seem intimidating and it’s not immediately clear how these technologies help.
This causes a general confusion as to where AI and machine learning should be used to get the best ROI on dollars invested. I’ve seen countless hours wasted on solving problems with machine learning when all it really needed was several if-else rules.
3. The hype factor
Let’s talk data-related hypes. We have seen everything from big data to data science to machine learning. We are currently in the AI and deep learning hype phase. There will always be hyped up technologies and some companies tend to get really caught up in it. They try to fit their data and automation problems into the mold of the current hype. That just doesn’t work.
You could potentially run into scalability issues due to the complexity of the new technology or the technology limits you in such a way that it works only on a handful of use cases. These are all based on true events.
Kavita Ganesan is a Senior Data Scientist at Github and holds a Ph.D in Text Mining, Analytics and Search. She has over a decade of experience in building scalable Machine Learning and NLP models for various companies she has worked for including eBay, 3M and GitHub. In 2017, Kavita led the launch of the first production scale NLP and Machine Learning pipeline at GitHub with the release of GitHub topics touching millions of repositories.
AI/Machine Learning Talent & Transparency
Artificial Intelligence has extensive, conceivable applicability. However, we’re still in the initial stages in terms of the adoption of these technologies, so there’s a long way to go.
AI has potential application across various sectors, e.g, healthcare, retail, semiconductors, etc.
However, when evaluating use of AI, business leaders have to keep in mind that this is still a very fast-evolving set of techniques and technologies.
There are confines that are purely technical. Typical questions when moving from, for example, a historically database-based system are:
1. Can we actually explain what the new AI/ML algorithm is doing?
2. Can we interpret why it’s making the selections and the consequences and forecasts that it’s making?
3. If the results are not much different, then why should I change?
And there are some real-world limitations as well. A lot of data is needed for the AI/ML algorithms to train and the data needs to be labeled. But is the data actually available? And is it labeled?
If data exists, there are still questions about:
• How clear are the algorithms?
• Is there any bias in the data?
• Is there any prejudice in the way the data was collected?
Though we have this idea of machine learning, we first have to train humans to collect and train the data and algorithms. Now companies have to find this talent pool of people who are able to do it.
And, companies have to keep an open mind. Like every new technology adaptation, things will eventually be easier and more evolved. Most companies have a lot of data, which is mostly wasted, so it’s very important to invest in prepping and cleansing the data so it can be made useful.
If you cannot hire new talent, train the current talent. The concept of AI is very human understandable. We do learn by observing the patterns in our surroundings and AI is training machines to learn from the data we feed them and reach conclusions in a way similar to a human. That means it can take examples and learn from them to increase the accurateness of its future conclusions.
I think business leaders should just start to understand the technology and what’s possible. Think of how AI works as the same way you learn what a dog looks like by looking at photographs of labradors, poodles, and pit bulls. Then later, when you’re shown a picture of a doberman, being able to identify that as another breed of dog.
Now try and understand what the implications of that type of learning could have on your organization. Consider real estate: by having the machine look at past house prices and house descriptions—it can predict the price of any house if it has the house data.
The machine can reach a new, accurate, conclusion based on what it has learned in the past.
AI/ML is widely applicable. So, understand where it can help you get value.
Anupama Joshi is the Senior Engineering Manager at Reddit, managing the search and discovery efforts from ingestion to results and infra to ranking. Expert in managing cross-disciplined teams, her organization is focused on the infrastructure and development of core ranking algorithms and ranking signals to optimize the quality of search results.
Legacy Code & Infrastructure
I think AI is extremely important for most companies right now to automate existing processes, create new innovative ones, and increase the efficiency of existing processes across the board. While the tools and talent for incorporating AI in business exists, there are some hurdles along the way.
A lot of companies have legacy code and infrastructure that are not easy to build AI into, and require a lot of investment from the business. AI algorithms are often built on top of a data layer and having easy access to reliable, structured data can be difficult. There are some difficulties around finding the right talent, picking the right tools and seeing and communicating results from AI in a reasonable amount of time across the organizations.
I think most of the difficulties around implementing AI in business stems from the difference between AI development vs classic software development.
Successful AI implementation and integration require a shift in mindset, strategy and clear communications between business and the machine learning or data science teams. In most cases, integrating AI in an organization requires an investment from the business. This is due to AI’s nature.
AI is an iterative process and in most cases immediate results are not possible before building the right underlying infrastructure and rounds of iterations and refining the input data, features, modeling, and inference.
I usually advise the AI teams to focus on building the correct data layer, infrastructure, and the simplest model, and focus on iterations involving more advanced modeling and featurization techniques afterwards. This requires clear communications with the organization and setting realistic goals and timelines.
I think organizations should define long-terms goals and vision around AI and shift away from quick wins or “fixes” mindset. Organizations can create clear goals around AI, invest in this iterative process and enable their data scientists and engineers to build what is necessary for successful implementation of AI!
Kamelia Aryafar, Ph.D., is the Chief Algorithms Officer at Overstock.com, leading the company’s ML, data science, data engineering and analytics functions across the marketing, customer, sourcing, and website verticals. Since joining Overstock.com in 2017, her teams have integrated ML and AI algorithms across various product teams, including personalization, pricing, ranking, search, recommender systems, marketing, CRM, advertising technologies, email, sourcing, and supply chain.
Human Talent & Vision Are Key
I’m one who thrives on envisioning and architecting how data, artificial intelligence, and technology can make our world a better, easier place to live. The reality is that AI systems are really hard to implement. AI is still in its infancy. Just because an AI system won against a human at a game, it doesn’t mean that it can be used in your business to drive immediate outcomes.
Building AI systems for an organization needs vision (a leader with a unique combination of business and technical strength), expertise (talent), and data (domain specific and in massive quantity)—none of these are easily available in most organizations.
Unless an organization is ready to make the investments necessary to get the above three factors in place, they will find it very hard to succeed in building and implementing AI systems.
Beena Ammanath, is the founder and CEO of nonprofit, Humans For AI Inc. and is an award-winning senior digital transformation leader with extensive global experience in Artificial Intelligence, big data, and IoT. Her knowledge spans across e-commerce, financial, marketing, telecom, retail, software products, services and industrial domains with companies such as Hewlett Packard Enterprise, GE, Thomson Reuters, British Telecom, Bank of America, e*trade and a number of Silicon Valley startups.
Want to Learn More?
Join us at Activate, the search and AI conference, where you can hear from these experts and more than 75 others, Oct 16-18, in Montreal.
Contact us today to learn how Lucidworks can help your team create powerful search and discovery applications for your customers and employees.