Where would you begin? Would you start by renting the most popular space in your city for your new restaurant? Would you buy all the appliances for the advanced kitchen? Would you hire a team of chefs to make new recipes? How can you make sure that the food that your chef cooks is liked by your customers?
Adopting AI is a similar scenario.
If you decide to adopt AI in your organization, where would you actually start? How can you make sure you get value out of AI use cases? What kind of services would you need to invest in? What kind of roadblocks would you face in your AI journey?
If these are the questions you have, this series will answer everything.
I divide this series into two phases, where each phase will have a machine learning outcome.
By the end of the last phase, you will have an architecture like a data lake that will help you in adopting AI for any kind of data.
In this first article, I will explain what AI has to offer, including the common roadblocks that organizations face while adopting AI and how you can overcome them.
Five years ago, most of the enterprises did not even think about artificial intelligence for their business. Maybe they were waiting for someone to build an end product, or for AI to get matured enough to implement in their business. If you think that I would say AI is matured enough now, I say no, it’s not! If AI was matured enough, this article would probably be written by a bot. For example, today, we can’t talk to chatbots as we talk to humans. But we can certainly ask a chatbot to raise a request on our behalf.
Why bother then? Why can’t we wait for AI to get more matured or for someone to build an end product for your business? Because if you don’t adopt these emerging technologies, someone else will. We are living in an era where startups are not just challenging the established giants, but also finding new markets, and that leaves every organization with no choice but to get in the race.
Before we get into AI adoption, let’s look what AI is all about.
What’s AI/ML/DL? — A bunch of algorithms that help us find patterns, make better decisions, and solve problems that were never possible before, thanks to data, cheaper computes, and brilliant minds!
The bigger picture of AI
Machine learning — Contains statistically inspired algorithms, which are efficient in working with flat files and uses CPU’s to compute.
Deep learning — Contains biologically inspired algorithms, which are mostly used to find complex patterns among images, videos, speech and text. Deep learning algorithms are data- and compute-hungry, so hungry that we use accelerated hardware like GPU’s to compute them.
Data science — Says how you condition data so that your algorithms could understand the data.
Data Science and ML
AI is like an advanced kitchen — algorithms are like kitchen appliances, data is like ingredients, and the data scientist/ML engineer is your chef, who can understand the appliance (algorithm) and cook a recipe (use case) for you.
Data science pipeline is an approach to making great recipes. Just like before, while making a recipe, you find your ingredients, some in the kitchen and some that need to be bought/ordered (identification of data sources). Once your ingredients are available, you would pick some of them by observing them (exploratory data analysis) – for example, maybe you want to use the larger potato for now. This is followed by the preparation, by cutting what you chose (data preparation) and then cooking it (modeling).
When a recipe is done, you evaluate it by tasting it (model evaluation). If everything is perfect, you carry on with it, or else you would add what is missing and then serve it (that’s like deployment, a big concern in the ML world).
Unlike every other technology adoption before, AI adoption comes up with two major challenges.
The first is the pace: Every week or month, a better algorithm or a small tweak pops up in the world, and that improves the accuracy of the model by a significant percent.
The second is the ubiquitous nature: AI/ML techniques find themselves everywhere we could imagine, just like how computers/software find themselves everywhere. For the first time, we are making computers understand our world rather than their world of binary. For example, in the past, our computers could never understand the world with images, videos and speech -- this means that one of the greatest inventions of mankind, the computer, was deaf and blind all these years! In addition to vision and speech, computers now have IoT sensors for things like gas, moisture, and temperature. This opens up a whole new world of opportunities (new recipes).
With this pace and ubiquitous nature, what’s AI going to do the software world?
AI will be embedded in every layer possible. All our apps will get smarter every day, and when a new use case or an efficient technique is found in the AI world, software vendors will embed those algorithms into their apps as smart features. This will eventually happen for all the applications that you are using today; all you need to do is pick a good vendor!
In order to stay ahead of your competition, you have to leverage AI as it is right now by building your own AI ecosystem — with data, data scientists/ML engineers, and services that enable both of them.
Many of you would already agree with building an AI ecosystem, but there are some common roadblocks that can be seen in AI adoption.
A study published by O’Reilly called "The State of Machine Learning Adoption in the Enterprise" (June 2018) has found that 49% of respondents are Explorers, or those who are just looking at ML; 36% are Early Adopters, those who have had models in production for 2+ years; and 15% are Sophisticated, those who have had models in production for 5+ years.
If you are wondering who the sophisticated users would be (15% of the study’s respondents), they are mostly the data-rich or cloud-born companies. The others are facing challenges in adopting AI, mostly roadblocks around the AI ecosystem — with data, data scientists/ML engineers, and services.
Let’s look at what one must do to keep up.
First — Collect valuable, good-quality data. Without the right ingredients, you can’t expect good recipes. Some enterprises started collecting every possible data point, hoping that, if not now, they will use it at some point of time in future. However, it’s very important to be aware of the value of the data to your business. Let your data scientist and business team discuss this.
Second — A small team of data scientists/ML engineers/data engineers – people who understand the AI/ML world. It would help a lot if they can also understand your business and Key Performance Indicators (KPI’s).
Third — Services that help you build things faster and securely without making you reinvent the wheel. The services that you select should bridge the gap between the data science team and your business. Why? The communication between tech specialists and executives is much more important now than ever.
They are from different worlds
How would a chef know what you or customers in your city like? There needs to be alignment between what the chef can cook and what the customers want. AI requires a similar alignment between tech and business, but there is a huge shortage of people who understand the worlds of both business and AI, because of the pace at which AI has grown.
Because of the pace at which things are progressing, we have roles opened up in industries before there are university courses or books published about them. In turn, a tech team has a lot of expectations to keep up, and the business team doesn’t have enough resources to understand tech at high levels.
Common roadblocks in adopting ML in the business world
Let’s go back to our question: If you are going to start a restaurant and serve new recipes with your ingredients, what would you need? You can hire a small team of chefs (data scientists), find the best appliances for your needs (services), and cook a new recipe (use case) with the guidance from someone who understands people’s tastes (business user). Then, you can serve it to your customers, get feedback, and, when you are ready, scale up.
In AI, especially for enterprises, it’s not about doing the most cutting-edge thing first; it’s about what your business wants. Your data is your secret sauce, and your data scientist can make something with the data, but you would want something that your business or consumers need.
How can you taste your first success with AI adoption? By adding two services: one closer to business, with a primary focus on quicker outcomes with less effort, and one for the data science team, with a primary focus on building and deployment of models.
Bridging the gap
Augmented analytics should be able to democratize AI to non-tech users with quick turnarounds in developing ML workflows and without needing to write any code.
Why? — This is a quick and easy way to leverage ML with a quick turnaround time by leveraging the business user’s expertise.
How is this possible, especially with the unprecedented pace of AI? You need a platform that has all the smart features embedded into the app; it should also help you play around with all ML algorithms.
One such service is explained here -- check out my article on End-to-End Machine Learning on Oracle Analytics Cloud.
Deploying machine learning models is a huge roadblock in the data science world right now. Along with easy deployment of models, our service would require a notebook-based environment for coding and collaboration, scalable in both compute and storage.
Why? Easily deployable ML models with a collaborative notebook-based environment save a lot of time and effort for the data science team.
One such service is explained here -- check out my article on End-to-End Machine Learning on Autonomous Data Warehouse.
These are the necessary steps to follow.
Find where your data is, and take a sample. Let the data scientist and your business user find some relevant use cases/ideas, test them on augmented analytics quickly, and show the results to your stakeholders. Even if the results aren’t as expected, don’t worry too much but accuracy/perfection at this stage. Once finalized, let the data science team build, fine-tune, and deploy the model in the ‘deploy as you build’ service.
In some use cases, you would feel that some unstructured data could help these models achieve better results. That’s our second phase -- stay tuned!!
Let me know in the comments if there are any other services that you think could help in Phase-1.
AI today is bunch of algorithms, but not yet an end product. It’s like a kitchen: it’s up to you how you make the most of your ingredients (data).
The best way is to start small and grow big with flexible cloud services. In the first phase, leverage your business user’s expertise, and find use cases that are valuable to your business. Quickly test these ideas in augmented analytics, and get the first taste of success by deploying the models on ‘deploy as you build’ type of services.