7 Artificial Intelligence Trends and How They Work With Operational Machine Learning
May 2, 2019 | 6 minute read
Nisha Talagala
As artificial intelligence (AI) becomes more prevalent and every industry races to develop AI solutions to advance their use cases, practical challenges have arisen around production deployment.
In my previous blog post [1], I described a process for taking machine learning (ML) experiments to production deployments. In this follow up post, I outline seven AI industry trends that help users simplify and scale the overall ML lifecycle. We describe each trend, discuss why it matters for operational ML and what factors should be considered as a business decides to exploit a trend to accelerate or improve their operational ML practice.
Figure 1 shows a typical machine learning (ML) lifecycle. Over time, the cycle repeats as the ML function is further optimized relative to the business need.
Trend 1: Data Marketplaces
The first challenge of many ML initiatives is finding an acceptable dataset. Data marketplaces attempt to address the shortage of datasets, particularly in key fields such as Healthcare [11] and IoT [10,] by providing platforms where individuals can share their data and companies can consume the data for AI and analytics purposes. The marketplace platform guarantees security, privacy and also provides an economic model to incentivize participants [9]. Data marketplaces can provide a rich source of otherwise unavailable data, and the marketplace can provide data provenance and lineage information that is later needed to manage the data and ensure quality.
Trend 2: Synthetic Data Services
Yet another angle to address the dataset shortage is the marketplace of synthetic datasets. Machine learning technology advances have demonstrated that ML itself can be useful to generate realistic datasets to train other ML algorithms, particularly in the deep learning space [12]. Synthetic data has been lauded for its potential to level the AI playing field for smaller companies relative to larger organizations who have access to massive data sets. Synthetic data can range from anonymized versions of actual datasets to extended datasets generated from real data samples, to simulation environments such as virtual environments used to train autonomous vehicles [13].
Trend 3: Labeling Services
Good data sets are scarce, but good datasets which are labeled are even more scarce. To address this problem, a marketplace has emerged for data labeling, focusing frequently on specific data types (such as objects in an image). Some of the labeling comes from human labelers coordinated across geographies and managed via coordination software [14]. Companies are innovating in this space with hybrids of human and ML-based labeling [15], a sub-trend that has the potential to reduce the cost of human-only labeling. Other innovations in this category include services which enable businesses to interact with labeling service providers directly [16].
Trend 4: AutoML - Automated ML models
Once a suitable dataset is found and labeled, the next challenge is finding a good algorithm and training a model. Automated machine learning (AutoML) technologies automate the process of algorithm/model selection and tuning, taking an input dataset and running a large number of training algorithms and hyperparameter options to select a final model that is recommended for deployment. Related to AutoML (and frequently provided within AutoML products) is the automation of feature engineering with technologies such as Deep Feature Synthesis [6]. AutoML software can also sometimes perform bias detection for the input dataset. Some AutoML solutions are SaaS offerings while others are downloadable software [4,5] that can run in a containerized form on cloud or on-premise environments.
Trend 5: Pre-built Containers
For those who may be developing their own model, containers are a well-established design pattern for production deployments since they enable any training or inference code to be run in a well-defined environment that is portable and scalable. Orchestration tools such as Kubernetes further enable scale and flexibility with container-based ML. However, assembling a container can be a challenging task since dependencies have to be resolved and the entire stack tuned and configured. Pre-built container marketplaces address this problem, providing pre-configured containers with necessary libraries pre-installed and configured, particularly for complex environments such as GPUs [17] .
Trend 6: Model Marketplaces
If you do not want to build or train your own model, there are model marketplaces. Model marketplaces enable customers to buy pre-built algorithms and sometimes trained models [7,8]. These can be useful for the following use cases. (a) the use case is sufficiently generic so there is no need to train a custom model and outfit training/inference code into a custom container, (b) mechanisms like Transfer Learning can be used to extend and customize base models, or (c) the user does not have enough training data to build their own model. In Model marketplaces the significant work of processing the data and training a good model can be offloaded, enabling the user to focus on the other aspects of operationalization. That said, a key challenge with model marketplaces is sifting through the content to find assets that suit your needs.
Trend 7: Application level AI Services
Finally, for common use cases that exist across businesses, application level AI services can remove the need for the entire operational ML lifecycle. Rather than creating models, training and deploying them, one can subscribe to an end service that performs the AI task. Application level AI services exist for vision, video analytics, Natural Language Processing (NLP), form processing, Natural Language Translation, Speech Recognition, Chatbots, and other tasks [18, 19].
Benefits and Considerations
All of the above trends enable users to simplify or accelerate one or more stages of the operational ML lifecycle, either by offloading, reusing pre-built items, or via automation of particular stages. Given how iterative ML processes can be (for example, training usually includes tens to hundreds of experiments), automating these processes can result in more trackable, reproducible, and manageable workflows. Outsourcing these tasks can be even easier, particularly if hardened models and algorithms (that have been tested in many environments besides your own) can be leveraged for basic tasks.
That said, there are several factors to consider before using one of these services in your environment:
Consideration 1: Applicability
Not all trends are applicable to all use cases. The most generally applicable trend is AutoML which spans a wide range of applications. Similarly, Model Marketplaces have a very wide range of models and algorithms available. Data Marketplaces and Synthetic datasets tend to be specific to classes of use cases and Pre-Built Containers can be specific to different hardware configurations (like GPUs) which then lend themselves to specific usages. Many data labeling services are also usage specific (such as for image classification and form reading), but some consulting firms do offer customized labeling services. Finally, the end-to-end AI services are extremely use case specific.
Consideration 2: AI Trust
As more ML gets deployed, ordinary human fears about black box AI systems are manifesting in trust concerns and increased regulations [20]. To benefit from AI, businesses have to consider not just the mechanics of production ML but also managing any customer and/or community concerns. Left unaddressed, these concerns can materialize in customer churn, corporate embarrassment, brand value loss, or legal risk.
Trust is a complex and expansive topic, but at its core, there is a need to understand and explain ML and feel confident that the ML is operating correctly, within expected parameters and free from malicious intrusion. In particular, the decisions made by the production ML should be explainable - i.e. a human-interpretable explanation must be provided. This is becoming needed in regulations such as the GDPR’s Right to Explanation Clause [21]. Explainability is closely tied to fairness - the need to be convinced that the AI is not accidentally or intentionally rendering biased decisions. For example, AI services such as Amazon’s Rekognition have been called out for concerns of bias [19].
Since virtually all of the trends above involve offloading or “outsourcing” some aspect of the ML lifecycle to a third party or automated system, additional awareness needs to be taken at each stage to make sure that the end production lifecycle can deliver the core tenets for trust. This includes knowing what algorithms are deployed, whether the datasets used to train them are free of bias, etc. These requirements do not change the lifecycle itself, but additional diligence needs to be paid to ensure proper lineage tracking, configuration tracking, and diagnostic reporting.
Consideration 3: Diagnosability and Operational Management
Regardless of where the components of the ML lifecycle came from, your business will be responsible for managing and maintaining the health of your ML services through their lifecycle (with the possible exception of entirely outsourced services as in Trend 7). If so, it is important that your data scientists and engineers understand the models that are being deployed, the datasets that were used to train them, and the expected safe operating parameters of those models. Since many of the services and marketplaces are nascent, there is no standardization at present. The consumer is responsible for understanding the services that they use and ensuring that the services can be adequately managed in conjunction with the rest of the lifecycle.
References
[4] https://www.h2o.ai/products/h2o-driverless-ai/
[5] https://towardsdatascience.com/introducing-ubers-ludwig-5bd275a73eda
[6] https://blog.featurelabs.com/deep-feature-synthesis/
[7] https://aws.amazon.com/marketplace/solutions/machinelearning/
[8] https://algorithmia.com/algorithms
[9] https://hackernoon.com/data-marketplaces-the-holy-grail-of-our-information-age-1211a6fec390
[11] https://www.computable.io
[12] https://synthetictrainingdata.com
[14] https://thehive.ai/hive-data
[17] https://www.nvidia.com/en-us/gpu-cloud/deep-learning-containers/
[18] https://cloud.google.com/vision/docs/beta
[21] https://iapp.org/news/a/is-there-a-right-to-explanation-for-machine-learning-in-the-gdpr/
Nisha Talagala
CEO at Pyxeda AI
am an entrepreneur and technologist in the AI space and the CEO/Co-founder of Pyxeda AI. Previously, I co-founded ParallelM and defined MLOps (Production Machine Learning and Deep Learning). MLOps is the practice for full lifecycle management of Machine Learning and AI in production. My background is in software development for distributed systems, focusing on machine learning, analytics, storage, I/O, file systems, and persistent memory. Prior to PM, I was Lead Architect/Fellow at Fusion-io (acquired by SanDisk), developing new technologies and software stacks for persistent memory, Non-Volatile Memory File System (NVMFS) and application acceleration. Before Fusion-io, I was the technology lead for server flash at Intel - heading up server platform non volatile memory technology development and partnerships and foundational work on NVM Express. Before that, I was Chief Technology Officer at Gear6, where we built clustered computing caches for high performance I/O environments. I got my PhD at UC Berkeley doing research on clusters and distributed storage. I hold 63 patents in distributed systems, networking, storage, performance, key-value stores, persistent memory and memory hierarchy optimization. I enjoy speaking at industry and academic conferences and serving on conference program committees. I am currently co-chairing USENIX OpML 2019 - the first industry conference on Operational Machine Learning. I also serve on the steering committees of both OpML and HotStorage.
Next Post