AutoAI

Last updated

Automated Artificial Intelligence (AutoAI) is a variation of the automated machine learning or AutoML technology, which extends the automation of model building towards automation of the full life cycle of a machine learning model. It applies intelligent automation to the task of building predictive machine learning models by preparing data for training and identifying the best type of model for the given data. then choosing the features or columns of data that best support the problem the model is solving. Finally, automation evaluates a variety of tuning options to reach the best result as it generates, then ranks, model-candidate pipelines. The best performing pipelines can be put into production to process new data, and deliver predictions based on the model training. [1] Automated artificial intelligence can also be applied to making sure the model doesn't have inherent bias and automating the tasks for continuous improvement of the model. Managing an AutoAI model requires frequent monitoring and updating, managed by a process known as model operations or ModelOps.

Contents

The Automated Machine Learning and Data Science (AMLDS) [2] is a small team within IBM Research, which was formed to apply techniques from artificial intelligence (AI), machine learning (ML) and data management to accelerate and optimize the creation of machine learning and data science workflows. AMLDS gets credit of driving the development of AutoAI.

Use case

A typical use case for AutoAI would be training a model to predict how customers might respond to a sales incentive. The model first gets training with actual data on how customers responded to the promotion. When the trained model presented with new data, can provide a prediction of how a new customer might respond, with a confidence score for the prediction. Prior to AutoML, data scientists had to build these predictive models by hand, testing various combinations of algorithms, then testing to see how predictions compared to actual results, whereas AutoML automated the processes of preparing the data for training, applying algorithms to process the data, and then further optimizing the results. Hence, AutoAI provides greater intelligent automation that allows for testing significantly more combinations of factors to generate model candidate pipelines that reflect and address the problem more accurately. Once built, the model evaluated for bias and updated to improve performance.

The AutoAI process

AutoAI-ml-process.png

The user initiates the process by providing a set of training data and identifying the prediction column, which sets up the problem to solve. For example, the prediction column might contain values of yes or no in response to an offered incentive. In the data pre-processing stage, AutoAI applies various algorithms, or estimators, to analyze, clean (for example, remove redundant information or impute missing data), and prepare structured raw data for machine learning (ML).

The next is automated model selection that matches the data with a model type, such as classification or regression. For example, if there are only two types of data in a prediction column, AutoAI prepares to build a binary classification model. If there is an unknowable set of answers, AutoAI prepares a regression model, which employs a distinct set of algorithms, or problem-solving transformations. AutoAI ranks after testing candidate algorithms against small sub-sets of the information, increasing the size of the subset gradually for the algorithms that turns most promising to reach at the best match. This process of iterative and incremental machine learning is what sets AutoAI apart from earlier versions of AutoML.

Feature engineering transforms the raw data into the combination that represents the problem to arrive at the best accurate prediction. Part of this process is to evaluate how data in the training data source can best support an accurate prediction using algorithms, it weights few data more important than others to achieve the desired result. AutoAI automates the consideration of various features construction options in a non-exhaustive, structured manner, meanwhile progressively maximizing the accuracy of model using reinforcement learning. This results from an optimized sequence of information and data transformations that matches the best algorithms of the step involving model selection.

Finally, AutoAI applies the hyperparameter optimization step to refine and advance the best performing model pipelines. Pipelines are model candidates, evaluated and ranked by metrics like accuracy, precision. At the end of the process, the user can review the pipelines and choose the pipeline(s) to put into production to deliver predictions on new data.

History

In August 2017, AMLDS announced that they were researching the use of automated feature engineering to eliminate guesswork in data science. [3] AMDLS members Udayan Khurana, Horst Samulowitz, Gregory Bramble, Deepak Toraga, and Peter Kirchner, along with Fatemeh Nargesian of the University of Toronto and Elias Khalil of Georgia Tech, presented their preliminary research at IJCAI that same year. [4]

Called “Learning-based Feature Engineering,” their method learned the correlations between feature distributions, target distributions, and transformations, built meta-models that used past observations to predict viable transformations, and generalized thousands of data sets spanning different domains. To address feature vectors of varied sizes, it used Quantile Sketch Array to capture the essential character of a feature. [4]

In 2018, IBM Research announced Deep Learning as a Service, which opened popular deep learning libraries such as Caffe, Torch and TensorFlow, to developers in the cloud. [5] Jean-Francois Puget, PhD, a distinguished engineer specializing machine learning (ML) and optimization at IBM, entered the competition. He found out and decided to be ready for IBM AI and data science platforms like IBM Watson. [6] In December 2018, IBM Research announced NeuNetS, a new capability that automated neural network model synthesis as part of automated AI model development and deployment. [7]

In 2020, Liu et al. proposed a method for AutoML that used the alternating direction method of multipliers (ADMM) to configure multiple stages of an ML pipeline, such as transformations, feature engineering and selection, and predictive modeling. [8] This was the first recorded time that IBM Research publicly applied the term “Auto” to machine-learning.

AutoAI: The evolution of AutoML

2019 was the year that AutoML became more widely discussed as a concept. “The Forrester New Wave™: Automation-Focused Machine Learning Solutions, Q2 2019,” evaluated AutoML solutions and found that the more powerful versions offered feature engineering. [9] A Gartner Technical Professional Advice report from August 2019 reported that, based on their research, AutoML could augment data science and machine learning. They described AutoML as the automation of data preparation, feature engineering and model engineering tasks. [10]

AutoAI is the evolution of AutoML. One of AutoAI's principal inventors, Jean-Francois Puget, PhD, describes it as automatically performing data preparation, feature engineering, machine learning algorithm selection, and hyper-parameter optimization to find the best possible machine learning model. [6] The hyper-parameter optimization algorithm used in AutoAI differs from the hyper-parameter tuning of AutoML. The algorithm, optimized for cost function evaluations such as model training and scoring which are typical in machine learning, enabling rapid convergence to a satisfactory solution despite evaluation times of each iteration being of long duration. [1]

Research scientists at IBM Research published a paper "Towards Automating the AI Operations Lifecycle", [11] which describes the advantages and available technologies for automating more of the process, with the goal of limiting the human involvement required to build, test, and maintain a machine learning application. However, some HCI researchers argue that the machine learning application and its recommendations are inevitably taken by human decision makers, thus it is impossible to eliminate human involvement in the process. [12] Rather, a more transparent and interpretable AutoAI design is the key to gain trust from human users, but such design itself is quite a challenge. [13]

Awards for AutoAI

See also

Related Research Articles

<span class="mw-page-title-main">Machine learning</span> Study of algorithms that improve automatically through experience

Machine learning (ML) is a field of study in artificial intelligence concerned with the development and study of statistical algorithms that can effectively generalize and thus perform tasks without explicit instructions. Recently, generative artificial neural networks have been able to surpass many previous approaches in performance. Machine learning approaches have been applied to large language models, computer vision, speech recognition, email filtering, agriculture and medicine, where it is too costly to develop algorithms to perform the needed tasks.

<span class="mw-page-title-main">Meta-learning (computer science)</span> Subfield of machine learning

Meta learning is a subfield of machine learning where automatic learning algorithms are applied to metadata about machine learning experiments. As of 2017, the term had not found a standard interpretation, however the main goal is to use such metadata to understand how automatic learning can become flexible in solving learning problems, hence to improve the performance of existing learning algorithms or to learn (induce) the learning algorithm itself, hence the alternative term learning to learn.

<span class="mw-page-title-main">Feature engineering</span> Extracting features from raw data for machine learning

Feature engineering or feature extraction or feature discovery is the process of extracting features from raw data. Due to deep learning networks, such as convolutional neural networks, that are able to learn features by themselves, domain-specific-based feature engineering has become obsolete for vision and speech processing.

<span class="mw-page-title-main">Glossary of artificial intelligence</span> List of definitions of terms and concepts commonly used in the study of artificial intelligence

This glossary of artificial intelligence is a list of definitions of terms and concepts relevant to the study of artificial intelligence, its sub-disciplines, and related fields. Related glossaries include Glossary of computer science, Glossary of robotics, and Glossary of machine vision.

Algorithm selection is a meta-algorithmic technique to choose an algorithm from a portfolio on an instance-by-instance basis. It is motivated by the observation that on many practical problems, different algorithms have different performance characteristics. That is, while one algorithm performs well in some scenarios, it performs poorly in others and vice versa for another algorithm. If we can identify when to use which algorithm, we can optimize for each scenario and improve overall performance. This is what algorithm selection aims to do. The only prerequisite for applying algorithm selection techniques is that there exists a set of complementary algorithms.

<span class="mw-page-title-main">Artificial intelligence in healthcare</span> Overview of the use of artificial intelligence in healthcare

Artificial intelligence in healthcare is a term used to describe the use of machine-learning algorithms and software, or artificial intelligence (AI), to copy human cognition in the analysis, presentation, and understanding of complex medical and health care data, or to exceed human capabilities by providing new ways to diagnose, treat, or prevent disease. Specifically, AI is the ability of computer algorithms to approximate conclusions based solely on input data.

<span class="mw-page-title-main">Outline of machine learning</span> Overview of and topical guide to machine learning

The following outline is provided as an overview of and topical guide to machine learning:

Industrial artificial intelligence, or industrial AI, usually refers to the application of artificial intelligence to industry. Unlike general artificial intelligence which is a frontier research discipline to build computerized systems that perform tasks requiring human intelligence, industrial AI is more concerned with the application of such technologies to address industrial pain-points for customer value creation, productivity improvement, cost reduction, site optimization, predictive analysis and insight discovery.

The cTuning Foundation is a global non-profit organization developing open-source tools and a common methodology to enable sustainable, collaborative and reproducible research in Computer science, perform collaborative optimization of realistic workloads across devices provided by volunteers, enable self-optimizing computer systems, and automate artifact evaluation at machine learning and systems conferences and journals.

In machine learning, hyperparameter optimization or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparameter is a parameter whose value is used to control the learning process. By contrast, the values of other parameters are learned.

<span class="mw-page-title-main">Explainable artificial intelligence</span> AI in which the results of the solution can be understood by humans

Explainable AI (XAI), often known as Interpretable AI, or Explainable Machine Learning (XML), either refers to an AI system over which it is possible for humans to retain intellectual oversight, or to the methods to achieve this. The main focus is usually on the reasoning behind the decisions or predictions made by the AI which are made more understandable and transparent. XAI counters the "black box" tendency of machine learning, where even the AI's designers cannot explain why it arrived at a specific decision.

<span class="mw-page-title-main">Automated machine learning</span> Process of automating the application of machine learning

Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems.

<span class="mw-page-title-main">Neural architecture search</span> Machine learning-powered structure design

Neural architecture search (NAS) is a technique for automating the design of artificial neural networks (ANN), a widely used model in the field of machine learning. NAS has been used to design networks that are on par or outperform hand-designed architectures. Methods for NAS can be categorized according to the search space, search strategy and performance estimation strategy used:

<span class="mw-page-title-main">MLOps</span> Approach to machine learning lifecycle management

MLOps or ML Ops is a paradigm that aims to deploy and maintain machine learning models in production reliably and efficiently. The word is a compound of "machine learning" and the continuous development practice of DevOps in the software field. Machine learning models are tested and developed in isolated experimental systems. When an algorithm is ready to be launched, MLOps is practiced between Data Scientists, DevOps, and Machine Learning engineers to transition the algorithm to production systems. Similar to DevOps or DataOps approaches, MLOps seeks to increase automation and improve the quality of production models, while also focusing on business and regulatory requirements. While MLOps started as a set of best practices, it is slowly evolving into an independent approach to ML lifecycle management. MLOps applies to the entire lifecycle - from integrating with model generation, orchestration, and deployment, to health, diagnostics, governance, and business metrics. According to Gartner, MLOps is a subset of ModelOps. MLOps is focused on the operationalization of ML models, while ModelOps covers the operationalization of all types of AI models.

Amazon SageMaker is a cloud based machine-learning platform that enables developers to create, train, and deploy machine-learning (ML) models on the cloud. It also enables developers to deploy ML models on embedded systems and edge-devices. SageMaker was launched in November 2017.

Artificial Intelligence for IT Operations (AIOps) is a term coined by Gartner in 2016 as an industry category for machine learning analytics technology that enhances IT operations analytics. AIOps is the acronym of "Artificial Intelligence Operations". Such operation tasks include automation, performance monitoring and event correlations among others.

<span class="mw-page-title-main">ModelOps</span>

ModelOps, as defined by Gartner, "is focused primarily on the governance and lifecycle management of a wide range of operationalized artificial intelligence (AI) and decision models, including machine learning, knowledge graphs, rules, optimization, linguistic and agent-based models". "ModelOps lies at the heart of any enterprise AI strategy". It orchestrates the model lifecycles of all models in production across the entire enterprise, from putting a model into production, then evaluating and updating the resulting application according to a set of governance rules, including both technical and business KPI's. It grants business domain experts the capability to evaluate AI models in production, independent of data scientists.

Himabindu "Hima" Lakkaraju is an Indian-American computer scientist who works on machine learning, artificial intelligence, algorithmic bias, and AI accountability. She is currently an Assistant Professor at the Harvard Business School and is also affiliated with the Department of Computer Science at Harvard University. Lakkaraju is known for her work on explainable machine learning. More broadly, her research focuses on developing machine learning models and algorithms that are interpretable, transparent, fair, and reliable. She also investigates the practical and ethical implications of deploying machine learning models in domains involving high-stakes decisions such as healthcare, criminal justice, business, and education. Lakkaraju was named as one of the world's top Innovators Under 35 by both Vanity Fair and the MIT Technology Review.

Automated decision-making (ADM) involves the use of data, machines and algorithms to make decisions in a range of contexts, including public administration, business, health, education, law, employment, transport, media and entertainment, with varying degrees of human oversight or intervention. ADM involves large-scale data from a range of sources, such as databases, text, social media, sensors, images or speech, that is processed using various technologies including computer software, algorithms, machine learning, natural language processing, artificial intelligence, augmented intelligence and robotics. The increasing use of automated decision-making systems (ADMS) across a range of contexts presents many benefits and challenges to human society requiring consideration of the technical, legal, ethical, societal, educational, economic and health consequences.

AI-assisted virtualization software is a type of technology that combines the principles of virtualization with advanced artificial intelligence (AI) algorithms. This fusion is designed to allow more efficient, dynamic, and intelligent management of virtual environments and resources. This novel technology has been employed in a range of industries, including cloud computing, healthcare, data centers, and network infrastructure, to optimize performance, resource allocation, and security protocols.

References

  1. 1 2 "AutoAI Overview". ibm.com. IBM. Retrieved 11 October 2019.
  2. ↑ "Automated Machine Learning and Data Science [AMLDS] Team". ibm.com. IBM. 25 July 2016. Retrieved 2017-08-23.
  3. ↑ "Removing the hunch in data science with AI-based automated feature engineering". ibm.com. IBM Research, Thomas J Watson Research Center. 23 August 2017. Retrieved 23 August 2017.
  4. 1 2 Khurana, Udayan; Samulowitz, Horst; Nargesian, Fatemeh; Pedapati, Tejaswini; Khalil, Elias; Bramble, Gregory; Turaga, Deepak; Kirchner, Peter. "Automated Feature Engineering for Predictive Modeling" (PDF). byu.edu. IBM Research. Archived from the original (PDF) on 2020-01-10. Retrieved 2017-12-31.
  5. ↑ Bhattacharjee, Bishwaranjan; Boag, Scott; Doshi, Chandani; Dube, Parijat; Herta, Ben; Ishakian, Vatche; Jayaram, K.R.; Khalaf, Rania; Krishna, Avesh; Bo Li, Yu; Muthusamy, Vinod; Puri, Ruchir; Ren, Yufei; Rosenberg, Florian; Seelam, Seetharami; Wang, Yandong; Zhang, Jian Ming; Zhang, Li (2017). "IBM Deep Learning Service". arXiv: 1709.05871 [cs.DC].
  6. 1 2 Delua, Julianna. "AutoAI wins AIconics Intelligent Automation Award: Meet a key inventor". ibm.com. IBM. Retrieved 25 September 2019.
  7. ↑ Malossi, Cristiano (18 December 2018). "NeuNetS: Automating Neural Network Model Synthesis for Broader Adoption of AI". ibm.com. IBM Research. Retrieved 18 December 2018.
  8. ↑ Liu, Sijia; Ram, Parikshit; Bouneffouf, Djallel; Vijaykeerthy, Deepak; Bramble, Gregory; Samulowitz, Horst; Wang, Dakuo; Conn, Andrew R.; Gray, Alexander (2019). "A Formal Method for AutoML via ADMM". arXiv: 1905.00424 [cs.LG].
  9. ↑ Carlsson, Kjell; Gualtieri, Mike. "The Forrester New Wave: Automation-Focused Machine Learning Solutions, Q2 2019". forrester.com. Forrester Research. Retrieved 28 May 2019.
  10. ↑ Sapp, Carlton. "Augment Data Science Initiatives with AutoML". gartner.com. Gartner Research. Retrieved 30 August 2019.
  11. ↑ Arnold, Matthew; Boston, Jeffrey; Desmond, Michael; Duesterwald, Evelyn; Elder, Benjamin; Murthi, Anupama; Navratil, Jiri; Reimer, Darrell (2020-03-28). "Towards Automating the AI Operations Lifecycle". arXiv: 2003.12808 [cs.LG].
  12. ↑ Wang, Dakuo; Weisz, Justin D.; Muller, Michael; Ram, Parikshit; Geyer, Werner; Dugan, Casey; Tausczik, Yla; Samulowitz, Horst; Gray, Alexander (2019-11-07). "Human-AI Collaboration in Data Science". Proceedings of the ACM on Human-Computer Interaction. 3 (CSCW): 1–24. arXiv: 1909.02309 . doi: 10.1145/3359313 . ISSN   2573-0142.
  13. ↑ Drozdal, Jaimie; Weisz, Justin; Wang, Dakuo; Dass, Gaurav; Yao, Bingsheng; Zhao, Changruo; Muller, Michael; Ju, Lin; Su, Hui (2020-03-17). "Trust in AutoML". Proceedings of the 25th International Conference on Intelligent User Interfaces. New York, NY, USA: ACM. pp. 297–307. doi: 10.1145/3377325.3377501 . ISBN   978-1-4503-7118-6.
  14. ↑ Smolaks, Max. "AIconics Awards San Francisco 2019: Winners Announced". aibusiness.com. AI Business. Retrieved 24 September 2019.
  15. ↑ "IBM Auto AI". iF WORLD DESIGN GUIDE. Retrieved 2020-04-23.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.