CatBoost

Last updated
CatBoost
Original author(s) Andrey Gulin: [1] / Yandex
Developer(s) Yandex and CatBoost Contributors [2]
Initial releaseJuly 18, 2017;7 years ago (2017-07-18) [3] [4]
Stable release
1.2.3 [5] / February 23, 2024;8 months ago (2024-02-23)
Written in Python, R, C++, Java
Operating system Linux, macOS, Windows
Type Machine learning
License Apache License 2.0
Website catboost.ai

CatBoost [6] is an open-source software library developed by Yandex. It provides a gradient boosting framework which, among other features, attempts to solve for categorical features using a permutation-driven alternative to the classical algorithm. [7] It works on Linux, Windows, macOS, and is available in Python, [8] R, [9] and models built using CatBoost can be used for predictions in C++, Java, [10] C#, Rust, Core ML, ONNX, and PMML. The source code is licensed under Apache License and available on GitHub. [6]

Contents

InfoWorld magazine awarded the library "The best machine learning tools" in 2017. [11] along with TensorFlow, Pytorch, XGBoost and 8 other libraries.

Kaggle listed CatBoost as one of the most frequently used machine learning (ML) frameworks in the world. It was listed as the top-8 most frequently used ML framework in the 2020 survey [12] and as the top-7 most frequently used ML framework in the 2021 survey. [13]

As of April 2022, CatBoost is installed about 100000 times per day from PyPI repository [14]

Features

CatBoost has gained popularity compared to other gradient boosting algorithms primarily due to the following features [15]

History

In 2009 Andrey Gulin developed MatrixNet, a proprietary gradient boosting library that was used in Yandex to rank search results. Since 2009 MatrixNet has been used in different projects in Yandex, including recommendation systems and weather prediction.

In 2014–2015 Andrey Gulin with a team of researchers has started a new project called Tensornet that was aimed at solving the problem of "how to work with categorical data". It resulted in several proprietary Gradient Boosting libraries with different approaches to handling categorical data.

In 2016 Machine Learning Infrastructure team led by Anna Dorogush started working on Gradient Boosting in Yandex, including Matrixnet and Tensornet. They implemented and open-sourced the next version of Gradient Boosting library called CatBoost, which has support of categorical and text data, GPU training, model analysis, visualisation tools.

CatBoost was open-sourced in July 2017 and is under active development in Yandex and the open-source community.

Application

See also

Related Research Articles

<span class="mw-page-title-main">OpenCV</span> Computer vision library

OpenCV is a library of programming functions mainly for real-time computer vision. Originally developed by Intel, it was later supported by Willow Garage, then Itseez. The library is cross-platform and licensed as free and open-source software under Apache License 2. Starting in 2011, OpenCV features GPU acceleration for real-time operations.

<span class="mw-page-title-main">Orange (software)</span> Open-source data analysis software

Orange is an open-source data visualization, machine learning and data mining toolkit. It features a visual programming front-end for exploratory qualitative data analysis and interactive data visualization.

scikit-learn Python library for machine learning

scikit-learn is a free and open-source machine learning library for the Python programming language. It features various classification, regression and clustering algorithms including support-vector machines, random forests, gradient boosting, k-means and DBSCAN, and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy. Scikit-learn is a NumFOCUS fiscally sponsored project.

mlpack

mlpack is a free, open-source and header-only software library for machine learning and artificial intelligence written in C++, built on top of the Armadillo library and the ensmallen numerical optimization library. mlpack has an emphasis on scalability, speed, and ease-of-use. Its aim is to make machine learning possible for novice users by means of a simple, consistent API, while simultaneously exploiting C++ language features to provide maximum performance and maximum flexibility for expert users. mlpack has also a light deployment infrastructure with minimum dependencies, making it perfect for embedded systems and low resource devices. Its intended target users are scientists and engineers.

<span class="mw-page-title-main">Torch (machine learning)</span> Deep learning software

Torch is an open-source machine learning library, a scientific computing framework, and a scripting language based on Lua. It provides LuaJIT interfaces to deep learning algorithms implemented in C. It was created by the Idiap Research Institute at EPFL. Torch development moved in 2017 to PyTorch, a port of the library to Python.

Eclipse Deeplearning4j is a programming library written in Java for the Java virtual machine (JVM). It is a framework with wide support for deep learning algorithms. Deeplearning4j includes implementations of the restricted Boltzmann machine, deep belief net, deep autoencoder, stacked denoising autoencoder and recursive neural tensor network, word2vec, doc2vec, and GloVe. These algorithms all include distributed parallel versions that integrate with Apache Hadoop and Spark.

<span class="mw-page-title-main">TensorFlow</span> Machine learning software library

TensorFlow is a software library for machine learning and artificial intelligence. It can be used across a range of tasks, but is used mainly for training and inference of neural networks. It is one of the most popular deep learning frameworks, alongside others such as PyTorch and PaddlePaddle. It is free and open-source software released under the Apache License 2.0.

The following tables compare notable software frameworks, libraries, and computer programs for deep learning applications.

<span class="mw-page-title-main">XGBoost</span> Gradient boosting machine learning library

XGBoost is an open-source software library which provides a regularizing gradient boosting framework for C++, Java, Python, R, Julia, Perl, and Scala. It works on Linux, Microsoft Windows, and macOS. From the project description, it aims to provide a "Scalable, Portable and Distributed Gradient Boosting Library". It runs on a single machine, as well as the distributed processing frameworks Apache Hadoop, Apache Spark, Apache Flink, and Dask.

<span class="mw-page-title-main">Keras</span> Neural network library

Keras is an open-source library that provides a Python interface for artificial neural networks. Keras was first independent software, then integrated into the TensorFlow library, and later supporting more. "Keras 3 is a full rewrite of Keras [and can be used] as a low-level cross-framework language to develop custom components such as layers, models, or metrics that can be used in native workflows in JAX, TensorFlow, or PyTorch — with one codebase." Keras 3 will be the default Keras version for TensorFlow 2.16 onwards, but Keras 2 can still be used.

Caffe is a deep learning framework, originally developed at University of California, Berkeley. It is open source, under a BSD license. It is written in C++, with a Python interface.

PyTorch is a machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella. It is one of the most popular deep learning frameworks, alongside others such as TensorFlow and PaddlePaddle, offering free and open-source software released under the modified BSD license. Although the Python interface is more polished and the primary focus of development, PyTorch also has a C++ interface.

<span class="mw-page-title-main">ROCm</span> Parallel computing platform: GPGPU libraries and application programming interface

ROCm is an Advanced Micro Devices (AMD) software stack for graphics processing unit (GPU) programming. ROCm spans several domains: general-purpose computing on graphics processing units (GPGPU), high performance computing (HPC), heterogeneous computing. It offers several programming models: HIP, OpenMP, and OpenCL.

<span class="mw-page-title-main">ML.NET</span> Machine learning library

ML.NET is a free software machine learning library for the C# and F# programming languages. It also supports Python models when used together with NimbusML. The preview release of ML.NET included transforms for feature engineering like n-gram creation, and learners to handle binary classification, multi-class classification, and regression tasks. Additional ML tasks like anomaly detection and recommendation systems have since been added, and other approaches like deep learning will be included in future versions.

<span class="mw-page-title-main">Dask (software)</span> Python library for parallel computing

Dask is an open-source Python library for parallel computing. Dask scales Python code from multi-core local machines to large distributed clusters in the cloud. Dask provides a familiar user interface by mirroring the APIs of other libraries in the PyData ecosystem including: Pandas, scikit-learn and NumPy. It also exposes low-level APIs that help programmers run custom algorithms in parallel.

Kubeflow is an open-source platform for machine learning and MLOps on Kubernetes introduced by Google. The different stages in a typical machine learning lifecycle are represented with different software components in Kubeflow, including model development (Kubeflow Notebooks), model training (Kubeflow Pipelines,Kubeflow Training Operator), model serving (KServe), and automated machine learning (Katib).

<span class="mw-page-title-main">DeepSpeed</span> Microsoft open source library

DeepSpeed is an open source deep learning optimization library for PyTorch.

<span class="mw-page-title-main">Neural Network Intelligence</span> Microsoft open source library

NNI is a free and open-source AutoML toolkit developed by Microsoft. It is used to automate feature engineering, model compression, neural architecture search, and hyper-parameter tuning.

LightGBM, short for Light Gradient-Boosting Machine, is a free and open-source distributed gradient-boosting framework for machine learning, originally developed by Microsoft. It is based on decision tree algorithms and used for ranking, classification and other machine learning tasks. The development focus is on performance and scalability.

CuPy is an open source library for GPU-accelerated computing with Python programming language, providing support for multi-dimensional arrays, sparse matrices, and a variety of numerical algorithms implemented on top of them. CuPy shares the same API set as NumPy and SciPy, allowing it to be a drop-in replacement to run NumPy/SciPy code on GPU. CuPy supports Nvidia CUDA GPU platform, and AMD ROCm GPU platform starting in v9.0.

References

  1. ↑ "Andrey Gulin - People - Research at Yandex". research.yandex.com.
  2. ↑ "catboost/catboost". GitHub.
  3. ↑ "Yandex open sources CatBoost, a gradient boosting machine learning library". TechCrunch. 18 July 2017. Retrieved 2020-08-30.
  4. ↑ Yegulalp, Serdar (2017-07-18). "Yandex open sources CatBoost machine learning library". InfoWorld. Retrieved 2020-08-30.
  5. ↑ "Releases · catboost/catboost". GitHub. Retrieved 2024-03-14.
  6. 1 2 "catboost/catboost". August 30, 2020 – via GitHub.
  7. 1 2 Prokhorenkova, Liudmila; Gusev, Gleb; Vorobev, Aleksandr; Dorogush, Anna Veronika; Gulin, Andrey (2019-01-20). "CatBoost: unbiased boosting with categorical features". arXiv: 1706.09516 [cs.LG].
  8. ↑ "Python Package Index PYPI: catboost" . Retrieved 2020-08-20.
  9. ↑ "Conda force package catboost-r" . Retrieved 2020-08-30.
  10. ↑ "Maven Repository: ai.catboost » catboost-prediction". mvnrepository.com. Retrieved 2020-08-30.
  11. ↑ staff, InfoWorld (27 September 2017). "Bossie Awards 2017: The best machine learning tools". InfoWorld.
  12. ↑ "State of Data Science and Machine Learning 2020".
  13. ↑ "State of Data Science and Machine Learning 2021".
  14. ↑ "PyPI Stats catboost". PyPI Stats.
  15. ↑ Joseph, Manu (2020-02-29). "The Gradient Boosters V: CatBoost". Deep & Shallow. Retrieved 2020-08-30.
  16. ↑ Dorogush, Anna Veronika; Ershov, Vasily; Gulin, Andrey (2018-10-24). "CatBoost: gradient boosting with categorical features support". arXiv: 1810.11363 [cs.LG].
  17. ↑ "CatBoost Enables Fast Gradient Boosting on Decision Trees Using GPUs". NVIDIA Developer Blog. 2018-12-13. Retrieved 2020-08-30.
  18. ↑ "Code Completion, Episode 4: Model Training". JetBrains Developer Blog. 2021-08-20.
  19. ↑ "Stop the Bots: Practical Lessons in Machine Learning". The Cloudflare Blog. 2019-02-20.
  20. ↑ "How Careem's Destination Prediction Service speeds up your ride". Careem. 2019-02-19.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.