Comparison of deep learning software

Last updated

The following table compares notable software frameworks, libraries and computer programs for deep learning.

Contents

Deep-learning software by name

SoftwareCreatorInitial releaseSoftware license [lower-alpha 1] Open sourcePlatformWritten inInterface OpenMP support OpenCL support CUDA support ROCm support [1] Automatic differentiation [2] Has pretrained models Recurrent nets Convolutional nets RBM/DBNs Parallel execution (multi node)Actively developed
BigDL Jason Dai (Intel)2016 Apache 2.0 YesApache SparkScalaScala, PythonNoNoYesYesYes
Caffe Berkeley Vision and Learning Center2013 BSD Yes Linux, macOS, Windows [3] C++ Python, MATLAB, C++ YesUnder development [4] YesNoYesYes [5] YesYesNo?No [6]
Chainer Preferred Networks2015 BSD Yes Linux, macOS Python Python NoNoYesNoYesYesYesYesNoYesNo [7]
Deeplearning4j Skymind engineering team; Deeplearning4j community; originally Adam Gibson2014 Apache 2.0 Yes Linux, macOS, Windows, Android (Cross-platform) C++, Java Java, Scala, Clojure, Python (Keras), Kotlin YesNo [8] Yes [9] [10] NoComputational GraphYes [11] YesYesYesYes [12] Yes
Dlib Davis King2002 Boost Software License Yes Cross-platform C++ C++, Python YesNoYesNoYesYesNoYesYesYes
Flux Mike Innes2017 MIT license Yes Linux, MacOS, Windows (Cross-platform) Julia Julia YesNoYesYes [13] YesYesNoYesYes
Intel Data Analytics Acceleration Library Intel2015 Apache License 2.0 Yes Linux, macOS, Windows on Intel CPU [14] C++, Python, Java C++, Python, Java [14] YesNoNoNoYesNoYesYes
Intel Math Kernel Library 2017 [15] and laterIntel2017 Proprietary No Linux, macOS, Windows on Intel CPU [16] C [17] Yes [18] NoNoNoYesNoYes [19] Yes [19] No
Google JAX Google2018 Apache License 2.0 Yes Linux, macOS, Windows Python Python Only on LinuxNoYesNoYesYes
Keras François Chollet2015 MIT license Yes Linux, macOS, Windows Python Python, R Only if using Theano as backendCan use Theano, Tensorflow or PlaidML as backendsYesNoYesYes [20] YesYesNo [21] Yes [22] Yes
MATLAB + Deep Learning Toolbox (formally Neural Network Toolbox) MathWorks 1992 Proprietary No Linux, macOS, Windows C, C++, Java, MATLAB MATLAB NoNoTrain with Parallel Computing Toolbox and generate CUDA code with GPU Coder [23] NoYes [24] Yes [25] [26] Yes [25] Yes [25] YesWith Parallel Computing Toolbox [27] Yes
Microsoft Cognitive Toolkit (CNTK) Microsoft Research 2016 MIT license [28] Yes Windows, Linux [29] (macOS via Docker on roadmap) C++ Python (Keras), C++, Command line, [30] BrainScript [31] (.NET on roadmap [32] )Yes [33] NoYesNoYesYes [34] Yes [35] Yes [35] No [36] Yes [37] No [38]
ML.NET MicrosoftYesWindows, Linux, macOSC#, F#Yes
Apache MXNet Apache Software Foundation2015 Apache 2.0 Yes Linux, macOS, Windows, [39] [40] AWS, Android, [41] iOS, JavaScript [42] Small C++ core library C++, Python, Julia, Matlab, JavaScript, Go, R, Scala, Perl, Clojure YesNoYesNoYes [43] Yes [44] YesYesYesYes [45] No
Neural Designer Artelnics2014 Proprietary No Linux, macOS, Windows C++ Graphical user interface YesNoYesNoAnalytical differentiationNoNoNoNoYesYes
OpenNN Artelnics2003 GNU LGPL Yes Cross-platform C++ C++ YesNoYesNo??NoNoNo?
PlaidML Vertex.AI, Intel 2017 Apache 2.0 Yes Linux, macOS, Windows Python, C++, OpenCL Python, C++ ?Some OpenCL ICDs are not recognizedNoNoYesYesYesYesYesYes
PyTorch Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan (Facebook)2016 BSD Yes Linux, macOS, Windows, Android [46] Python, C, C++, CUDA Python, C++, Julia, R [47] YesVia separately maintained package [48] [49] [50] YesYesYesYesYesYesYes [51] YesYes
Apache SINGA Apache Software Foundation 2015 Apache 2.0 Yes Linux, macOS, Windows C++ Python, C++, Java NoSupported in V1.0YesNo?YesYesYesYesYes
TensorFlow Google Brain 2015 Apache 2.0 Yes Linux, macOS, Windows, [52] [53] Android C++, Python, CUDA Python (Keras), C/C++, Java, Go, JavaScript, R, [54] Julia, Swift NoOn roadmap [55] but already with SYCL [56] supportYesYesYes [57] Yes [58] YesYesYesYesYes
Theano Université de Montréal 2007 BSD Yes Cross-platform Python Python (Keras)YesUnder development [59] YesNoYes [60] [61] Through Lasagne's model zoo [62] YesYesYesYes [63] No
Torch Ronan Collobert, Koray Kavukcuoglu, Clement Farabet2002 BSD Yes Linux, macOS, Windows, [64] Android, [65] iOS C, Lua Lua, LuaJIT, [66] C, utility library for C++/OpenCL [67] YesThird party implementations [68] [69] Yes [70] [71] NoThrough Twitter's Autograd [72] Yes [73] YesYesYesYes [64] No
Wolfram Mathematica 10 [74] and later Wolfram Research 2014 Proprietary No Windows, macOS, Linux, Cloud computing C++, Wolfram Language, CUDA Wolfram Language YesNoYesNoYesYes [75] YesYesYesYes [76] Yes
SoftwareCreatorInitial releaseSoftware license [lower-alpha 1] Open sourcePlatformWritten inInterface OpenMP support OpenCL support CUDA support ROCm support [77] Automatic differentiation [2] Has pretrained models Recurrent nets Convolutional nets RBM/DBNs Parallel execution (multi node)Actively developed
  1. 1 2 Licenses here are a summary, and are not taken to be complete statements of the licenses. Some libraries may use other libraries internally under different licenses

Comparison of compatibility of machine learning models

[ further explanation needed ]

Format nameDesign goalCompatible with other formatsSelf-contained DNN ModelPre-processing and Post-processingRun-time configuration for tuning & calibrationDNN model interconnectCommon platform
TensorFlow, Keras, Caffe, Torch, ONNX,Algorithm trainingNoNo / Separate files in most formatsNoNoNoYes
ONNX Algorithm trainingYesNo / Separate files in most formatsNoNoNoYes

See also

Related Research Articles

<span class="mw-page-title-main">OpenCV</span> Computer vision library

OpenCV is a library of programming functions mainly for real-time computer vision. Originally developed by Intel, it was later supported by Willow Garage, then Itseez. The library is cross-platform and licensed as free and open-source software under Apache License 2. Starting in 2011, OpenCV features GPU acceleration for real-time operations.

<span class="mw-page-title-main">CUDA</span> Parallel computing platform and programming model

CUDA is a proprietary and closed source parallel computing platform and application programming interface (API) that allows software to use certain types of graphics processing units (GPUs) for general purpose processing, an approach called general-purpose computing on GPUs (GPGPU). CUDA is a software layer that gives direct access to the GPU's virtual instruction set and parallel computational elements, for the execution of compute kernels.

Theano is a Python library and optimizing compiler for manipulating and evaluating mathematical expressions, especially matrix-valued ones. In Theano, computations are expressed using a NumPy-esque syntax and compiled to run efficiently on either CPU or GPU architectures.

<span class="mw-page-title-main">Deeplearning4j</span> Open-source deep learning library

Eclipse Deeplearning4j is a programming library written in Java for the Java virtual machine (JVM). It is a framework with wide support for deep learning algorithms. Deeplearning4j includes implementations of the restricted Boltzmann machine, deep belief net, deep autoencoder, stacked denoising autoencoder and recursive neural tensor network, word2vec, doc2vec, and GloVe. These algorithms all include distributed parallel versions that integrate with Apache Hadoop and Spark.

<span class="mw-page-title-main">TensorFlow</span> Machine learning software library

TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks.

<span class="mw-page-title-main">Keras</span> Neural network library

Keras is an open-source library that provides a Python interface for artificial neural networks. Keras acts as an interface for the TensorFlow library.

Apache MXNet is an open-source deep learning software framework that trains and deploys deep neural networks. It is scalable, allows fast model training, and supports a flexible programming model and multiple programming languages. The MXNet library is portable and can scale to multiple GPUs and machines. It was co-developed by Carlos Guestrin at the University of Washington.

<span class="mw-page-title-main">AMD Instinct</span> Brand name by AMD; professional GPUs for high-performance-computing, machine learning

AMD Instinct is AMD's brand of professional GPUs. It replaced AMD's FirePro S brand in 2016. Compared to the Radeon brand of mainstream consumer/gamer products, the Instinct product line is intended to accelerate deep learning, artificial neural network, and high-performance computing/GPGPU applications.

spaCy Software library

spaCy is an open-source software library for advanced natural language processing, written in the programming languages Python and Cython. The library is published under the MIT license and its main developers are Matthew Honnibal and Ines Montani, the founders of the software company Explosion.

Caffe is a deep learning framework, originally developed at University of California, Berkeley. It is open source, under a BSD license. It is written in C++, with a Python interface.

PyTorch is a machine learning framework based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella. It is free and open-source software released under the modified BSD license. Although the Python interface is more polished and the primary focus of development, PyTorch also has a C++ interface.

The Open Neural Network Exchange (ONNX) [] is an open-source artificial intelligence ecosystem of technology companies and research organizations that establish open standards for representing machine learning algorithms and software tools to promote innovation and collaboration in the AI sector. ONNX is available on GitHub.

<span class="mw-page-title-main">ROCm</span> Parallel computing platform: GPGPU libraries and application programming interface

ROCm is an Advanced Micro Devices (AMD) software stack for graphics processing unit (GPU) programming. ROCm spans several domains: general-purpose computing on graphics processing units (GPGPU), high performance computing (HPC), heterogeneous computing. It offers several programming models: HIP, OpenMP/Message Passing Interface (MPI), OpenCL.

In computer vision, SqueezeNet is the name of a deep neural network for image classification that was released in 2016. SqueezeNet was developed by researchers at DeepScale, University of California, Berkeley, and Stanford University. In designing SqueezeNet, the authors' goal was to create a smaller neural network with fewer parameters while achieving competitive accuracy.

PlaidML is a portable tensor compiler. Tensor compilers bridge the gap between the universal mathematical descriptions of deep learning operations, such as convolution, and the platform and chip specific code needed to perform those operations with good performance. Internally, PlaidML makes use of the Tile eDSL to generate OpenCL, OpenGL, LLVM, or CUDA code. It enables deep learning on devices where the available computing hardware is either not well supported or the available software stack contains only proprietary components. For example, it does not require the usage of CUDA or cuDNN on Nvidia hardware, while achieving comparable performance.

OpenVINO toolkit is a free toolkit facilitating the optimization of a deep learning model from a framework and deployment using an inference engine onto Intel hardware. The toolkit has two versions: OpenVINO toolkit, which is supported by open source community and Intel Distribution of OpenVINO toolkit, which is supported by Intel. OpenVINO was developed by Intel. The toolkit is cross-platform and free for use under Apache License version 2.0. The toolkit enables a write-once, deploy-anywhere approach to deep learning deployments on Intel platforms, including CPU, integrated GPU, Intel Movidius VPU, and FPGAs.

<span class="mw-page-title-main">DeepSpeed</span> Microsoft open source library

DeepSpeed is an open source deep learning optimization library for PyTorch. The library is designed to reduce computing power and memory use and to train large distributed models with better parallelism on existing computer hardware. DeepSpeed is optimized for low latency, high throughput training. It includes the Zero Redundancy Optimizer (ZeRO) for training models with 1 trillion or more parameters. Features include mixed precision training, single-GPU, multi-GPU, and multi-node training as well as custom model parallelism. The DeepSpeed source code is licensed under MIT License and available on GitHub.

<span class="mw-page-title-main">Horovod (machine learning)</span>

Horovod is a free and open-source software framework for distributed deep learning training using TensorFlow, Keras, PyTorch, and Apache MXNet. Horovod is hosted under the Linux Foundation AI. Horovod has the goal of improving the speed, scale, and resource allocation when training a machine learning model.

<span class="mw-page-title-main">Neural Network Intelligence</span> Microsoft open source library

NNI is a free and open-source AutoML toolkit developed by Microsoft. It is used to automate feature engineering, model compression, neural architecture search, and hyper-parameter tuning.

References

  1. ↑ "Deep Learning — ROCm 4.5.0 documentation". Archived from the original on 2022-12-05. Retrieved 2022-09-27.
  2. 1 2 Atilim Gunes Baydin; Barak A. Pearlmutter; Alexey Andreyevich Radul; Jeffrey Mark Siskind (20 February 2015). "Automatic differentiation in machine learning: a survey". arXiv: 1502.05767 [cs.LG].
  3. ↑ "Microsoft/caffe". GitHub. 30 October 2021.
  4. ↑ "Caffe: a fast open framework for deep learning". July 19, 2019 – via GitHub.
  5. ↑ "Caffe | Model Zoo". caffe.berkeleyvision.org.
  6. ↑ GitHub - BVLC/caffe: Caffe: a fast open framework for deep learning., Berkeley Vision and Learning Center, 2019-09-25, retrieved 2019-09-25
  7. ↑ Preferred Networks Migrates its Deep Learning Research Platform to PyTorch, 2019-12-05, retrieved 2019-12-27
  8. ↑ "Support for Open CL · Issue #27 · deeplearning4j/nd4j". GitHub.
  9. ↑ "N-Dimensional Scientific Computing for Java". Archived from the original on 2016-10-16. Retrieved 2016-02-05.
  10. ↑ "Comparing Top Deep Learning Frameworks". Deeplearning4j. Archived from the original on 2017-11-07. Retrieved 2017-10-31.
  11. ↑ Chris Nicholson; Adam Gibson. "Deeplearning4j Models". Archived from the original on 2017-02-11. Retrieved 2016-03-02.
  12. ↑ Deeplearning4j. "Deeplearning4j on Spark". Deeplearning4j. Archived from the original on 2017-07-13. Retrieved 2016-09-01.{{cite web}}: CS1 maint: numeric names: authors list (link)
  13. ↑ "Metalhead". FluxML. 29 October 2021.
  14. 1 2 "Intel® Data Analytics Acceleration Library (Intel® DAAL)". software.intel.com. November 20, 2018.
  15. ↑ "Intel® Math Kernel Library Release Notes and New Features". Intel.
  16. ↑ "Intel® Math Kernel Library (Intel® MKL)". software.intel.com. September 11, 2018.
  17. ↑ "Deep Neural Network Functions". software.intel.com. May 24, 2019.
  18. ↑ "Using Intel® MKL with Threaded Applications". software.intel.com. June 1, 2017.
  19. 1 2 "Intel® Xeon Phi™ Delivers Competitive Performance For Deep Learning—And Getting Better Fast". software.intel.com. March 21, 2019.
  20. ↑ "Applications - Keras Documentation". keras.io.
  21. ↑ "Is there RBM in Keras? · Issue #461 · keras-team/keras". GitHub.
  22. ↑ "Does Keras support using multiple GPUs? · Issue #2436 · keras-team/keras". GitHub.
  23. ↑ "GPU Coder - MATLAB & Simulink". MathWorks. Retrieved 13 November 2017.
  24. ↑ "Automatic Differentiation Background - MATLAB & Simulink". MathWorks. September 3, 2019. Retrieved November 19, 2019.
  25. 1 2 3 "Neural Network Toolbox - MATLAB". MathWorks. Retrieved 13 November 2017.
  26. ↑ "Deep Learning Models - MATLAB & Simulink". MathWorks. Retrieved 13 November 2017.
  27. ↑ "Parallel Computing Toolbox - MATLAB". MathWorks. Retrieved 13 November 2017.
  28. ↑ "CNTK/LICENSE.md at master · Microsoft/CNTK · GitHub". GitHub.
  29. ↑ "Setup CNTK on your machine". GitHub.
  30. ↑ "CNTK usage overview". GitHub.
  31. ↑ "BrainScript Network Builder". GitHub.
  32. ↑ ".NET Support · Issue #960 · Microsoft/CNTK". GitHub.
  33. ↑ "How to train a model using multiple machines? · Issue #59 · Microsoft/CNTK". GitHub.
  34. ↑ "Prebuilt models for image classification · Issue #140 · microsoft/CNTK". GitHub.
  35. 1 2 "CNTK - Computational Network Toolkit". Microsoft Corporation.
  36. ↑ "Restricted Boltzmann Machine with CNTK #534". GitHub, Inc. 27 May 2016. Retrieved 30 October 2023.
  37. ↑ "Multiple GPUs and machines". Microsoft Corporation.
  38. ↑ "Disclaimer". CNTK TEAM. 6 November 2021.
  39. ↑ "Releases · dmlc/mxnet". Github.
  40. ↑ "Installation Guide — mxnet documentation". Readthdocs.
  41. ↑ "MXNet Smart Device". ReadTheDocs. Archived from the original on 2016-09-21. Retrieved 2016-05-19.
  42. ↑ "MXNet.js". Github. 28 October 2021.
  43. ↑ "— Redirecting to mxnet.io". mxnet.readthedocs.io.
  44. ↑ "Model Gallery". GitHub. 29 October 2022.
  45. ↑ "Run MXNet on Multiple CPU/GPUs with Data Parallel". GitHub.
  46. ↑ "PyTorch". Dec 17, 2021.
  47. ↑ "Falbel D, Luraschi J (2023). torch: Tensors and Neural Networks with 'GPU' Acceleration". torch.mlverse.org. Retrieved 2023-11-28.
  48. ↑ "OpenCL build of pytorch: (in-progress, not useable) - hughperkins/pytorch-coriander". July 14, 2019 – via GitHub.
  49. ↑ "DLPrimitives/OpenCL out of tree backend for pytorch - artyom-beilis/pytorch_dlprim". Jan 21, 2022 – via GitHub.
  50. ↑ "OpenCL Support · Issue #488 · pytorch/pytorch". GitHub.
  51. ↑ "Restricted Boltzmann Machines (RBMs) in PyTorch". GitHub. 14 November 2022.
  52. ↑ "Install TensorFlow with pip".
  53. ↑ "TensorFlow 0.12 adds support for Windows".
  54. ↑ Allaire, J.J.; Kalinowski, T.; Falbel, D.; Eddelbuettel, D.; Yuan, T.; Golding, N. (28 September 2023). "tensorflow: R Interface to 'TensorFlow'". The Comprehensive R Archive Network. Retrieved 30 October 2023.
  55. ↑ "tensorflow/roadmap.md at master · tensorflow/tensorflow · GitHub". GitHub. January 23, 2017. Retrieved May 21, 2017.
  56. ↑ "OpenCL support · Issue #22 · tensorflow/tensorflow". GitHub.
  57. ↑ "TensorFlow". TensorFlow.
  58. ↑ "Models and examples built with TensorFlow". July 19, 2019 – via GitHub.
  59. ↑ "Using the GPU — Theano 0.8.2 documentation". Archived from the original on 2017-04-01. Retrieved 2016-01-21.
  60. ↑ "gradient – Symbolic Differentiation — Theano 1.0.0 documentation". deeplearning.net.
  61. ↑ "Automatic vs. Symbolic differentiation".
  62. ↑ "Recipes/modelzoo at master · Lasagne/Recipes · GitHub". GitHub.
  63. ↑ "Using multiple GPUs — Theano 1.0.0 documentation". deeplearning.net.
  64. 1 2 "torch/torch7". July 18, 2019 – via GitHub.
  65. ↑ "GitHub - soumith/torch-android: Torch-7 for Android". GitHub. 13 October 2021.
  66. ↑ "Torch7: A Matlab-like Environment for Machine Learning" (PDF).
  67. ↑ "GitHub - jonathantompson/jtorch: An OpenCL Torch Utility Library". GitHub. 18 November 2020.
  68. ↑ "Cheatsheet". GitHub.
  69. ↑ "cltorch". GitHub.
  70. ↑ "Torch CUDA backend". GitHub.
  71. ↑ "Torch CUDA backend for nn". GitHub.
  72. ↑ "Autograd automatically differentiates native Torch code: twitter/torch-autograd". July 9, 2019 – via GitHub.
  73. ↑ "ModelZoo". GitHub.
  74. ↑ "Launching Mathematica 10". Wolfram.
  75. ↑ "Wolfram Neural Net Repository of Neural Network Models". resources.wolframcloud.com.
  76. ↑ "Parallel Computing—Wolfram Language Documentation". reference.wolfram.com.
  77. ↑ "Deep Learning — ROCm 4.5.0 documentation". Archived from the original on 2022-12-05. Retrieved 2022-09-27.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.