Torch (machine learning)

Last updated
Torch
Original author(s) Ronan Collobert, Samy Bengio, Johnny Mariéthoz [1]
Initial releaseOctober 2002;21 years ago (2002-10) [1]
Final release
7.0 / February 27, 2017;6 years ago (2017-02-27) [2]
Repository
Written in Lua, C, C++
Operating system Linux, Android, Mac OS X, iOS
Type Library for machine learning and deep learning
License BSD License
Website torch.ch

Torch is an open-source machine learning library, a scientific computing framework, and a scripting language based on Lua. [3] It provides LuaJIT interfaces to deep learning algorithms implemented in C. It was created by the Idiap Research Institute at EPFL. Torch development moved in 2017 to PyTorch, a port of the library to Python. [4] [5] [ better source needed ]

Contents

torch

The core package of Torch is torch. It provides a flexible N-dimensional array or Tensor, which supports basic routines for indexing, slicing, transposing, type-casting, resizing, sharing storage and cloning. This object is used by most other packages and thus forms the core object of the library. The Tensor also supports mathematical operations like max, min, sum, statistical distributions like uniform, normal and multinomial, and BLAS operations like dot product, matrix–vector multiplication, matrix–matrix multiplication and matrix product.

The following exemplifies using torch via its REPL interpreter:

>a=torch.randn(3,4)>=a-0.2381-0.3401-1.7844-0.26150.14111.62490.17080.8299-1.04342.22911.05250.8465[torch.DoubleTensorofdimension3x4]>a[1][2]-0.34010116549482>a:narrow(1,1,2)-0.2381-0.3401-1.7844-0.26150.14111.62490.17080.8299[torch.DoubleTensorofdimension2x4]>a:index(1,torch.LongTensor{1,2})-0.2381-0.3401-1.7844-0.26150.14111.62490.17080.8299[torch.DoubleTensorofdimension2x4]>a:min()-1.7844365427828

The torch package also simplifies object-oriented programming and serialization by providing various convenience functions which are used throughout its packages. The torch.class(classname, parentclass) function can be used to create object factories (classes). When the constructor is called, torch initializes and sets a Lua table with the user-defined metatable, which makes the table an object.

Objects created with the torch factory can also be serialized, as long as they do not contain references to objects that cannot be serialized, such as Lua coroutines, and Lua userdata. However, userdata can be serialized if it is wrapped by a table (or metatable) that provides read() and write() methods.

nn

The nn package is used for building neural networks. It is divided into modular objects that share a common Module interface. Modules have a forward() and backward() method that allow them to feedforward and backpropagate, respectively. Modules can be joined using module composites, like Sequential, Parallel and Concat to create complex task-tailored graphs. Simpler modules like Linear, Tanh and Max make up the basic component modules. This modular interface provides first-order automatic gradient differentiation. What follows is an example use-case for building a multilayer perceptron using Modules:

>mlp=nn.Sequential()>mlp:add(nn.Linear(10,25))-- 10 input, 25 hidden units>mlp:add(nn.Tanh())-- some hyperbolic tangent transfer function>mlp:add(nn.Linear(25,1))-- 1 output>=mlp:forward(torch.randn(10))-0.1815[torch.Tensorofdimension1]

Loss functions are implemented as sub-classes of Criterion, which has a similar interface to Module. It also has forward() and backward() methods for computing the loss and backpropagating gradients, respectively. Criteria are helpful to train neural network on classical tasks. Common criteria are the Mean Squared Error criterion implemented in MSECriterion and the cross-entropy criterion implemented in ClassNLLCriterion. What follows is an example of a Lua function that can be iteratively called to train an mlp Module on input Tensor x, target Tensor y with a scalar learningRate: 

functiongradUpdate(mlp,x,y,learningRate)localcriterion=nn.ClassNLLCriterion()pred=mlp:forward(x)localerr=criterion:forward(pred,y);mlp:zeroGradParameters();localt=criterion:backward(pred,y);mlp:backward(x,t);mlp:updateParameters(learningRate);end

It also has StochasticGradient class for training a neural network using Stochastic gradient descent, although the optim package provides much more options in this respect, like momentum and weight decay regularization.

Other packages

Many packages other than the above official packages are used with Torch. These are listed in the torch cheatsheet. [6] These extra packages provide a wide range of utilities such as parallelism, asynchronous input/output, image processing, and so on. They can be installed with LuaRocks, the Lua package manager which is also included with the Torch distribution.

Applications

Torch is used by the Facebook AI Research Group, [7] IBM, [8] Yandex [9] and the Idiap Research Institute. [10] Torch has been extended for use on Android [11] [ better source needed ] and iOS. [12] [ better source needed ] It has been used to build hardware implementations for data flows like those found in neural networks. [13]

Facebook has released a set of extension modules as open source software. [14]

See also

Related Research Articles

In computing, serialization is the process of translating a data structure or object state into a format that can be stored or transmitted and reconstructed later. When the resulting series of bits is reread according to the serialization format, it can be used to create a semantically identical clone of the original object. For many complex objects, such as those that make extensive use of references, this process is not straightforward. Serialization of object-oriented objects does not include any of their associated methods with which they were previously linked.

<span class="mw-page-title-main">OpenCV</span> Computer vision library

OpenCV is a library of programming functions mainly for real-time computer vision. Originally developed by Intel, it was later supported by Willow Garage, then Itseez. The library is cross-platform and licensed as free and open-source software under Apache License 2. Starting in 2011, OpenCV features GPU acceleration for real-time operations.

<span class="mw-page-title-main">Orange (software)</span>

Orange is an open-source data visualization, machine learning and data mining toolkit. It features a visual programming front-end for explorative qualitative data analysis and interactive data visualization.

LuaTeX is a TeX-based computer typesetting system which started as a version of pdfTeX with a Lua scripting engine embedded. After some experiments it was adopted by the TeX Live distribution as a successor to pdfTeX. Later in the project some functionality of Aleph was included. The project was originally sponsored by the Oriental TeX project, founded by Idris Samawi Hamid, Hans Hagen, and Taco Hoekwater.

Tensor software is a class of mathematical software designed for manipulation and calculation with tensors.

Gson is an open-source Java library to serialize and deserialize Java objects to JSON.

LuaRocks is a package manager for the Lua programming language that provides a standard format for distributing Lua modules, a tool designed to easily manage the installation of rocks, and a server for distributing them. While not included with the Lua distribution, it has been called the "de facto package manager for community-contributed Lua modules".

<span class="mw-page-title-main">TensorFlow</span> Machine learning software library

TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks.

The following table compares notable software frameworks, libraries and computer programs for deep learning.

<span class="mw-page-title-main">Keras</span> Neural network library

Keras is an open-source library that provides a Python interface for artificial neural networks. Keras acts as an interface for the TensorFlow library.

spaCy Software library

spaCy is an open-source software library for advanced natural language processing, written in the programming languages Python and Cython. The library is published under the MIT license and its main developers are Matthew Honnibal and Ines Montani, the founders of the software company Explosion.

<span class="mw-page-title-main">Caffe (software)</span> Deep learning framework

Caffe is a deep learning framework, originally developed at University of California, Berkeley. It is open source, under a BSD license. It is written in C++, with a Python interface.

<span class="mw-page-title-main">PyTorch</span> Open source machine learning library

PyTorch is a machine learning framework based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella. It is free and open-source software released under the modified BSD license. Although the Python interface is more polished and the primary focus of development, PyTorch also has a C++ interface.

ROCm is an Advanced Micro Devices (AMD) software stack for graphics processing unit (GPU) programming. ROCm spans several domains: general-purpose computing on graphics processing units (GPGPU), high performance computing (HPC), heterogeneous computing. It offers several programming models: HIP, OpenMP/Message Passing Interface (MPI), OpenCL.

SqueezeNet is the name of a deep neural network for computer vision that was released in 2016. SqueezeNet was developed by researchers at DeepScale, University of California, Berkeley, and Stanford University. In designing SqueezeNet, the authors' goal was to create a smaller neural network with fewer parameters that can more easily fit into computer memory and can more easily be transmitted over a computer network.

rnn (software) Machine Learning framework written in the R language

rnn is an open-source machine learning framework that implements recurrent neural network architectures, such as LSTM and GRU, natively in the R programming language, that has been downloaded over 100,000 times.

Kubeflow is an open-source platform for machine learning and MLOps on Kubernetes introduced by Google. The different stages in a typical machine learning lifecycle are represented with different software components in Kubeflow, including model development (Kubeflow Notebooks), model training (Kubeflow Pipelines, Kubeflow Training Operator), model serving (KServe), and automated machine learning (Katib).

<span class="mw-page-title-main">Neural Network Intelligence</span> Microsoft open source library

NNI is a free and open-source AutoML toolkit developed by Microsoft. It is used to automate feature engineering, model compression, neural architecture search, and hyper-parameter tuning.

LightGBM, short for light gradient-boosting machine, is a free and open-source distributed gradient-boosting framework for machine learning, originally developed by Microsoft. It is based on decision tree algorithms and used for ranking, classification and other machine learning tasks. The development focus is on performance and scalability.

<span class="mw-page-title-main">Owl Scientific Computing</span> Numerical programming library for the OCaml programming language

Owl Scientific Computing is a software system for scientific and engineering computing developed in the Department of Computer Science and Technology, University of Cambridge. The System Research Group (SRG) in the department recognises Owl as one of the representative systems developed in SRG in the 2010s. The source code is licensed under the MIT License and can be accessed from the GitHub repository.

References

  1. 1 2 "Torch: a modular machine learning software library". 30 October 2002. CiteSeerX   10.1.1.8.9850 .
  2. ↑ Collobert, Ronan. "Torch7". GitHub .
  3. ↑ "Torch7: A Matlab-like Environment for Machine Learning" (PDF). Neural Information Processing Systems. 2011.
  4. ↑ Torch GitHub repository ReadMe
  5. ↑ PyTorch GitHub repository
  6. ↑ "Cheatsheet · torch/torch7 Wiki". GitHub .
  7. ↑ KDnuggets Interview with Yann LeCun, Deep Learning Expert, Director of Facebook AI Lab
  8. ↑ Hacker News
  9. ↑ Yann Lecun's Facebook Page
  10. ↑ IDIAP Research Institute : Torch
  11. ↑ Torch-android GitHub repository
  12. ↑ Torch-ios GitHub repository
  13. ↑ NeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision
  14. ↑ "Facebook Open-Sources a Trove of AI Tools". Wired . 16 January 2015.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.