Cellular neural network

Last updated

In computer science and machine learning, cellular neural networks (CNN) or cellular nonlinear networks (CNN) are a parallel computing paradigm similar to neural networks, with the difference that communication is allowed between neighbouring units only. Typical applications include image processing, analyzing 3D surfaces, solving partial differential equations, reducing non-visual problems to geometric maps, modelling biological vision and other sensory-motor organs. [1]

Contents

CNN is not to be confused with convolutional neural networks (also colloquially called CNN).

CNN architecture

Due to their number and variety of architectures, it is difficult to give a precise definition for a CNN processor. From an architecture standpoint, CNN processors are a system of finite, fixed-number, fixed-location, fixed-topology, locally interconnected, multiple-input, single-output, nonlinear processing units. The nonlinear processing units are often referred to as neurons or cells. Mathematically, each cell can be modeled as a dissipative, nonlinear dynamical system where information is encoded via its initial state, inputs and variables used to define its behavior. Dynamics are usually continuous, as in the case of Continuous-Time CNN (CT-CNN) processors, but can be discrete, as in the case of Discrete-Time CNN (DT-CNN) processors. [2] [3] [4]

Each cell has one output, by which it communicates its state with both other cells and external devices. Output is typically real-valued, but can be complex or even quaternion, i.e. a Multi-Valued CNN (MV-CNN). Most CNN processors, processing units are identical, but there are applications that require non-identical units, which are called Non-Uniform Processor CNN (NUP-CNN) processors, and consist of different types of cells.

Chua-Yang CNN

In the original Chua-Yang CNN (CY-CNN) processor, the state of the cell was a weighted sum of the inputs and the output was a piecewise linear function. However, like the original perceptron-based neural networks, the functions it could perform were limited: specifically, it was incapable of modeling non-linear functions, such as XOR. More complex functions are realizable via Non-Linear CNN (NL-CNN) processors. [5]

Cells are defined in a normed gridded space like two-dimensional Euclidean geometry. However, the cells are not limited to two-dimensional spaces; they can be defined in an arbitrary number of dimensions and can be square, triangle, hexagonal, or any other spatially invariant arrangement. Topologically, cells can be arranged on an infinite plane or on a toroidal space. Cell interconnect is local, meaning that all connections between cells are within a specified radius (with distance measured topologically). Connections can also be time-delayed to allow for processing in the temporal domain.

Most CNN architectures have cells with the same relative interconnects, but there are applications that require a spatially variant topology, i.e. Multiple-Neighborhood-Size CNN (MNS-CNN) processors. Also, Multiple-Layer CNN (ML-CNN) processors, where all cells on the same layer are identical, can be used to extend the capability of CNN processors.

The definition of a system is a collection of independent, interacting entities forming an integrated whole, whose behavior is distinct and qualitatively greater than its entities. Although connections are local, information exchange can happen globally through diffusion. In this sense, CNN processors are systems because their dynamics are derived from the interaction between the processing units and not within processing units. As a result, they exhibit emergent and collective behavior. Mathematically, the relationship between a cell and its neighbors, located within an area of influence, can be defined by a coupling law, and this is what primarily determines the behavior of the processor. When the coupling laws are modeled by fuzzy logic, it is a fuzzy CNN. [6] When these laws are modeled by computational verb logic, it becomes a computational verb CNN. [7] [8] [9] Both fuzzy and verb CNNs are useful for modelling social networks when the local couplings are achieved by linguistic terms. [10]

History

The idea of CNN processors was introduced by Leon Chua and Lin Yang in 1988. [11] In these articles, Chua and Yang outline the underlying mathematics behind CNN processors. They use this mathematical model to demonstrate, for a specific CNN implementation, that if the inputs are static, the processing units will converge, and can be used to perform useful calculations. They then suggest one of the first applications of CNN processors: image processing and pattern recognition (which is still the largest application to date). Leon Chua is still active in CNN research and publishes many of his articles in the International Journal of Bifurcation and Chaos, of which he is an editor. Both IEEE Transactions on Circuits and Systems and the International Journal of Bifurcation also contain a variety of useful articles on CNN processors authored by other knowledgeable researchers. The former tends to focus on new CNN architectures and the latter more on the dynamical aspects of CNN processors.

In 1993, Tamas Roska and Leon Chua introduced the first algorithmically programmable analog CNN processor in the world. [12] The multi-national effort was funded by the Office of Naval Research, the National Science Foundation, and the Hungarian Academy of Sciences, and researched by the Hungarian Academy of Sciences and the University of California. This article proved that CNN processors were producible and provided researchers a physical platform to test their CNN theories. After this article, companies started to invest into larger, more capable processors, based on the same basic architecture as the CNN Universal Processor. Tamas Roska is another key contributor to CNNs. His name is often associated with biologically inspired information processing platforms and algorithms, and he has published numerous key articles and has been involved with companies and research institutions developing CNN technology.

Literature

Two references are considered invaluable since they manage to organize the vast amount of CNN literature into a coherent framework:

Other resources include

CNN processors could be thought of as a hybrid between ANN and Continuous Automata (CA).

Artificial Neural Networks

The processing units of CNN and NN are similar. In both cases, the processor units are multi-input, dynamical systems, and the behavior of the overall systems is driven primarily through the weights of the processing unit’s linear interconnect.

However, in CNN processors, connections are made locally, whereas in ANN, connections are global.

For example, neurons in one layer are fully connected to another layer in a feed-forward NN and all the neurons are fully interconnected in Hopfield networks. In ANNs, the weights of interconnections contain information on the processing system’s previous state or feedback.

But in CNN processors, the weights are used to determine the dynamics of the system.

Furthermore, due to the high inter-connectivity of ANNs, they tend not exploit locality in either the data set or the processing and as a result, they usually are highly redundant systems that allow for robust, fault-tolerant behavior without catastrophic errors.

A cross between an ANN and a CNN processor is a Ratio Memory CNN (RMCNN). In RMCNN processors, the cell interconnect is local and topologically invariant, but the weights are used to store previous states and not to control dynamics. The weights of the cells are modified during some learning state creating long-term memory. [14] [15]

Continuous Automata

The topology and dynamics of CNN processors closely resembles that of CA. Like most CNN processors, CA consists of a fixed-number of identical processors that are spatially discrete and topologically uniform. The difference is that most CNN processors are continuous-valued whereas CA have discrete-values. Furthermore, the CNN processor's cell behavior is defined via some non-linear function whereas CA processor cells are defined by some state machine.

However, there are some exceptions. Continuous Valued Cellular Automata are CA with continuous resolution. Depending on how a given Continuous Automata is specified, it can also be a CNN.

There are also Continuous Spatial Automata, which consist of an infinite number of spatially continuous, continuous-valued automata. There is considerable work being performed in this field since continuous spaces are easier to mathematically model than discrete spaces, thus allowing a more quantitative approach as opposed to an empirical approach taken by some researchers of cellular automata. Continuous Spatial Automata processors can be physically realized though an unconventional information processing platform such as a chemical computer. Furthermore, it is conceivable that large CNN processors (in terms of the resolution of the input and output) can be modeled as a Continuous Spatial Automata.

Model of computation

The dynamical behavior of CNN processors can be expressed using differential equations, where each equation represents the state of an individual processing unit. The behavior of the entire CNN processor is defined by its initial conditions, inputs, cell interconnections (topology and weights), and the cells themselves. One possible use of CNN processors is to generate and respond to signals of specific dynamical properties. For example, CNN processors have been used to generate multiscroll chaos (like the Chen attractor), [16] synchronize with chaotic systems, and exhibit multi-level hysteresis. [17] [18] [19] CNN processors are designed specifically to solve local, low-level, processor intensive problems expressed as a function of space and time. For example, CNN processors can be used to implement high-pass and low-pass filters and morphological operators. They can also be used to approximate a wide range of Partial differential equations (PDE) [20] such as heat dissipation and wave propagation. [21]

Reaction-Diffusion

CNN processors can be used as Reaction-Diffusion (RD) processors. RD processors are spatially invariant, topologically invariant, analog, parallel processors characterized by reactions, where two agents can combine to create a third agent, and diffusion, the spreading of agents. RD processors are typically implemented through chemicals in a Petri dish (processor), light (input), and a camera (output) however RD processors can also be implemented through a multi-layer CNN processor. D processors can be used to create Voronoi diagrams and perform skeletonisation. The main difference between the chemical implementation and the CNN implementation is that CNN implementations are considerably faster than their chemical counterparts and chemical processors are spatially continuous whereas the CNN processors are spatially discrete. The most researched RD processor, Belousov-Zhabotinsky (BZ) processors, has already been simulated using a four-layer CNN processors and has been implemented in a semiconductor. [22] [23] [24] [25]

Boolean functions

Like CA, computations can be performed through the generation and propagation of signals that either grow or change over time. Computations can occur within a signal or can occur through the interaction between signals. One type of processing, which uses signals and is gaining momentum is wave processing, which involves the generation, expanding, and eventual collision of waves. Wave processing can be used to measure distances and find optimal paths. Computations can also occur through particles, gliders, solutions, and filterons localized structures that maintain their shape and velocity.[ clarification needed ] Given how these structures interact/collide with each other and with static signals, they can be used to store information as states and implement different Boolean functions. Computations can also occur between complex, potentially growing or evolving localized behavior through worms, ladders, and pixel-snakes. In addition to storing states and performing Boolean functions, these structures can interact, create, and destroy static structures. [26]

The applications of CNNs to Boolean functions is discussed in the paper by Fangyue Chen, Guolong He, Xiubin Xu, and Guanrong Chen, "Implementation of Arbitrary Boolean Functions via CNN". [26]

Automata and Turing machines

Although CNN processors are primarily intended for analog calculations, certain types of CNN processors can implement any Boolean function, allowing simulating CA. Since some CA are Universal Turing machines (UTM), capable of simulating any algorithm can be performed on processors based on the von Neumann architecture, that makes this type of CNN processors, universal CNN, a UTM. One CNN architecture consists of an additional layer. CNN processors have resulted in the simplest realization of Conway’s Game of Life and Wolfram’s Rule 110, the simplest known universal Turing Machine. This unique, dynamical representation of an old systems, allows researchers to apply techniques and hardware developed for CNN to better understand important CA. Furthermore, the continuous state space of CNN processors, with slight modifications that have no equivalent in Cellular Automata, creates emergent behavior never seen before. [27]

Any information processing platform that allows the construction of arbitrary Boolean functions is called universal, and as result, this class CNN processors are commonly referred to as universal CNN processors. The original CNN processors can only perform linearly separable Boolean functions. By translating functions from digital logic or look-up table domains into the CNN domain, some functions can be considerably simplified. For example, the nine-bit, odd parity generation logic, which is typically implemented by eight nested exclusive-or gates, can also be represented by a sum function and four nested absolute value functions. Not only is there a reduction in the function complexity, but the CNN implementation parameters can be represented in the continuous, real-number domain. [26]

There are two methods by which to select a CNN processor along with a template or weights. The first is by synthesis, which involves determine the coefficients offline. This can be done by leveraging previous work, i.e. libraries, papers, and articles, or by mathematically deriving co that best suits the problem. The other is through training the processor. Researchers have used back-propagation and genetic algorithms to learn and perform functions. Back-propagation algorithms tend to be faster, but genetic algorithms are useful because they provide a mechanism to find a solution in a discontinuous, noisy search space. [28] [29] [30]

Physical implementations

There are toy models simulating CNN processors using billiard balls, but these are used for theoretical studies. In practice, CNN are physically implemented on hardware and current technologies such as semiconductors. There are plans to migrate CNN processors to emerging technologies in the future. [31] [32] [33] [34] [35] [36]

Semiconductors

Semiconductor-based CNN processors can be segmented into analog CNN processors, digital CNN processors, and CNN processors emulated using digital processors. Analog CNN processors were the first to be developed. Analog computers were fairly common during the 1950 and 1960s, but they gradually were replaced by digital computers the 1970s. Analog processors were considerably faster in certain applications such as optimizing differential equations and modeling nonlinearities, but the reason why analog computing lost favor was the lack of precision and the difficulty to configure an analog computer to solve a complex equation.

Analog CNN processors share some of the same advantages as their predecessors, specifically speed. The first analog CNN processors were able to perform real-time ultra-high frame-rate (>10,000 frame/s) processing unachievable by digital processors. The analog implementation of CNN processors requires less area and consumes less power than their digital counterparts. Although the accuracy of analog CNN processors does not compare to their digital counterparts, for many applications, noise and process variances are small enough not to perceptually affect the image quality.

The first algorithmically programmable, analog CNN processor was created in 1993. [12] It was named the CNN Universal Processor because its internal controller allowed multiple templates to be performed on the same data set, thus simulating multiple layers and allowing for universal computation. Included in the design was a single layer 8x8 CCN, interfaces, analog memory, switching logic, and software. The processor was developed in order to determine CNN processor producibility and utility. The CNN concept proved promising and by 2000, there were at least six organizations designing algorithmically programmable, analog CNN processors. [12]

AnaFocus, AnaLogic

In the 2000s, AnaFocus, a mixed-signal semiconductor company from the University of Seville, introduced their ACE prototype CNN processor product line. Their first ACE processor contained 20x20 B/W processor units; and subsequent processors provided 48x48 and 128x128 grayscale processor units, improving the speed and processing elements. AnaFocus also had a multilayer CASE prototype CNN processors line. Their processors allowed real-time interaction between the sensing and processing. In 2014, AnaFocus had been sold to e2v technologies. [37]

Another company, AnaLogic Computers was founded in 2000 by many of the same researchers behind the first algorithmically programmable CNN Universal Processor. In 2003, AnaLogic Computers developed a PCI-X visual processor board that included the ACE 4K processor, [38] with a Texas Instrument DIP module and a high-speed frame-grabber. This allowed CNN processing to be easily included in a desktop computer. In 2006, AnaLogic Computers developed their Bi-I Ultra High Speed Smart Camera product line, which includes the ACE 4K processor in their high-end models. [39]

In 2006, Roska et al. published a paper on designing a Bionic Eyeglass for AnaLogic. The Bionic Eyeglass is a dual-camera, wearable platform, based on the Bi-I Ultra High Speed Smart Camera, designed to provide assistance to blind people. Some of its functions include route number recognition and color processing. [40] [41] [42]

Analog CNN processors

Some researchers developed their own custom analog CNN processors. For example:

Despite their speed and low power consumption, there are some significant drawbacks to analog CNN processors. First, analog CNN processors can potentially create erroneous results due to environment and process variation. In most applications, these errors are not noticeable, but there are situations where minor deviations can result in catastrophic system failures. For example, in chaotic communication, process variation will change the trajectory of a given system in phase space, resulting in a loss of synchronicity/stability. Due to the severity of the problem, there is considerable research being performed to ameliorate the problem. Some researchers are optimizing templates to accommodate greater variation. Other researchers are improving the semiconductor process to more closely match theoretical CNN performance. Other researchers are investigating different, potentially more robust CNN architectures. Lastly, researchers are developing methods to tune templates to target a specific chip and operating conditions. In other words, the templates are being optimized to match the information processing platform. Not only does process variation limit what can be done with current analog CNN processors, it is also a barrier for creating more complex processing units. Unless this process variation is resolved, ideas such as nested processing units, non-linear inputs, etc. cannot be implemented in a real-time analog CNN processor. Also, the semiconductor "real estate" for processing units limits the size of CNN processors.

Currently the largest AnaVision CNN-based vision processor consists of a 4K detector, which is significantly less than the megapixel detectors found in affordable, consumer cameras. Unfortunately, feature size reductions, as predicted by Moore’s Law, will only result in minor improvements. For this reason, alternate technologies such as Resonant Tunneling Diodes and Neuron-Bipolar Junction Transistors are being explored. [45] Also, CNN processor architecture is being re-evaluated. For example, Star-CNN processors, where one analog multiplier is time-shared between multiple processor units, have been proposed and are expected to result in processor unit reduction size of 80%. [46]

Digital CNN processors, FPGA

Although not nearly as fast and energy efficient, digital CNN processors do not share the problems of process variation and feature size of their analog counterparts. This allows digital CNN processors to include nested processor units, non-linearities, etc. In addition, digital CNN are more flexible, cost less and are easier to integrate. The most common implementation of digital CNN processors uses an FPGA. Eutecus, founded in 2002 and operating in Berkeley, provides intellectual property that can be synthesized into an Altera FPGA. Their digital 320x280, FPGA-based CNN processors run at 30 frame/s and there are plans to make a fast digital ASIC. Eustecus is a strategic partner of AnaLogic computers, and their FPGA designs can be found in several of AnaLogic’s products. Eutecus is also developing software libraries to perform tasks including but not limited to video analytics for the video security market, feature classification, multi-target tracking, signal and image processing and flow processing. Many of these routines are derived using CNN-like processing. For those wanting to perform CNN simulations for prototyping, low-speed applications, or research, there are several options. First, there are precise CNN emulation software packages like SCNN 2000. If the speed is prohibitive, there are mathematical techniques, such as Jacobi’s Iterative Method or Forward-Backward Recursions that can be used to derive the steady state solution of a CNN processor. Lastly, digital CNN processors can be emulated on highly parallel, application-specific processors, such as graphics processors. Implementing neural networks using graphics processors is an area of further research. [47] [48] [49] [50] [51]

Holography, nanotechnology

Researchers are also perusing alternate technologies for CNN processors. Although current CNN processors circumvent some of the problems associated with their digital counterparts, they do share some of the same long-term problems common to all semiconductor-based processors. These include, but are not limited to, speed, reliability, power-consumption, etc. AnaLogic Computers, is developing optical CNN processors, which combine optics, lasers, and biological and holographic memories. What initially was technology exploration resulted in a 500x500 CNN processor able to perform 300 giga-operations per second.

Another promising technology for CNN processors is nanotechnology. One nanotechnology concept being investigated is using single electron tunneling junctions, which can be made into single-electron or high-current transistors, to create McCulloch-Pitts CNN processing units. In summary, CNN processors have been implemented and provide value to their users. They have been able to effectively leverage the advantages and address some of the disadvantages associated with their underling technology, i.e. semiconductors. Researchers are also transitioning CNN processors into emerging technologies. Therefore, if the CNN architecture is suited for a specific information processing system, there are processors available for purchase (as there will be for the foreseeable future). [52]

Applications

CNN researchers have diverse interests, ranging from physical, engineering, theoretical, mathematical, computational, and philosophical applications.

Image processing

CNN processors were designed to perform image processing; specifically, real-time ultra-high frame-rate (>10,000 frame/s) processing for applications like particle detection in jet engine fluids and spark-plug detection. Currently, CNN processors can achieve up to 50,000 frames per second, and for certain applications such as missile tracking, flash detection, and spark-plug diagnostics these microprocessors have outperformed a conventional supercomputer. CNN processors lend themselves to local, low-level, processor intensive operations and have been used in feature extraction, [53] level and gain adjustments, color constancy detection, [54] contrast enhancement, deconvolution, [55] image compression, [56] [57] motion estimation, [58] [59] image encoding, image decoding, image segmentation, [60] [61] orientation preference maps, [62] pattern learning/recognition, [43] [63] multi-target tracking, [64] image stabilization, [58] resolution enhancement, [65] image deformations and mapping, image inpainting, [66] optical flow, [67] contouring, [68] [69] moving object detection, [70] axis of symmetry detection, [71] and image fusion. [72] [73] [74]

Due to their processing capabilities and flexibility, CNN processors have been used and prototyped for novel field applications such as flame analysis for monitoring combustion at a waste incinerator, [75] mine-detection using infrared imagery, calorimeter cluster peak for high energy physics, [76] anomaly detection in potential field maps for geophysics, [77] laser dot detection, [78] metal inspection for detecting manufacturing defects, [79] and seismic horizon picking. They have also been used to perform biometric functions [80] such as fingerprint recognition, [81] vein feature extraction, face tracking, [82] and generating visual stimuli via emergent patterns to gauge perceptual resonances.

Biology and medicine

CNN processors have been used for medical and biological research in performing automated nucleated cell counting for detecting hyperplasia, [83] segment images into anatomically and pathologically meaningful regions, measure and quantify cardiac function, measure the timing of neurons, and detect brain abnormalities that would lead to seizures. [84] [85]

One potential future application of CNN microprocessors is to combine them with DNA microarrays to allow for a near-real time DNA analysis of hundreds of thousands of different DNA sequences. Currently, the major bottleneck of DNA microarray analysis is the amount of time needed to process data in the form of images, and using a CNN microprocessor, researchers have reduced the amount of time needed to perform this calculation to 7ms.

Texture recognition

CNN processors have also been used to generate and analyze patterns and textures. One motivation was to use CNN processors to understand pattern generation in natural systems. They were used to generate Turing patterns in order to understand the situations in which they form, the different types of patterns which can emerge, and the presence of defects or asymmetries. [27] Also, CNN processors were used to approximate pattern generation systems that create stationary fronts, spatio-temporal patterns oscillating in time, hysteresis, memory, and heterogeneity. Furthermore, pattern generation was used to aid high-performance image generation and compression via real-time generation of stochastic and coarse-grained biological patterns, texture boundary detection, and pattern and texture recognition and classification. [86] [87]

Control and Actuator Systems

There is an ongoing effort to incorporate CNN processors into sensory-computing-actuating machines as part of the emerging field of Cellular Machines. The basic premise is to create an integrated system that uses CNN processors for the sensory signal-processing and potentially the decision-making and control. The reason is that CNN processors can provide a low power, small size, and eventually low-cost computing and actuating system suited for Cellular Machines. These Cellular Machines will eventually create a Sensor-Actuator Network (SAN), [88] a type of Mobile Ad Hoc Networks (MANET) which can be used for military intelligence gathering, surveillance of inhospitable environments, maintenance of large areas, planetary exploration, etc.

CNN processors have been proven versatile enough for some control functions. They have been used to optimize function via a genetic algorithm, [89] to measure distances, to perform optimal path-finding in a complex, dynamic environment, and theoretically can be used to learn and associate complex stimuli. They have also been used to create antonymous gaits and low-level motors for robotic nematodes, spiders, and lamprey gaits using a Central Pattern Generator (CPG). They were able to function using only feedback from the environment, allowing for a robust, flexible, biologically inspired robot motor system. CNN-based systems were able to operate in different environments and still function if some of the processing units are disabled. [90] [91] [92]

Communication systems

The variety of dynamical behavior seen in CNN processors make them intriguing for communication systems. Chaotic communications using CNN processors is being researched due to their potential low power consumption, robustness and spread spectrum features. The premise behind chaotic communication is to use a chaotic signal for the carrier wave and to use chaotic phase synchronization to reconstruct the original message. CNN processors can be used on both the transmitter and receiver end to encode and decode a given message. They can also be used for data encryption and decryption, source authentication through watermarking, [93] detecting of complex patterns in spectrogram images [94] (sound processing), and transient spectral signals detection.

CNN processors are neuromorphic processors, meaning that they emulate certain aspects of biological neural networks. The original CNN processors were based on mammalian retinas, which consist of a layer of photodetectors connected to several layers of locally coupled neurons. [95] This makes CNN processors part of an interdisciplinary research area whose goal is to design systems that leverage knowledge and ideas from neuroscience and contribute back via real-world validation of theories. CNN processors have implemented a real-time system that replicates mammalian retinas, validating that the original CNN architecture chosen modeled the correct aspects of the biological neural networks used to perform the task in mammalian life. [95] However, CNN processors are not limited to verifying biological neural networks associated with vision processing; they have been used to simulate dynamic activity seen in mammalian neural networks found in the olfactory bulb and locust antennal lobe, responsible for pre-processing sensory information to detect differences in repeating patterns. [96] [97]

CNN processors are being used to understand systems that can be modeled using simple, coupled units, such as living cells, biological networks, physiological systems, and ecosystems. The CNN architecture captures some of the dynamics often seen in nature and is simple enough to analyze and conduct experiments. They are also being used for stochastic simulation techniques, which allow scientists to explore spin problems, population dynamics, lattice-based gas models, percolation, and other phenomena. Other simulation applications include heat transfer, mechanical vibrating systems, protein production, [98] Josephson Junction problems, [99] seismic wave propagation, [100] and geothermal structures. [101] Instances of 3d CNN have been used to prove certain emergent phenomena in complex systems, establishing a link between art, dynamical systems and VLSI technology. [102] [103] [104] [105]

CNN processors have been used to research a variety of mathematical concepts, such as non-equilibrium systems, constructing non-linear systems of arbitrary complexity, emergent chaotic dynamics, and discovering new dynamic behavior. They are often used in researching systemics, a trans-disciplinary, scientific field that studies natural systems. The goal of systemics researchers is to develop a conceptual and mathematical framework necessary to analyze, model, and understand systems, including, but not limited to, atomic, mechanical, molecular, chemical, biological, ecological, social and economic systems. Topics explored are emergence, collective behavior, local activity and its impact on global behavior, and quantifying the complexity of an approximately spatial and topologically invariant system. [106] With another definition of complexity (MIT professor Seth Lloyd has identified 32 different definitions of complexity [107] ), it can potentially be mathematically advantageous when analyzing systems such as economic and social systems.

Notes

  1. ↑ Slavova, A. (2003-03-31). Cellular Neural Networks: Dynamics and Modelling. Springer Science & Business Media. ISBN   978-1-4020-1192-4.
  2. ↑ S. Malki, Y. Fuqiang, and L. Spaanenburg, "Vein Feature Extraction Using DT-CNNs", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  3. ↑ H. Harrer and J.Nossek, "Discrete-Time Cellular Neural Networks", International Journal of Circuit Theory and Applications, 20:453-467, 1992.
  4. ↑ M. Brugge, "Morphological Design of Discrete−Time Cellular Neural Networks", University of Groningen Dissertation, 2005.
  5. ↑ E. Gomez-Ramirez, G. Pazienza, X. Vilasis-Cardona, "Polynomial Discrete Time Cellular Neural Networks to solve the XOR Problem", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  6. ↑ Yang, T.; et al. (October 1996). "The global stability of fuzzy cellular neural network". IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications . 43 (10). IEEE: 880–883. doi:10.1109/81.538999.
  7. ↑ Yang, T. (March 2009). "Computational Verb Cellular Networks: Part I--A New Paradigm of Human Social Pattern Formation". International Journal of Computational Cognition. 7 (1). Yang's Scientific Press: 1–34.
  8. ↑ Yang, T. (March 2009). "Computational Verb Cellular Networks: Part II--One-Dimensional Computational Verb Local Rules". International Journal of Computational Cognition. 7 (1). Yang's Scientific Press: 35–51.
  9. ↑ Yang, T. (June 2009). "Computational Verb Cellular Networks: Part III--Solutions of One-Dimensional Computational Verb Cellular Networks". International Journal of Computational Cognition. 7 (2). Yang's Scientific Press: 1–11.
  10. ↑ E. Gunay, M. Alci and S. Parmaksizoglu, "N-Scroll Generation in SC-CNN via Neuro Fuzzy Based Non-Linear Function", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  11. ↑ https://www.researchgate.net/publication/3183706_Cellular_neural_networks_Theory ("Cellular Neural Networks: Theory" and "Cellular Neural Networks: Applications" in IEEE Transactions on Circuits and Systems)
  12. 1 2 3 Roska, T.; Chua, L.O. (March 1993). "The CNN universal machine: an analogic array computer". IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing. 40 (3): 163–173. doi:10.1109/82.222815. ISSN   1057-7130.
  13. ↑ Cellular Neural Networks: A Review (Neural Nets WIRN Vietri 1993)
  14. ↑ C. Wu and Y. Wu, "The Design of CMOS Non-Self-Feedback Ratio Memory Cellular Nonlinear Network without Elapsed Operation for Pattern Learning and Recognition", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  15. ↑ C. Wu and C. Cheng, "The Design of Cellular Neural Network with Ratio Memory for Pattern Learning and Recognition", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
  16. ↑ M. Yalcin, J. Suykens, and J. Vandewalle, Cellular Neural Networks, Multi-Scroll Chaos And Synchronization, 2005.
  17. ↑ A. Slavova and M. Markovat, "Receptor Based CNN Model with Hysteresis for Pattern Generation", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  18. ↑ K. Yokosawa, Y. Tanji and M. Tanaka, "CNN with Multi-Level Hysteresis Quantization Output" Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  19. ↑ T. Nakaguchi, K. Omiya and M. Tanaka, "Hysteresis Cellular Neural Networks for Solving Combinatorial Optimization Problems", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  20. ↑ G. Cserey, C. Rekeczky and P. Foldesy, "PDE Based Histogram Modification with Embedded Morphological Processing of the Level Sets", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  21. ↑ P. Sonkolyt, P. Kozmat, Z. Nagyt and P. Szolgay, "Acoustic Wave Propagation Modeling on CNN-UM Architecture", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  22. ↑ A. Adamatzky, B. Costello, T Asai "Reaction-Diffusion Computers", 2005.
  23. ↑ F. Gollas and R. Tetzlaff, "Modeling Complex Systems by Reaction-Diffusion Cellular Nonlinear Networks with Polynomial Weight-Functions", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  24. ↑ A. Selikhov, "mL-CNN: A CNN Model for Reaction Diffusion Processes in m Component Systems", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  25. ↑ B. Shi and T. Luo, "Spatial Pattern Formation via Reaction–Diffusion Dynamics in 32x32x4 CNN Chip", IEEE Trans. On Circuits And Systems-I, 51(5):939-947, 2004.
  26. 1 2 3 Chen, F.; He, G.; Xu, X.; Chen, G. (August 2006). "Implementation of Arbitrary Boolean Functions via CNN". 2006 10th International Workshop on Cellular Neural Networks and Their Applications. pp. 1–6. doi:10.1109/CNNA.2006.341641. ISBN   1-4244-0639-0. S2CID   9648461.
  27. 1 2 Goras, L.; Chua, L.O.; Leenaerts, D.M.W. (1995). "Turing patterns in CNNs. I. Once over lightly". IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. 42 (10): 602–611. doi:10.1109/81.473567. ISSN   1057-7122.
  28. ↑ T. Kozek, T. Roska, and L. Chua, "Genetic Algorithms for CNN Template Learning," IEEE Trans. on Circuits and Systems I, 40(6):392-402, 1993.
  29. ↑ G. Pazienza, E. Gomez-Ramirezt and X. Vilasis-Cardona, "Genetic Programming for the CNN-UM", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  30. ↑ E. Lopez, M. Balsif, D. Vilarilio and D. Cabello, "Design and Training of Multilayer Discrete Time Cellular Neural Networks for Antipersonnel Mine Detection Using Genetic Algorithms", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
  31. ↑ R. Carmona, F. Jimenez-Garrido, R. Dominguez-Castro, S. Espejo and A. Rodriguez-Vazquez, "CMOS Realization of a 2-layer CNN Universal Machine", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  32. ↑ C. Dominguez-Matas, R. Carmona-Galan, F. Sainchez-Fernaindez, A. Rodriguez-Vazquez, "3-Layer CNN Chip for Focal-Plane Complex Dynamics with Adaptive Image Capture", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  33. ↑ C. Dominguez-Matas, F. Sainchez-Femaindez, R. Carmona-Galan, and E. Roca-Moreno, "Experiments on Global and Local Adaptation to Illumination Conditions based on Focal Plane Average Computation", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  34. ↑ S. Xavier-de-Souza, M. Yalcın, J. Suykens, and J. Vandewalle, "Toward CNN Chip-Specific Robustness", IEEE Trans. On Circuits And Systems - I, 51(5): 892-902, 2004.
  35. ↑ D. Hillier, S. Xavier de Souza, J. Suykens, J. Vandewalle, "CNNOPT Learning CNN Dynamics and Chip-specific Robustness", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  36. ↑ D. Vilarino and C. Rekeczky, "Implementation of a Pixel-Level Snake Algorithm on a CNNUM-Based Chip Set Architecture", IEEE Trans. On Circuits And Systems - I, 51(5): 885-891, 2004.
  37. ↑ "Acquisition of AnaFocus: Fast-growing CMOS imaging business to be integrated into High Performance Imaging division". e2v. Retrieved 2020-12-27.
  38. ↑ "StackPath". www.vision-systems.com. February 2003. Retrieved 2020-12-27.
  39. ↑ A. Rodríguez-Vázquez, G. Liñán-Cembrano, L. Carranza, E. Roca-Moreno, R. Carmona-Galán, F. Jiménez-Garrido, R. Domínguez-Castro, and S. Meana, "ACE16k: The Third Generation of Mixed-Signal SIMD-CNN ACE Chips Toward VSoCs," IEEE Trans. on Circuits and Systems - I, 51(5): 851-863, 2004.
  40. ↑ Roska, T.; Balya, D.; Lazar, A.; Karacs, K.; Wagner, R.; Szuhaj, M. (May 2006). "System aspects of a bionic eyeglass". 2006 IEEE International Symposium on Circuits and Systems. pp. 4 pp.–164. doi:10.1109/ISCAS.2006.1692547. ISBN   0-7803-9389-9. S2CID   3842486.
  41. ↑ K. Karacst and T. Roskatt, "Route Number Recognition of Public Transport Vehicles via the Bionic Eyeglass", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  42. ↑ R. Wagner and M. Szuhajt, "Color Processing in Wearable Bionic Glasses"
  43. 1 2 C. Wu and C. Cheng, "The Design of Cellular Neural Network with Ratio Memory for Pattern Learning and Recognition", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
  44. ↑ W. Yen, R. Chen and J. Lai, "Design of Min/Max Cellular Neural Networks in CMOS Technology" (IEEE), Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  45. ↑ W. Yen and C. Wu, "The Design of Neuron-Bipolar Junction Transistor (vBJT) Cellular Neural Network(CNN) Structure with Multi-Neighborhood-Layer Template", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
  46. ↑ F. Sargeni, V. Bonaiuto and M. Bonifazi, "Multiplexed Star-CNN Architecture", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  47. ↑ Z. Nagyt, Z. Voroshazi and P. Szolgay, "A Real-time Mammalian Retina Model Implementation on FPGA", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  48. ↑ S. Kocsardit, Z. Nagyt, S. Kostianevt and P. Szolgay, "FPGA Based Implementation of Water Injection in Geothermal Structure", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  49. ↑ P. Arena, L. Fortuna, G. Vagliasindi and A. Basile, "CNN Chip And FPGA To Explore Complexity", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  50. ↑ W. Fang, C. Wang and L. Spaanenburg, "In Search of a Robust Digital CNN System" Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  51. ↑ Z. Kincsest, Z. Nagyl, and P. Szolgay, "Implementation of Nonlinear Template Runner Emulated Digital CNN-UM on FPGA", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  52. ↑ W. Porod, F. Werblin, L. Chua, T. Roska, A. Rodriguez-Vázquez, B. Roska, R. Faya, G. Bernstein, Y. Huang, and A. Csurgay, "Bio-Inspired Nano-Sensor-Enhanced CNN Visual Computer", Annals of the New York Academy of Sciences, 1013: 92–109, 2004.
  53. ↑ O. Lahdenoja, M. Laiho and A. Paasio, "Local Binary Pattern Feature Vector Extraction with CNN", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  54. ↑ L. Torok and A. Zarandy, "CNN Based Color Constancy Algorithm", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  55. ↑ L. Orzo, "Optimal CNN Templates for Deconvolution", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006
  56. ↑ P. Venetianer and T. Roska, "Image Compression by Cellular Neural Networks," IEEE Trans. Circuits Syst., 45(3): 205-215, 1998.
  57. ↑ R. Dogarut, R. Tetzlaffl and M. Glesner, "Semi-Totalistic CNN Genes for Compact Image Compression", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  58. 1 2 Y. Cheng, J. Chung, C. Lin and S. Hsu, "Local Motion Estimation Based On Cellular Neural Network Technology for Image Stabilization Processing", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  59. ↑ A. Gacsadi, C. Grava, V. Tiponut, and P. Szolgay, "A CNN Implementation of the Horn & Schunck Motion Estimation Method", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  60. ↑ S. Chen, M. Kuo and J. Wang, "Image Segmentation Based on Consensus Voting", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  61. ↑ G. Grassi, E. Sciascio, A. Grieco and P. Vecchio, "A New Object-oriented Segmentation Algorithm based on CNNs - Part II: Performance Evaluation", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  62. ↑ J. Wu, Z. Lin and C. Liou, "Formation and Variability of Orientation Preference Maps in Visual Cortex: an Approach Based on Normalized Gaussian Arrays", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  63. ↑ C. Wu and S. Tsai, "Autonomous Ratio-Memory Cellular Nonlinear Network (ARMCNN) for Pattern Learning and Recognition", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  64. ↑ G. Timar and C. Rekeczky, "Multitarget Tracking Applications of the Bi-I Platform: Attention-selection, Tracking and Navigation", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  65. ↑ T. Otake, T. Konishi, H. Aomorit, N. Takahashit and M. Tanakat, "Image Resolution Upscaling Via Two-Layered Discrete Cellular Neural Network", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  66. ↑ A. Gacsadi and P. Szolgay, "Image Inpainting Methods by Using Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  67. ↑ B. Shi, T. Roska and L. Chua, "Estimating Optical Flow with Cellular Neural Networks," Int’l Journal of Circuit Theory and Applications, 26: 344-364, 1998.
  68. ↑ Szalka, G. Soos, D. Hillier, L. Kek, G. Andrassy and C. Rekeczky, "Space-time Signature Analysis of 2D Echocardiograms Based on Topographic Cellular Active Contour Techniques", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  69. ↑ T. Szabot and P. Szolgay, "CNN-UM-Based Methods Using Deformable Contours on Smooth Boundaries", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  70. ↑ G. Costantini, D. Casali, and R. Perfetti, "Detection of Moving Objects in a Binocular Video Sequence", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  71. ↑ G Costantini, D. Casafi., and R. Perfetti, "A New CNN-based Method for Detection of the Axis of Symmetry.", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  72. ↑ I. Szatmari, P. Foldesy, C. Rekeczky and A. Zarandy, "Image Processing Library for the Aladdin Computer", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  73. ↑ I. Szatmari, P. Foldesy, C. Rekeczky and A. Zarandy, "Image processing library for the Aladdin Visual Computer", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  74. ↑ K. Wiehler, M. Perezowsky, R. Grigat, "A Detailed Analysis of Different CNN Implementations for a Real-Time Image Processing System", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
  75. ↑ L. Bertucco, A. Fichaa, G. Nmari and A. Pagano, "A Cellular Neural Networks Approach to Flame Image Analysis for Combustion Monitoring", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
  76. ↑ C. Baldanza, F. Bisi, M. Bruschi, I. D’Antone, S. Meneghini, M. Riui, M. Zufa, "A Cellular Neural Network For Peak Finding In High-Energy Physics", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
  77. ↑ E. Bilgili, O. Ucan, A. Albora and I. Goknar, "Potential Anomaly Separation Using Genetically Trained Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  78. ↑ C. Rekeczky and G. Timar "Multiple Laser Dot Detection and Localization within an Attention Driven Sensor Fusion Framework", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  79. ↑ Z. Szlavikt, R. Tetzlaff, A. Blug and H. Hoefler, "Visual Inspection of Metal Objects Using Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  80. ↑ R. Dogaru and I. Dogaru, "Biometric Authentication Based on Perceptual Resonance Between CNN Emergent Patterns and Humans", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  81. ↑ T. Su, Y. Du, Y. Cheng, and Y. Su, "A Fingerprint Recognition System Using Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  82. ↑ S. Xavier-de-Souza, M. Van Dyck, J. Suykens and J. Vandewalle, "Fast and Robust Face Tracking for CNN Chips: Application to Wheelchair Driving", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  83. ↑ Q. Feng, S. Yu and H. Wang, "An New Automatic Nucleated Cell Counting Method With Improved Cellular Neural Networks (ICNN)", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  84. ↑ D. Krug, A. Chernihovskyi, H. Osterhage, C. Elger, and K. Lehnertz, "Estimating Generalized Synchronization in Brain Electrical Activity from Epilepsy Patients with Cellular Nonlinear Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  85. ↑ C. Niederhoefer and R. Tetzlaff, "Prediction Error Profiles allowing a Seizure Forecasting in Epilepsy?", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  86. ↑ E. David, P. Ungureanu, and L. Goras, "On he Feature Extraction Performances of Gabor-Type Filters in Texture Recognition Applications", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  87. ↑ C. Lin and S. Chen, "Biological Visual Processing for Hybrid-Order Texture Boundary Detection with CNN-UM", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  88. ↑ M. Haenggi, "Mobile Sensor-Actuator Networks: Opportunities and Challenges", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  89. ↑ D. Balya and V. Galt, "Analogic Implementation of the Genetic Algorithm", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  90. ↑ Asli, A. E. Niaraki; Roghair, J.; Jannesari, A. (2020-03-11). "Energy-aware Goal Selection and Path Planning of UAV Systems via Reinforcement Learning". arXiv: 1909.12217 [eess.SP].
  91. ↑ I. Gavrilut, V. Tiponut, and A. Gacsadi, "Path Planning of Mobile Robots by Using Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  92. ↑ P. Arena, P. Crucitti, L. Fortuna, M. Frasca, D. Lombardo and L. Patane, "Perceptive Patterns For Mobile Robots via RD-CNN and Reinforcement Learning", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  93. ↑ P. Arena, A. Basile, L. Fortuna, M. E. Yalcin, and J. Vandewalle, "Watermarking for the Authentication of Video on CNN-UM", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  94. ↑ K. Slot, P. Korbe, M. Gozdzik, and Hyongsuk Kim, "Pattern detection in spectrograms by means of Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  95. 1 2 D. Balya and B. Roska, "A Handy Retina Exploration Device", Workshop on Cellular Neural Networks and Their Applications, 2005.
  96. ↑ C. Dominguez-Matas, R. Carmona-Galan, F. Sanchez-Fernaindez, J. Cuadri, and A. Rodriguez-Vaizquez, "A Bio-Inspired Vision Front-End Chip with Spatio-Temporal Processing and Adaptive Image Capture", Int’l Workshop on Computer Architecture for Machine Perception and Sensing, 2006.
  97. ↑ A. Zarandy and C. Rekeczky, "Bi-i: a standalone ultra high speed cellular vision system", IEEE Circuits and Systems Magazine, 5(2):36-45, 2005.
  98. ↑ W. Samarrai, J. Yeol, I. Bajis and Y. Ryu, "System Biology Modeling of Protein Process using Deterministic Finite Automata (DFA)", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  99. ↑ V. Mladenovt, and A. Slavoval, "On the Period Solutions in One Dimensional Cellular Neural Networks based on Josephson Junctions", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  100. ↑ K. Huang, C. Chang, W. Hsieh, S. Hsieh, L. Wang and F. Tsai, "Cellular Neural Network For Seismic Horizon Picking", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
  101. ↑ P. Lopez, D. Vilarino, D. Cabello, H. Sahli and M. Balsi, "CNN Based Thermal Modeling of the Soil for Anitpersonnel Mine Detection", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
  102. ↑ H. Ip, E. Drakakis, and A. Bharath, "Towards Analog VLSI Arrays for Nonseparable 3D Spatiotemporal Filtering", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  103. ↑ P.Arena, L. Fortuna, M. Frasca, L. Patane, and M. Pollino, "An Autonomous Mini-Hexapod Robot Controller through a CNN Based VLSI Chip", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
  104. ↑ A. Adamatzky, P. Arena, A. Basile, R. Carmona-Galán, B. Costello, L. Fortuna, M. Frasca, and A. Rodríguez-Vázquez, "Reaction-Diffusion Navigation Robot Control: From Chemical to VLSI Analogic Processors", IEEE Trans. On Circuits And Systems – I, 51(5):926-938, 2004.
  105. ↑ L. Chua, L. Yang, and K. R. Krieg, "Signal Processing Using Cellular Neural Networks", Journal of VLSI Signal Processing, 3:25-51, 1991.
  106. ↑ L. Chua, "Local Activity is the Origin of Complexity", Int’l Journal of Bifurcation and Chaos, 15(11):3435-2456, 2005.
  107. ↑ S. Lloyd, Programming the Universe, 2006.

Related Research Articles

<span class="mw-page-title-main">Neural network (machine learning)</span> Computational model used in machine learning, based on connected, hierarchical functions

In machine learning, a neural network is a model inspired by the structure and function of biological neural networks in animal brains.

Neuromorphic computing is an approach to computing that is inspired by the structure and function of the human brain. A neuromorphic computer/chip is any device that uses physical artificial neurons to do computations. In recent times, the term neuromorphic has been used to describe analog, digital, mixed-mode analog/digital VLSI, and software systems that implement models of neural systems. The implementation of neuromorphic computing on the hardware level can be realized by oxide-based memristors, spintronic memories, threshold switches, transistors, among others. Training software-based neuromorphic systems of spiking neural networks can be achieved using error backpropagation, e.g., using Python based frameworks such as snnTorch, or using canonical learning rules from the biological learning literature, e.g., using BindsNet.

In artificial intelligence, artificial immune systems (AIS) are a class of computationally intelligent, rule-based machine learning systems inspired by the principles and processes of the vertebrate immune system. The algorithms are typically modeled after the immune system's characteristics of learning and memory for use in problem-solving.

<span class="mw-page-title-main">Neural circuit</span> Network or circuit of neurons

A neural circuit is a population of neurons interconnected by synapses to carry out a specific function when activated. Multiple neural circuits interconnect with one another to form large scale brain networks.

Neural engineering is a discipline within biomedical engineering that uses engineering techniques to understand, repair, replace, or enhance neural systems. Neural engineers are uniquely qualified to solve design problems at the interface of living neural tissue and non-living constructs.

<span class="mw-page-title-main">Chua's circuit</span> Electronic circuit that behaves chaotically

Chua's circuit is a simple electronic circuit that exhibits classic chaotic behavior. This means roughly that it is a "nonperiodic oscillator"; it produces an oscillating waveform that, unlike an ordinary electronic oscillator, never "repeats". It was invented in 1983 by Leon O. Chua, who was a visitor at Waseda University in Japan at that time. The ease of construction of the circuit has made it a ubiquitous real-world example of a chaotic system, leading some to declare it "a paradigm for chaos".

Unconventional computing is computing by any of a wide range of new or unusual methods. It is also known as alternative computing.

The floating-gate MOSFET (FGMOS), also known as a floating-gate MOS transistor or floating-gate transistor, is a type of metal–oxide–semiconductor field-effect transistor (MOSFET) where the gate is electrically isolated, creating a floating node in direct current, and a number of secondary gates or inputs are deposited above the floating gate (FG) and are electrically isolated from it. These inputs are only capacitively connected to the FG. Since the FG is surrounded by highly resistive material, the charge contained in it remains unchanged for long periods of time, typically longer than 10 years in modern devices. Usually Fowler-Nordheim tunneling and hot-carrier injection mechanisms are used to modify the amount of charge stored in the FG.

<span class="mw-page-title-main">Spiking neural network</span> Artificial neural network that mimics neurons

Spiking neural networks (SNNs) are artificial neural networks (ANN) that more closely mimic natural neural networks. In addition to neuronal and synaptic state, SNNs incorporate the concept of time into their operating model. The idea is that neurons in the SNN do not transmit information at each propagation cycle, but rather transmit information only when a membrane potential—an intrinsic quality of the neuron related to its membrane electrical charge—reaches a specific value, called the threshold. When the membrane potential reaches the threshold, the neuron fires, and generates a signal that travels to other neurons which, in turn, increase or decrease their potentials in response to this signal. A neuron model that fires at the moment of threshold crossing is also called a spiking neuron model.

<span class="mw-page-title-main">Misha Mahowald</span> American computational neuroscientist

Michelle Anne Mahowald was an American computational neuroscientist in the emerging field of neuromorphic engineering. In 1996 she was inducted into the Women in Technology International Hall of Fame for her development of the Silicon Eye and other computational systems. She died by suicide at age 33.

<span class="mw-page-title-main">Leon O. Chua</span> American electrical engineer and computer scientist

Leon Ong Chua is an American electrical engineer and computer scientist. He is a professor in the electrical engineering and computer sciences department at the University of California, Berkeley, which he joined in 1971. He has contributed to nonlinear circuit theory and cellular neural network theory.

A physical neural network is a type of artificial neural network in which an electrically adjustable material is used to emulate the function of a neural synapse or a higher-order (dendritic) neuron model. "Physical" neural network is used to emphasize the reliance on physical hardware used to emulate neurons as opposed to software-based approaches. More generally the term is applicable to other artificial neural networks in which a memristor or other electrically adjustable resistance material is used to emulate a neural synapse.

Models of neural computation are attempts to elucidate, in an abstract and mathematical fashion, the core principles that underlie information processing in biological nervous systems, or functional components thereof. This article aims to provide an overview of the most definitive models of neuro-biological computation as well as the tools commonly used to construct and analyze them.

Natural computing, also called natural computation, is a terminology introduced to encompass three classes of methods: 1) those that take inspiration from nature for the development of novel problem-solving techniques; 2) those that are based on the use of computers to synthesize natural phenomena; and 3) those that employ natural materials to compute. The main fields of research that compose these three branches are artificial neural networks, evolutionary algorithms, swarm intelligence, artificial immune systems, fractal geometry, artificial life, DNA computing, and quantum computing, among others.

<span class="mw-page-title-main">Deep learning</span> Branch of machine learning

Deep learning is the subset of machine learning methods based on neural networks with representation learning. The adjective "deep" refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Kwabena Adu Boahen is a Ghanaian-born Professor of Bioengineering and Electrical Engineering at Stanford University. He previously taught at the University of Pennsylvania.

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer, 10,000 weights would be required for processing an image sized 100 × 100 pixels. However, applying cascaded convolution kernels, only 25 neurons are required to process 5x5-sized tiles. Higher-layer features are extracted from wider context windows, compared to lower-layer features.

An AI accelerator, deep learning processor, or neural processing unit (NPU) is a class of specialized hardware accelerator or computer system designed to accelerate artificial intelligence and machine learning applications, including artificial neural networks and machine vision. Typical applications include algorithms for robotics, Internet of Things, and other data-intensive or sensor-driven tasks. They are often manycore designs and generally focus on low-precision arithmetic, novel dataflow architectures or in-memory computing capability. As of 2024, a typical AI integrated circuit chip contains tens of billions of MOSFETs.

<span class="mw-page-title-main">AlexNet</span> Convolutional neural network

AlexNet is the name of a convolutional neural network (CNN) architecture, designed by Alex Krizhevsky in collaboration with Ilya Sutskever and Geoffrey Hinton, who was Krizhevsky's Ph.D. advisor at the University of Toronto.

Artificial neural networks (ANNs) are models created using machine learning to perform a number of tasks. Their creation was inspired by neural circuitry. While some of the computational implementations ANNs relate to earlier discoveries in mathematics, the first implementation of ANNs was by psychologist Frank Rosenblatt, who developed the perceptron. Little research was conducted on ANNs in the 1970s and 1980s, with the AAAI calling that period an "AI winter".

References

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.