Walter Harry Pitts, Jr.
23 April 1923
|Died||14 May 1969 46) (aged|
Walter Harry Pitts, Jr. (23 April 1923 – 14 May 1969) was a logician who worked in the field of computational neuroscience. He proposed landmark theoretical formulations of neural activity and generative processes that influenced diverse fields such as cognitive sciences and psychology, philosophy, neurosciences, computer science, artificial neural networks, cybernetics and artificial intelligence, together with what has come to be known as the generative sciences. He is best remembered for having written along with Warren McCulloch, a seminal paper in scientific history, titled "A Logical Calculus of Ideas Immanent in Nervous Activity" (1943). This paper proposed the first mathematical model of a neural network. The unit of this model, a simple formalized neuron, is still the standard of reference in the field of neural networks. It is often called a McCulloch–Pitts neuron. Prior to that paper, he formalized his ideas regarding the fundamental steps to building a Turing machine in the "The Bulletin of Mathematical Biophysics" in an essay titled "Some observations on the simple neuron circuit"
Computational neuroscience is a branch of neuroscience which uses computational approaches, to study the nervous system. Computational approaches include mathematics, statistics, computer simulations, and abstractions which are used across many subareas of neuroscience including development, structure, physiology and cognitive abilities of the nervous system.
Psychology is the science of behavior and mind. Psychology includes the study of conscious and unconscious phenomena, as well as feeling and thought. It is an academic discipline of immense scope. Psychologists seek an understanding of the emergent properties of brains, and all the variety of phenomena linked to those emergent properties, joining this way the broader neuroscientific group of researchers. As a social science it aims to understand individuals and groups by establishing general principles and researching specific cases.
Philosophy is the study of general and fundamental questions about existence, knowledge, values, reason, mind, and language. Such questions are often posed as problems to be studied or resolved. The term was probably coined by Pythagoras. Philosophical methods include questioning, critical discussion, rational argument, and systematic presentation. Classic philosophical questions include: Is it possible to know anything and to prove it? What is most real? Philosophers also pose more practical and concrete questions such as: Is there a best way to live? Is it better to be just or unjust? Do humans have free will?
Walter Pitts was born in Detroit, Michigan on April 23, 1923, the son of Walter and Marie (née Welsia). He was an autodidact who taught himself logic and mathematics and was able to read several languages including Greek and Latin. He is widely remembered for having spent three days in a library, at the age of 12, reading Principia Mathematica and sent a letter to Bertrand Russell pointing out what he considered serious problems with the first half of the first volume. Russell was appreciative and invited him to study in the United Kingdom. Although Bertrand Russell invited him to Cambridge University at age 12, the offer was not taken up; however, Pitts did decide to become a logician.
Detroit is the largest and most populous city in the U.S. state of Michigan, the largest American city on the United States–Canada border, and the seat of Wayne County. The municipality of Detroit had a 2017 estimated population of 673,104, making it the 23rd-most populous city in the United States. The metropolitan area, known as Metro Detroit, is home to 4.3 million people, making it the second-largest in the Midwest after the Chicago metropolitan area. Regarded as a major cultural center, Detroit is known for its contributions to music and as a repository for art, architecture and design.
Michigan is a state in the Great Lakes and Midwestern regions of the United States. The state's name, Michigan, originates from the Ojibwe word mishigamaa, meaning "large water" or "large lake". With a population of about 10 million, Michigan is the tenth most populous of the 50 United States, with the 11th most extensive total area, and is the largest state by total area east of the Mississippi River. Its capital is Lansing, and its largest city is Detroit. Metro Detroit is among the nation's most populous and largest metropolitan economies.
Autodidacticism or self-education is education without the guidance of masters or institutions. Generally, an autodidact is an individual who chooses the subject they will study, their studying material, and the studying rhythm and time. An autodidact may or may not have formal education, and their study may be either a complement or an alternative to it. Many notable contributions have been made by autodidacts.
Pitts probably continued to correspond with Bertrand Russell; and at the age of 15 he attended Russell's lectures at the University of Chicago.He stayed there, without registering as a student. While there, in 1938 he met Jerome Lettvin, a pre-medical student, and the two became close friends. Russell was a visiting professor at the University of Chicago in the fall of 1938, and he directed Pitts to study with the logician Rudolf Carnap. Pitts met Carnap at Chicago by walking into his office during office hours, and presenting him with an annotated version of Carnap's recent book on logic, The Logical Syntax of Language. Since Pitts did not introduce himself, Carnap spent months searching for him, and, when he found him, he obtained for him a menial job at the university and had Pitts study with him. Pitts at the time was homeless and without income. He mastered Carnap's abstract logic, then met with and was intrigued by the work of the Ukrainian mathematical physicist Nicolas Rashevsky, who was also at Chicago and was the founder of mathematical biophysics, remodeling biology on the structure of the physical sciences and mathematical logic. Pitts also worked closely with the mathematician Alston Scott Householder, who was a member of Rashevsky's group.
The University of Chicago is a private research university in Chicago, Illinois. Founded in 1890, the school is located on a 217-acre campus in Chicago's Hyde Park neighborhood, near Lake Michigan. The University of Chicago holds top-ten positions in various national and international rankings.
Jerome Ysroael Lettvin, often known as Jerry Lettvin, was an American cognitive scientist, and Professor of Electrical and Bioengineering and Communications Physiology at the Massachusetts Institute of Technology (MIT). He is best known as the lead author of the paper, "What the Frog's Eye Tells the Frog's Brain" (1959), one of the most cited papers in the Science Citation Index. He wrote it along with Humberto Maturana, Warren McCulloch and Walter Pitts, and in the paper they gave special thanks and mention to Oliver Selfridge at MIT. Lettvin carried out neurophysiological studies in the spinal cord, made the first demonstration of "feature detectors" in the visual system, and studied information processing in the terminal branches of single axons. Around 1969, he originated the term "grandmother cell" to illustrate the logical inconsistency of the concept.
Pre-medical is an educational track that undergraduate students in the United States and Canada pursue prior to becoming medical students. It involves activities that prepare a student for medical school, such as pre-med coursework, volunteer activities, clinical experience, research, and the application process. Some pre-med programs providing broad preparation are referred to as “pre-professional” and may simultaneously prepare students for entry into a variety of first professional degree or graduate school programs that require similar prerequisites.
Later Warren McCulloch also arrived at the University of Chicago, and in early 1942 invited Pitts, who was still homeless, together with Lettvin to live with his family. In the evenings, McCulloch and Pitts collaborated. Pitts was familiar with the work of Gottfried Leibniz on computing and they considered the question of whether the nervous system could be considered a kind of universal computing device as described by Leibniz. This led to their seminal neural networks paper "A Logical Calculus of Ideas Immanent in Nervous Activity". After five years of unofficial studies, the University of Chicago awarded Pitts an Associate of Arts (his only earned degree) for his work on the paper.
A neural network is a network or circuit of neurons, or in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus a neural network is either a biological neural network, made up of real biological neurons, or an artificial neural network, for solving artificial intelligence (AI) problems. The connections of the biological neuron are modeled as weights. A positive weight reflects an excitatory connection, while negative values mean inhibitory connections. All inputs are modified by a weight and summed. This activity is referred as a linear combination. Finally, an activation function controls the amplitude of the output. For example, an acceptable range of output is usually between 0 and 1, or it could be −1 and 1.
In 1943, Lettvin introduced Pitts to Norbert Wiener at the Massachusetts Institute of Technology. Their first meeting, where they discussed Wiener's proof of the ergodic theorem, went so well that Pitts moved to Greater Boston to work with Wiener. While Pitts was an unofficial student under the aegis of Wiener at MIT until their acrimonious parting in 1952, he formally enrolled as a graduate student in the physics department during the 1943-1944 academic year and in the electrical engineering-computer science department from 1956-1958.
Norbert Wiener was an American mathematician and philosopher. He was a professor of mathematics at the Massachusetts Institute of Technology (MIT). A child prodigy, Wiener later became an early researcher in stochastic and mathematical noise processes, contributing work relevant to electronic engineering, electronic communication, and control systems.
Massachusetts Institute of Technology (MIT) is a private research university in Cambridge, Massachusetts. The Institute is a land-grant, sea-grant, and space-grant university, with an urban campus that extends more than a mile (1.6 km) alongside the Charles River. The Institute also encompasses a number of major off-campus facilities such as the MIT Lincoln Laboratory, the Bates Center, and the Haystack Observatory, as well as affiliated laboratories such as the Broad and Whitehead Institutes. Founded in 1861 in response to the increasing industrialization of the United States, MIT adopted a European polytechnic university model and stressed laboratory instruction in applied science and engineering. It has since played a key role in the development of many aspects of modern science, engineering, mathematics, and technology, and is widely known for its innovation and academic strength, making it one of the most prestigious institutions of higher learning in the world.
Ergodic theory is a branch of mathematics that studies dynamical systems with an invariant measure and related problems. Its initial development was motivated by problems of statistical physics.
In 1944, Pitts was hired by Kellex Corporation (later acquired in 1950 by Vitro Corporation) in New York City, part of the Atomic Energy Project.
The Kellex Corporation was a wholly owned subsidiary of M. W. Kellogg Company. Kellex was formed in 1942 so that Kellogg's operations relating to the Manhattan Project could be kept separate and secret. "Kell" stood for "Kellogg" and "X" for secret. The new company's goal was to design a facility for the production of enriched uranium through gaseous diffusion. In gaseous diffusion, isotopes of Uranium-235 could be separated from Uranium-238 by turning uranium metal into uranium hexafluoride gas and straining it through a barrier material.
Vitro Corporation was a major United States defense contractor which became part of BAE Systems Inc. in 1999.
The Manhattan Project was a research and development undertaking during World War II that produced the first nuclear weapons. It was led by the United States with the support of the United Kingdom and Canada. From 1942 to 1946, the project was under the direction of Major General Leslie Groves of the U.S. Army Corps of Engineers. Nuclear physicist Robert Oppenheimer was the director of the Los Alamos Laboratory that designed the actual bombs. The Army component of the project was designated the Manhattan District; Manhattan gradually superseded the official codename, Development of Substitute Materials, for the entire project. Along the way, the project absorbed its earlier British counterpart, Tube Alloys. The Manhattan Project began modestly in 1939, but grew to employ more than 130,000 people and cost nearly US$2 billion. Over 90% of the cost was for building factories and to produce fissile material, with less than 10% for development and production of the weapons. Research and production took place at more than 30 sites across the United States, the United Kingdom, and Canada.
From 1946, Pitts was a core member and involved with the Macy conferences, whose principal purpose was to set the foundations for a general science of the workings of the human mind.
In 1951, Wiener convinced Jerome Wiesner to hire some physiologists of the nervous system. A group was established with Pitts, Lettvin, McCulloch, and Pat Wall. Pitts wrote a large dissertation on the properties of neural nets connected in three dimensions. Lettvin described him as "in no uncertain sense the genius of the group … when you asked him a question, you would get back a whole textbook." Pitts never married. Pitts was also described as an eccentric, refusing to allow his name to be made publicly available. He continued to refuse all offers of advanced degrees or positions of authority at MIT, in part as he would have to sign his name.
In 1952, Wiener suddenly turned against McCulloch—his wife, Margaret Wiener, hated McCulloch—and broke off relations with anyone connected to him, including Pitts.
Although he remained employed as a research associate in the Research Laboratory of Electronics at MIT "as little more than a technicality"for the rest of his life, Pitts became increasingly socially isolated. In 1959, the paradigmatic "What the Frog’s Eye Tells the Frog’s Brain" (credited to Humberto Maturana, Lettvin, McCulloch and Pitts) conclusively demonstrated that "analog processes in the eye were doing at least part of the interpretive work" in image processing as opposed to "the brain computing information digital neuron by digital neuron using the exacting implement of mathematical logic", leading Pitts to burn his unpublished doctoral dissertation on probabilistic three dimensional neural networks and years of unpublished research. He took little further interest in work, excepting only a collaboration with Lettvin and Robert Gesteland which produced a paper on olfaction in 1965.
Pitts died in 1969 of bleeding esophageal varices, a condition usually associated with cirrhosis and alcoholism.
The nervous system is a highly complex part of an animal that coordinates its actions and sensory information by transmitting signals to and from different parts of its body. The nervous system detects environmental changes that impact the body, then works in tandem with the endocrine system to respond to such events. Nervous tissue first arose in wormlike organisms about 550 to 600 million years ago. In vertebrates it consists of two main parts, the central nervous system (CNS) and the peripheral nervous system (PNS). The CNS consists of the brain and spinal cord. The PNS consists mainly of nerves, which are enclosed bundles of the long fibers or axons, that connect the CNS to every other part of the body. Nerves that transmit signals from the brain are called motor or efferent nerves, while those nerves that transmit information from the body to the CNS are called sensory or afferent. Spinal nerves serve both functions and are called mixed nerves. The PNS is divided into three separate subsystems, the somatic, autonomic, and enteric nervous systems. Somatic nerves mediate voluntary movement. The autonomic nervous system is further subdivided into the sympathetic and the parasympathetic nervous systems. The sympathetic nervous system is activated in cases of emergencies to mobilize energy, while the parasympathetic nervous system is activated when organisms are in a relaxed state. The enteric nervous system functions to control the gastrointestinal system. Both autonomic and enteric nervous systems function involuntarily. Nerves that exit from the cranium are called cranial nerves while those exiting from the spinal cord are called spinal nerves.
Warren Sturgis McCulloch was an American neurophysiologist and cybernetician, known for his work on the foundation for certain brain theories and his contribution to the cybernetics movement. Along with Walter Pitts, McCulloch created computational models based on mathematical algorithms called threshold logic which split the inquiry into two distinct approaches, one approach focused on biological processes in the brain and the other focused on the application of neural networks to artificial intelligence.
Rudolf Carnap was a German-language philosopher who was active in Europe before 1935 and in the United States thereafter. He was a major member of the Vienna Circle and an advocate of logical positivism. He is considered "one of the giants among twentieth-century philosophers."
The Vienna Circle of Logical Empiricism was a group of philosophers and scientists drawn from the natural and social sciences, logic and mathematics who met regularly from 1924 to 1936 at the University of Vienna, chaired by Moritz Schlick.
Connectionism is an approach in the fields of cognitive science, that hopes to explain mental phenomena using artificial neural networks (ANN).
An artificial neuron is a mathematical function conceived as a model of biological neurons, a neural network. Artificial neurons are elementary units in an artificial neural network. The artificial neuron receives one or more inputs and sums them to produce an output. Usually each input is separately weighted, and the sum is passed through a non-linear function known as an activation function or transfer function. The transfer functions usually have a sigmoid shape, but they may also take the form of other non-linear functions, piecewise linear functions, or step functions. They are also often monotonically increasing, continuous, differentiable and bounded. The thresholding function has inspired building logic gates referred to as threshold logic; applicable to building logic circuits resembling brain processing. For example, new devices such as memristors have been extensively used to develop such logic in recent times.
The year 1943 in science and technology involved some significant events, listed below.
The First Draft of a Report on the EDVAC is an incomplete 101-page document written by John von Neumann and distributed on June 30, 1945 by Herman Goldstine, security officer on the classified ENIAC project. It contains the first published description of the logical design of a computer using the stored-program concept, which has controversially come to be known as the von Neumann architecture.
The Calculus ratiocinator is a theoretical universal logical calculation framework, a concept described in the writings of Gottfried Leibniz, usually paired with his more frequently mentioned characteristica universalis, a universal conceptual language.
A Hopfield network is a form of recurrent artificial neural network popularized by John Hopfield in 1982, but described earlier by Little in 1974. Hopfield nets serve as content-addressable ("associative") memory systems with binary threshold nodes. They are guaranteed to converge to a local minimum and, therefore, may converge to a false pattern rather than the stored pattern. Hopfield networks also provide a model for understanding human memory.
A neural circuit is a population of neurons interconnected by synapses to carry out a specific function when activated. Neural circuits interconnect to one another to form large scale brain networks. Biological neural networks have inspired the design of artificial neural networks, but artificial neural networks are usually not strict copies of their biological counterparts.
Neural computation is the hypothetical information processing performed by networks of neurons. Neural computation is affiliated with the philosophical tradition known as Computational theory of mind, also referred to as computationalism, which advances the thesis that neural computation explains cognition. The first persons to propose an account of neural activity as being computational was Warren McCullock and Walter Pitts in their seminal 1943 paper, A Logical Calculus of the Ideas Immanent in Nervous Activity. There are three general branches of computationalism, including classicism, connectionism, and computational neuroscience. All three branches agree that cognition is computation, however they disagree on what sorts of computations constitute cognition. The classicism tradition believes that computation in the brain is digital, analogous with digital computing. Both connectionism and computational neuroscience do not require that the computations which realize cognition are necessarily digital computations. However, the two branches greatly disagree upon which sorts of experimental data should be used to construct explanatory models of cognitive phenomena. Connectionists rely upon behavioral evidence to construct models to explain cognitive phenomenon, whereas computational neuroscience leverages neuroanatomical and neurophysiological information to construct mathematical models which explain cognition.
Howard Jerome Keisler is an American mathematician, currently professor emeritus at University of Wisconsin–Madison. His research has included model theory and non-standard analysis.
Memory is the process of storing and recalling information that was previously acquired. Memory occurs through three fundamental stages: encoding, storage, and retrieval. Storing refers to the process of placing newly acquired information into memory, which is modified in the brain for easier storage. Encoding this information makes the process of retrieval easier for the brain where it can be recalled and brought into conscious thinking. Modern memory psychology differentiates between the two distinct types of memory storage: short-term memory and long-term memory. Several models of memory have been proposed over the past century, some of them suggesting different relationships between short- and long-term memory to account for different ways of storing memory.
Nicolas Rashevsky was an American theoretical physicist who was one of the pioneers of mathematical biology, and is also considered the father of mathematical biophysics and theoretical biology.
The Society for Mathematical Biology (SMB) is an international association co-founded in 1972 in USA by Drs.George Karreman, Herbert Daniel Landahl and by Anthony Bartholomay for the furtherance of joint scientific activities between Mathematics and Biology research communities,. The society publishes the Bulletin of Mathematical Biology, as well as the SMB annual newsletter.
Network of human nervous system comprises nodes that are connected by links. The connectivity may be viewed anatomically, functionally, or electrophysiologically. These are presented in several Wikipedia articles that include Connectionism, Biological neural network, Artificial neural network, Computational neuroscience, as well as in several books by Ascoli, G. A. (2002), Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011), Gerstner, W., & Kistler, W. (2002), and Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986) among others. The focus of this article is a comprehensive view of modeling a neural network. Once an approach based on the perspective and connectivity is chosen, the models are developed at microscopic, mesoscopic, or macroscopic (system) levels. Computational modeling refers to models that are developed using computing tools.
There was just one person who wasn’t happy about the reunion: Wiener's wife. Margaret Wiener was, by all accounts, a controlling, conservative prude—and she despised McCulloch's influence on her husband. McCulloch hosted wild get-togethers at his family farm in Old Lyme, Connecticut, where ideas roamed free and everyone went skinny-dipping. It had been one thing when McCulloch was in Chicago, but now he was coming to Cambridge and Margaret wouldn’t have it. And so she invented a story. She sat Wiener down and informed him that when their daughter, Barbara, had stayed at McCulloch's house in Chicago, several of "his boys" had seduced her. Wiener immediately sent an angry telegram to Wiesner: "Please inform [Pitts and Lettvin] that all connection between me and your projects is permanently abolished. They are your problem. Wiener." He never spoke to Pitts again.