Brain-reading or thought identification uses the responses of multiple voxels in the brain evoked by stimulus then detected by fMRI in order to decode the original stimulus. Advances in research have made this possible by using human neuroimaging to decode a person's conscious experience based on non-invasive measurements of an individual's brain activity. [1] Brain reading studies differ in the type of decoding (i.e. classification, identification and reconstruction) employed, the target (i.e. decoding visual patterns, auditory patterns, cognitive states), and the decoding algorithms (linear classification, nonlinear classification, direct reconstruction, Bayesian reconstruction, etc.) employed.
In 2024 -2025, professor of neuropsychology Barbara Sahakian qualified, "A lot of neuroscientists in the field are very cautious and say we can't talk about reading individuals' minds, and right now that is very true, but we're moving ahead so rapidly, it's not going to be that long before we will be able to tell whether someone's making up a story, or whether someone intended to do a crime with a certain degree of certainty." [2]
Identification of complex natural images is possible using voxels from early and anterior visual cortex areas forward of them (visual areas V3A, V3B, V4, and the lateral occipital) together with Bayesian inference. This brain reading approach uses three components: [3] a structural encoding model that characterizes responses in early visual areas; a semantic encoding model that characterizes responses in anterior visual areas; and a Bayesian prior that describes the distribution of structural and semantic scene statistics. [3]
Experimentally the procedure is for subjects to view 1750 black and white natural images that are correlated with voxel activation in their brains. Then subjects viewed another 120 novel target images, and information from the earlier scans is used reconstruct them. Natural images used include pictures of a seaside cafe and harbor, performers on a stage, and dense foliage. [3]
In 2008 IBM applied for a patent on how to extract mental images of human faces from the human brain. It uses a feedback loop based on brain measurements of the fusiform gyrus area in the brain which activates proportionate with degree of facial recognition. [4]
In 2011, a team led by Shinji Nishimoto used only brain recordings to partially reconstruct what volunteers were seeing. The researchers applied a new model, about how moving object information is processed in human brains, while volunteers watched clips from several videos. An algorithm searched through thousands of hours of external YouTube video footage (none of the videos were the same as the ones the volunteers watched) to select the clips that were most similar. [5] [6] The authors have uploaded demos comparing the watched and the computer-estimated videos. [7] [8]
In 2017 a face perception study in monkeys reported the reconstruction of human faces by analyzing electrical activity from 205 neurons. [9] [10]
In 2023 image reconstruction was reported utilizing Stable Diffusion on human brain activity obtained via fMRI. [11] [12]
In 2024, a study demonstrated that images imagined in the mind, without visual stimulation, can be reconstructed from fMRI brain signals utilizing machine learning and generative AI technology. [13] [14] [15] Another 2024 study reported the reconstruction of images from EEG. [16]
Brain-reading has been suggested as an alternative to polygraph machines as a form of lie detection. [17] Another alternative to polygraph machines is blood oxygenated level dependent functional MRI technology. This technique involves the interpretation of the local change in the concentration of oxygenated hemoglobin in the brain, although the relationship between this blood flow and neural activity is not yet completely understood. [17] Another technique to find concealed information is brain fingerprinting, which uses EEG to ascertain if a person has a specific memory or information by identifying P300 event related potentials. [18]
A number of concerns have been raised about the accuracy and ethical implications of brain-reading for this purpose. Laboratory studies have found rates of accuracy of up to 85%; however, there are concerns about what this means for false positive results among non-criminal populations: "If the prevalence of "prevaricators" in the group being examined is low, the test will yield far more false-positive than true-positive results; about one person in five will be incorrectly identified by the test." [17] Ethical problems involved in the use of brain-reading as lie detection include misapplications due to adoption of the technology before its reliability and validity can be properly assessed and due to misunderstanding of the technology, and privacy concerns due to unprecedented access to individual's private thoughts. [17] However, it has been noted that the use of polygraph lie detection carries similar concerns about the reliability of the results [17] and violation of privacy. [19]
Brain-reading has also been proposed as a method of improving human–machine interfaces, by the use of EEG to detect relevant brain states of a human. [20] In recent years, there has been a rapid increase in patents for technology involved in reading brainwaves, rising from fewer than 400 from 2009–2012 to 1600 in 2014. [21] These include proposed ways to control video games via brain waves and "neuro-marketing" to determine someone's thoughts about a new product or advertisement.[ citation needed ]
Emotiv Systems, an Australian electronics company, has demonstrated a headset that can be trained to recognize a user's thought patterns for different commands. Tan Le demonstrated the headset's ability to manipulate virtual objects on screen, and discussed various future applications for such brain-computer interface devices, from powering wheel chairs to replacing the mouse and keyboard. [22]
It is possible to track which of two forms of rivalrous binocular illusions a person was subjectively experiencing from fMRI signals. [23]
When humans think of an object, such as a screwdriver, many different areas of the brain activate. Marcel Just and his colleague, Tom Mitchell, have used fMRI brain scans to teach a computer to identify the various parts of the brain associated with specific thoughts. [24] This technology also yielded a discovery: similar thoughts in different human brains are surprisingly similar neurologically. To illustrate this, Just and Mitchell used their computer to predict, based on nothing but fMRI data, which of several images a volunteer was thinking about. The computer was 100% accurate, but so far the machine is only distinguishing between 10 images. [24]
The category of event which a person freely recalls can be identified from fMRI before they say what they remembered. [25]
16 December 2015, a study conducted by Toshimasa Yamazaki at Kyushu Institute of Technology found that during a rock-paper-scissors game a computer was able to determine the choice made by the subjects before they moved their hand. An EEG was used to measure activity in the Broca's area to see the words two seconds before the words were uttered. [26] [27] [28]
In 2023, the University of Texas in Austin trained a non-invasive brain decoder to translate volunteers' brainwaves into the GPT-1 language model. After lengthy training on each individual volunteer, the decoder usually failed to reconstruct the exact words, but could nevertheless reconstruct meanings close enough that the decoder could, most of the time, identify what timestamp of a given book the subject was listening to. [29] [30]
Statistical analysis of EEG brainwaves has been claimed to allow the recognition of phonemes, [31] and (in 1999) at a 60% to 75% level color and visual shape words. [32]
On 31 January 2012 Brian Pasley and colleagues of University of California Berkeley published their paper in PLoS Biology wherein subjects' internal neural processing of auditory information was decoded and reconstructed as sound on computer by gathering and analyzing electrical signals directly from subjects' brains. [33] The research team conducted their studies on the superior temporal gyrus, a region of the brain that is involved in higher order neural processing to make semantic sense from auditory information. [34] The research team used a computer model to analyze various parts of the brain that might be involved in neural firing while processing auditory signals. Using the computational model, scientists were able to identify the brain activity involved in processing auditory information when subjects were presented with recording of individual words. [35] Later, the computer model of auditory information processing was used to reconstruct some of the words back into sound based on the neural processing of the subjects. However the reconstructed sounds were not of good quality and could be recognized only when the audio wave patterns of the reconstructed sound were visually matched with the audio wave patterns of the original sound that was presented to the subjects. [35] However this research marks a direction towards more precise identification of neural activity in cognition.
Some researchers in 2008 were able to predict, with 60% accuracy, whether a subject was going to push a button with their left or right hand. This is notable, not just because the accuracy is better than chance, but also because the scientists were able to make these predictions up to 10 seconds before the subject acted – well before the subject felt they had decided. [36] This data is even more striking in light of other research suggesting that the decision to move, and possibly the ability to cancel that movement at the last second, [37] may be the results of unconscious processing. [38]
John Dylan-Haynes has also demonstrated that fMRI can be used to identify whether a volunteer is about to add or subtract two numbers in their head. [24]
Neural decoding techniques have been used to test theories about the predictive brain, and to investigate how top-down predictions affect brain areas such as the visual cortex. Studies using fMRI decoding techniques have found that predictable sensory events [39] and the expected consequences of our actions [40] are better decoded in visual brain areas, suggesting that prediction 'sharpens' representations in line with expectations.
It has also been shown that brain-reading can be achieved in a complex virtual environment. [41]
Just and Mitchell also claim they are beginning to be able to identify kindness, hypocrisy, and love in the brain. [24]
In 2013 a project led by University of California Berkeley professor John Chuang published findings on the feasibility of brainwave-based computer authentication as a substitute for passwords. Improvements in the use of biometrics for computer authentication has continually improved since the 1980s, but this research team was looking for a method faster and less intrusive than today's retina scans, fingerprinting, and voice recognition. The technology chosen to improve security measures is an electroencephalogram (EEG), or brainwave measurer, to improve passwords into "pass thoughts." Using this method Chuang and his team were able to customize tasks and their authentication thresholds to the point where they were able to reduce error rates under 1%, significantly better than other recent methods. In order to better attract users to this new form of security the team is still researching mental tasks that are enjoyable for the user to perform while having their brainwaves identified. In the future this method could be as cheap, accessible, and straightforward as thought itself. [42]
John-Dylan Haynes states that fMRI can also be used to identify recognition in the brain. He provides the example of a criminal being interrogated about whether he recognizes the scene of the crime or murder weapons. [24]
In classification, a pattern of activity across multiple voxels is used to determine the particular class from which the stimulus was drawn. [43] Many studies have classified visual stimuli, but this approach has also been used to classify cognitive states.[ citation needed ]
In reconstruction brain reading the aim is to create a literal picture of the image that was presented. Early studies used voxels from early visual cortex areas (V1, V2, and V3) to reconstruct geometric stimuli made up of flickering checkerboard patterns. [44] [45]
EEG has also been used to identify recognition of specific information or memories by the P300 event related potential, which has been dubbed 'brain fingerprinting'. [46]
Brain-reading accuracy is increasing steadily as the quality of the data and the complexity of the decoding algorithms improve. In one recent experiment it was possible to identify which single image was being seen from a set of 120. [47] In another it was possible to correctly identify 90% of the time which of two categories the stimulus came and the specific semantic category (out of 23) of the target image 40% of the time. [3]
It has been noted that so far brain-reading is limited. "In practice, exact reconstructions are impossible to achieve by any reconstruction algorithm on the basis of brain activity signals acquired by fMRI. This is because all reconstructions will inevitably be limited by inaccuracies in the encoding models and noise in the measured signals. Our results [ who? ] demonstrate that the natural image prior is a powerful (if unconventional) tool for mitigating the effects of these fundamental limitations. A natural image prior with only six million images is sufficient to produce reconstructions that are structurally and semantically similar to a target image." [3]
With brain scanning technology becoming increasingly accurate, experts predict important debates over how and when it should be used. One potential area of application is criminal law. Haynes states that simply refusing to use brain scans on suspects also prevents the wrongly accused from proving their innocence. [2] US scholars generally believe that involuntary brain reading, and involuntary polygraph tests, would violate the Fifth Amendment's right to not self-incriminate. [48] [49] One perspective is to consider whether brain imaging is like testimony, or instead like DNA, blood, or semen. Paul Root Wolpe, director of the Center for Ethics at Emory University in Atlanta predicts that this question will be decided by a Supreme Court case. [50]
In other countries outside the United States, thought identification has already been used in criminal law. In 2008 an Indian woman was convicted of murder after an EEG of her brain allegedly revealed that she was familiar with the circumstances surrounding the poisoning of her ex-fiancé. [50] Some neuroscientists and legal scholars doubt the validity of using thought identification as a whole for anything past research on the nature of deception and the brain. [51]
The Economist cautioned people to be "afraid" of the future impact, and some ethicists argue that privacy laws should protect private thoughts. Legal scholar Hank Greely argues that the court systems could benefit from such technology, and neuroethicist Julian Savulescu states that brain data is not fundamentally different from other types of evidence. [52] In Nature , journalist Liam Drew writes about emerging projects to attach brain-reading devices to speech synthesizers or other output devices for the benefit of tetraplegics. Such devices could create concerns of accidentally broadcasting the patient's "inner thoughts" rather than merely conscious speech. [53]
Psychologist John-Dylan Haynes experienced breakthroughs in brain imaging research in 2006 by using fMRI. This research included new findings on visual object recognition, tracking dynamic mental processes, lie detecting, and decoding unconscious processing. The combination of these four discoveries revealed such a significant amount of information about an individual's thoughts that Haynes termed it "brain reading". [1]
The fMRI has allowed research to expand by significant amounts because it can track the activity in an individual's brain by measuring the brain's blood flow. It is currently thought to be the best method for measuring brain activity, which is why it has been used in multiple research experiments in order to improve the understanding of how doctors and psychologists can identify thoughts. [54]
In a 2020 study, AI using implanted electrodes could correctly transcribe a sentence read aloud from a fifty-sentence test set 97% of the time, given 40 minutes of training data per participant. [55]
Experts are unsure of how far thought identification can expand, but Marcel Just believed in 2014 that in 3–5 years there will be a machine that is able to read complex thoughts such as 'I hate so-and-so'. [50]
Donald Marks, founder and chief science officer of MMT, is working on playing back thoughts individuals have after they have already been recorded. [56]
Researchers at the University of California Berkeley have already been successful in forming, erasing, and reactivating memories in rats. Marks says they are working on applying the same techniques to humans. This discovery could be monumental for war veterans who suffer from PTSD. [56]
Further research is also being done in analyzing brain activity during video games to detect criminals, neuromarketing, and using brain scans in government security checks. [50] [54]
The episode Black Hole of American medical drama House , which aired on 15 March 2010, featured an experimental "cognitive imaging" device that supposedly allowed seeing into a patient's subconscious mind. The patient was first put in a preparation phase of six hours while watching video clips, attached to a neuroimaging device looking like electroencephalography or functional near-infrared spectroscopy, to train the neuroimaging classifier. Then the patient was put under twilight anesthesia, and the same device was used to try to infer what was going through the patient's mind. The fictional episode somewhat anticipated the study by Nishimoto et al. published the following year, in which fMRI was used instead. [5] [6] [7] [8]
In the movie Dumb and Dumber To', one scene shows a brain reader.
In the Henry Danger episode, "Dream Busters," a machine shows Henry's dream.
Functional magnetic resonance imaging or functional MRI (fMRI) measures brain activity by detecting changes associated with blood flow. This technique relies on the fact that cerebral blood flow and neuronal activation are coupled. When an area of the brain is in use, blood flow to that region also increases.
Functional neuroimaging is the use of neuroimaging technology to measure an aspect of brain function, often with a view to understanding the relationship between activity in certain brain areas and specific mental functions. It is primarily used as a research tool in cognitive neuroscience, cognitive psychology, neuropsychology, and social neuroscience.
Neurofeedback is a form of biofeedback that uses electrical potentials in the brain to reinforce desired brain states through operant conditioning. This process is non-invasive and typically collects brain activity data using electroencephalography (EEG). Several neurofeedback protocols exist, with potential additional benefit from use of quantitative electroencephalography (QEEG) or functional magnetic resonance imaging (fMRI) to localize and personalize treatment. Related technologies include functional near-infrared spectroscopy-mediated (fNIRS) neurofeedback, hemoencephalography biofeedback (HEG), and fMRI biofeedback.
An event-related potential (ERP) is the measured brain response that is the direct result of a specific sensory, cognitive, or motor event. More formally, it is any stereotyped electrophysiological response to a stimulus. The study of the brain in this way provides a noninvasive means of evaluating brain functioning.
A brain–computer interface (BCI), sometimes called a brain–machine interface (BMI), is a direct communication link between the brain's electrical activity and an external device, most commonly a computer or robotic limb. BCIs are often directed at researching, mapping, assisting, augmenting, or repairing human cognitive or sensory-motor functions. They are often conceptualized as a human–machine interface that skips the intermediary of moving body parts (hands...), although they also raise the possibility of erasing the distinction between brain and machine. BCI implementations range from non-invasive and partially invasive to invasive, based on how physically close electrodes are to brain tissue.
Neurotechnology encompasses any method or electronic device which interfaces with the nervous system to monitor or modulate neural activity.
Alpha waves, or the alpha rhythm, are neural oscillations in the frequency range of 8–12 Hz likely originating from the synchronous and coherent electrical activity of thalamic pacemaker cells in humans. Historically, they are also called "Berger's waves" after Hans Berger, who first described them when he invented the EEG in 1924.
Brainwave entrainment, also referred to as brainwave synchronization or neural entrainment, refers to the observation that brainwaves will naturally synchronize to the rhythm of periodic external stimuli, such as flickering lights, speech, music, or tactile stimuli.
Neural oscillations, or brainwaves, are rhythmic or repetitive patterns of neural activity in the central nervous system. Neural tissue can generate oscillatory activity in many ways, driven either by mechanisms within individual neurons or by interactions between neurons. In individual neurons, oscillations can appear either as oscillations in membrane potential or as rhythmic patterns of action potentials, which then produce oscillatory activation of post-synaptic neurons. At the level of neural ensembles, synchronized activity of large numbers of neurons can give rise to macroscopic oscillations, which can be observed in an electroencephalogram. Oscillatory activity in groups of neurons generally arises from feedback connections between the neurons that result in the synchronization of their firing patterns. The interaction between neurons can give rise to oscillations at a different frequency than the firing frequency of individual neurons. A well-known example of macroscopic neural oscillations is alpha activity.
The P300 (P3) wave is an event-related potential (ERP) component elicited in the process of decision making. It is considered to be an endogenous potential, as its occurrence links not to the physical attributes of a stimulus, but to a person's reaction to it. More specifically, the P300 is thought to reflect processes involved in stimulus evaluation or categorization.
Neuroimaging is the use of quantitative (computational) techniques to study the structure and function of the central nervous system, developed as an objective way of scientifically studying the healthy human brain in a non-invasive manner. Increasingly it is also being used for quantitative research studies of brain disease and psychiatric illness. Neuroimaging is highly multidisciplinary involving neuroscience, computer science, psychology and statistics, and is not a medical specialty. Neuroimaging is sometimes confused with neuroradiology.
Electrocorticography (ECoG), a type of intracranial electroencephalography (iEEG), is a type of electrophysiological monitoring that uses electrodes placed directly on the exposed surface of the brain to record electrical activity from the cerebral cortex. In contrast, conventional electroencephalography (EEG) electrodes monitor this activity from outside the skull. ECoG may be performed either in the operating room during surgery or outside of surgery. Because a craniotomy is required to implant the electrode grid, ECoG is an invasive procedure.
A mental representation, in philosophy of mind, cognitive psychology, neuroscience, and cognitive science, is a hypothetical internal cognitive symbol that represents external reality or its abstractions.
Connectomics is the production and study of connectomes: comprehensive maps of connections within an organism's nervous system. More generally, it can be thought of as the study of neuronal wiring diagrams with a focus on how structural connectivity, individual synapses, cellular morphology, and cellular ultrastructure contribute to the make up of a network. The nervous system is a network made of up to billions of connections and these connections are responsible for our thoughts, emotions, actions, memories, function and dysfunction. Therefore, the study of connectomics aims to advance our understanding of mental health and cognition by understanding how cells in the nervous system are connected and communicate. Because these structures are extremely complex, methods within this field use a high-throughput application of functional and structural neural imaging, most commonly magnetic resonance imaging (MRI), electron microscopy, and histological techniques in order to increase the speed, efficiency, and resolution of these nervous system maps. To date, tens of large scale datasets have been collected spanning the nervous system including the various areas of cortex, cerebellum, the retina, the peripheral nervous system and neuromuscular junctions.
Marcel Just is D. O. Hebb Professor of Psychology at Carnegie Mellon University. His research uses brain imaging (fMRI) in high-level cognitive tasks to study the neuroarchitecture of cognition. Just's areas of expertise include psycholinguistics, object recognition, and autism, with particular attention to cognitive and neural substrates. Just directs the Center for Cognitive Brain Imaging and is a member of the Center for the Neural Basis of Cognition at CMU.
Electroencephalography (EEG) is a method to record an electrogram of the spontaneous electrical activity of the brain. The biosignals detected by EEG have been shown to represent the postsynaptic potentials of pyramidal neurons in the neocortex and allocortex. It is typically non-invasive, with the EEG electrodes placed along the scalp using the International 10–20 system, or variations of it. Electrocorticography, involving surgical placement of electrodes, is sometimes called "intracranial EEG". Clinical interpretation of EEG recordings is most often performed by visual inspection of the tracing or quantitative EEG analysis.
Imagined speech is thinking in the form of sound – "hearing" one's own voice silently to oneself, without the intentional movement of any extremities such as the lips, tongue, or hands. Logically, imagined speech has been possible since the emergence of language, however, the phenomenon is most associated with its investigation through signal processing and detection within electroencephalograph (EEG) data as well as data obtained using alternative non-invasive, brain–computer interface (BCI) devices.
Dynamic functional connectivity (DFC) refers to the observed phenomenon that functional connectivity changes over a short time. Dynamic functional connectivity is a recent expansion on traditional functional connectivity analysis which typically assumes that functional networks are static in time. DFC is related to a variety of different neurological disorders, and has been suggested to be a more accurate representation of functional brain networks. The primary tool for analyzing DFC is fMRI, but DFC has also been observed with several other mediums. DFC is a recent development within the field of functional neuroimaging whose discovery was motivated by the observation of temporal variability in the rising field of steady state connectivity research.
Neuroprivacy, or "brain privacy," is a concept which refers to the rights people have regarding the imaging, extraction and analysis of neural data from their brains. This concept is highly related to fields like neuroethics, neurosecurity, and neurolaw, and has become increasingly relevant with the development and advancement of various neuroimaging technologies. Neuroprivacy is an aspect of neuroethics specifically regarding the use of neural information in legal cases, neuromarketing, surveillance and other external purposes, as well as corresponding social and ethical implications.
Neural synchrony is the correlation of brain activity across two or more people over time. In social and affective neuroscience, neural synchrony specifically refers to the degree of similarity between the spatio-temporal neural fluctuations of multiple people. This phenomenon represents the convergence and coupling of different people's neurocognitive systems, and it is thought to be the neural substrate for many forms of interpersonal dynamics and shared experiences. Some research also refers to neural synchrony as inter-brain synchrony, brain-to-brain coupling, inter-subject correlation, between-brain connectivity, or neural coupling. In the current literature, neural synchrony is notably distinct from intra-brain synchrony—sometimes also called neural synchrony—which denotes the coupling of activity across regions of a single individual's brain.
{{cite journal}}
: Cite journal requires |journal=
(help)Ronald Allen and Kristen Mace discern 'universal agreement' that the (Mind Reader Machine) is unacceptable.