Tsachy Weissman

Last updated
Itschak (Tsachy) Weissman
Alma mater Technion – Israel Institute of Technology
Known forInformation Theory and Communications, Statistical Signal Processing, Weissman score
Scientific career
FieldsInformation Theory, Digital Communications, Statistical Signal Processing, Applications
Institutions Stanford University

Tsachy (Itschak) Weissman is a Professor of Electrical Engineering at Stanford University. [1] He is the founding director of the Stanford Compression Forum. [1] His research interests include information theory, statistical signal processing, their applications, with recent emphasis on biological applications, in genomics in particular, lossless compression, lossy compression, delay-constrained and complexity-constrained compression and communication, network information theory, feedback communications, directed information, the interplay between estimation theory and information theory, entropy, noise reduction (denoising), filtering, prediction, sequential decision making, learning, and connections with probability, statistics, and computer science (as listed in Weissman's CV PDF link). [1]

Contents

He was the Senior Technical Advisor to the HBO show Silicon Valley , and namesake of the Weissman score therein. [2] Weissman is the co-inventor of the Discrete Universal Denoiser (DUDE) algorithm. [3]

On his personal website, Weissman has spoken out against intimidation and sexual harassment in the information theory community. [4]

Education

Weissman received his Bachelor of Science in Electrical Engineering (Summa Cum Laude) in 1997, and his PhD (2001) from Technion – Israel Institute of Technology. [5]

Career

In 2002, Weissman joined Hewlett-Packard (HP) Laboratories as a researcher; in 2003, he became a Visiting Scientist at HP. [6] At HP, he was co-inventor of a denoising algorithm named the Discrete Universal Denoiser (DUDE).

Weissman became Assistant Professor of Electrical Engineering at Stanford University in 2003. [7] [8] [9] He was promoted to Associate Professor in 2010, [10] and professor in 2015. [11] He was named Fellow of the Institute of Electrical and Electronics Engineers IEEE in 2013 [12] for contributions to information theory and its applications in signal processing.

Patents

Tsachy Weissman has been granted 15 U.S. patents. [13]

Universal lossy compression methods

Patent number: 8320687
Abstract: The present invention provides methods for universal lossy compression that provide performance at or near the rate-distortion limit and that are based on universal, implementable lossy source coding algorithms.
Type: Grant
Filed: February 5, 2010
Date of Patent: November 27, 2012
Assignee: The Board of Trustees of the Leland Stanford Junior University
Inventors: Itschak Weissman, Shirin Jalali

Discrete universal denoising with error correction coding

Publication number: 20050289433
Abstract: A method of and system for denoising and decoding a noisy error correction coded signal received through a noise-introducing channel to produce a recovered signal. In one embodiment, noisy message blocks are separated from noisy check blocks in the noisy error correction coded signal. The noisy message blocks are denoised. Error correction decoding is performed on the denoised message blocks using the noisy check blocks to produce the recovered signal.
Type: Application
Filed: June 25, 2004
Publication date: December 29, 2005
Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger

Context-based denoiser that simultaneously updates probabilities for multiple contexts

Publication number: 20060070256
Abstract: A discrete, universal denoising method is applied to a noisy signal for which the source alphabet is typically large. The method exploits a priori information regarding expected characteristics of the signal. In particular, using characteristics of a continuous tone image such as continuity and small-scale symmetry allows definition of context classes containing large numbers of image contexts having similar statistical characteristics. Use of the context classes allows extraction of more reliable indications of the characteristic of a clean signal.
Type: Application
Filed: July 12, 2005
Publication date: April 6, 2006
Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Marcelo Weinberger, Sergio Verdu, Giovanni Motta

Denoising video

Patent number: 7420487
Abstract: A denoising process statistically processes a series of frames of a motion picture to construct respective data structures for the frames. Each data structure indicates for each of multiple contexts, occurrences of symbols that have the same context and are in the corresponding one of the frames. The data structures for multiple frames are combined to construct an enhanced data structure for one of the frames, and symbols in that frame are replaced with values determined using the enhanced data structure. Type: Grant
Filed: October 12, 2006
Date of Patent: September 2, 2008
Assignee: Hewlett-Packard Development Company, L.P.
Inventors: Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich, Gadiel Seroussi

Methods for compression using a denoiser

Publication number: 20060045360
Abstract: Various embodiments of the present invention provide a compression method and system that compresses received data by first denoising the data and then losslessly compressing the denoised data. Denoising removes high entropy features of the data to produce lower entropy, denoised data that can be efficiently compressed by a lossless compression technique. One embodiment of the invention is a universal lossy compression method obtained by cascading a denoising technique with a universal lossless compression method. Alternative embodiments include methods obtained by cascading a denoising technique with one or more lossy or lossless compression methods.
Type: Application
Filed: September 2, 2004
Publication date: March 2, 2006
Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger

Method and system for denoising signals

Publication number: 20110274225
Abstract: The application is directed to generally applicable denoising methods and systems for recovering, from a noise-corrupted signal, a cleaned signal equal to, or close to, the original, clean signal that suffered corruption due to one or more noise-inducing processes, devices, or media In a first pass, noise-corrupted-signal-reconstruction systems and methods receive an instance of one of many different types of neighborhood rules and use the received neighborhood rule to acquire statistics from a noisy signal. In a second pass, the noise-corrupted-signal-reconstruction systems and methods receive an instance of one of many different types of denoising rules, and use the received denoising rule to denoise a received, noisy signal in order to produce a cleaned signal.
Type: Application
Filed: July 18, 2011
Publication date: November 10, 2011
Inventor: Itschak Weissman


Method and system for optimizing denoising parameters using compressibility

Publication number: 20060047484
Abstract: In various embodiments of the present invention, a noisy signal denoiser is tuned and optimized by selecting denoiser parameters that provide relatively highly compressible denoiser output. When the original signal can be compared to the output of a denoiser, the denoiser can be accurately tuned and adjusted in order to produce a denoised signal that resembles as closely as possible the clear signal originally transmitted through a noise-introducing channel. However, when the clear signal is not available, as in many communications applications, other methods are needed. By adjusting the parameters to provide a denoised signal that is globally or locally maximally compressible, the denoiser can be optimized despite inaccessibility of the original, clear signal.
Type: Application
Filed: September 2, 2004
Publication date: March 2, 2006
Inventors: Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich

Discrete universal denoising with reliability information

Publication number: 20050289406
Abstract: A method of and system for generating reliability information for a noisy signal received through a noise-introducing channel. In one embodiment, symbol-transition probabilities are determined for the noise-introducing channel. Occurrences of metasymbols in the noisy signal are counted, each metasymbol providing a context for a symbol of the metasymbol. For each metasymbol occurring in the noisy signal, reliability information for each possible value of the symbol of the metasymbol is determined, the reliability information representing a probability that the value in the original signal corresponding to the symbol of the metasymbol assumed each of the possible values. In another embodiment, error correction coding may be performed by adding redundant data to an original signal prior to transmission by the noise-introducing channel and performing error correction decoding after transmission.
Type: Application
Filed: June 25, 2004
Publication date: December 29, 2005
Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Krishnamurthy Viswanathan

Method and system for producing variable length context models

Publication number: 20060047501
Abstract: Various embodiments of the present invention provide methods and systems for determining, representing, and using variable-length contexts in a variety of different computational applications. In one embodiment of the present invention, a balanced tree is used to represent all possible contexts of a fixed length, where the depth of the balanced tree is equal to the fixed length of the considered contexts. Then, in the embodiment, a pruning technique is used to sequentially coalesce the children of particular nodes in the tree in order to produce an unbalanced tree representing a set of variable-length contexts. The pruning method is selected, in one embodiment, to coalesce nodes, and, by doing so, to truncate the tree according to statistical considerations in order to produce a representation of a variably sized context model suitable for a particular application.
Type: Application
Filed: September 2, 2004
Publication date: March 2, 2006
Inventors: Gadiell Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich

Discrete denoising using blended counts

Publication number: 20060045218
Abstract: Various embodiments of the present invention relate to a discrete denoiser that replaces all of one type of symbol in a received, noisy signal with a replacement symbol in order to produce a recovered signal less distorted with respect to an originally transmitted, clean signal than the received, noisy signal. Certain, initially developed discrete denoisers employ an analysis of the number of occurrences of metasymbols within the received, noisy signal in order to select symbols for replacement, and to select the replacement symbols for the symbols that are replaced. Embodiments of the present invention use blended counts that are combinations of the occurrences of metasymbol families within a noisy signal, rather than counts of individual, single metasymbols, to determine the symbols to be replaced and the replacement symbols corresponding to them.
Type: Application
Filed: September 2, 2004
Publication date: March 2, 2006
Inventors: Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman

Enhanced denoising system utilizing incremental parsing

Publication number: 20060115017
Abstract: An apparatus for operating on a received signal that includes a noise-free signal that has been corrupted by a channel is disclosed. A memory stores a channel corruption function specifying the probability that a symbol having a value I was converted to a symbol having a value J by the channel, and a degradation function measuring the signal degradation that occurs if a symbol having the value I is replaced by symbol having a value J. The controller parses one of the received signal or the processed signal into phrases, and replaces one of the symbol having a value I in a context of that symbol in the received signal with a symbol having a value J if the replacement would reduce the estimated overall signal degradation in the processed signal. The context of a symbol depends on the phrase associated with the symbol.
Type: Application
Filed: November 29, 2004
Publication date: June 1, 2006
Inventors: Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman

Method and system for optimizing denoising parameters using compressibility

Patent number: 7436969
Abstract: In various embodiments of the present invention, a noisy signal denoiser is tuned and optimized by selecting denoiser parameters that provide relatively highly compressible denoiser output. When the original signal can be compared to the output of a denoiser, the denoiser can be accurately tuned and adjusted in order to produce a denoised signal that resembles as closely as possible the clear signal originally transmitted through a noise-introducing channel. However, when the clear signal is not available, as in many communications applications, other methods are needed. By adjusting the parameters to provide a denoised signal that is globally or locally maximally compressible, the denoiser can be optimized despite inaccessibility of the original, clear signal.
Type: Grant
Filed: September 2, 2004
Date of Patent: October 14, 2008
Assignee: Hewlett-Packard Development Company, L.P.
Inventors: Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich

Discrete universal denoising with reliability information

Patent number: 7269781
Abstract: A method of and system for generating reliability information for a noisy signal received through a noise-introducing channel. In one embodiment, symbol-transition probabilities are determined for the noise-introducing channel. Occurrences of metasymbols in the noisy signal are counted, each metasymbol providing a context for a symbol of the metasymbol. For each metasymbol occurring in the noisy signal, reliability information for each possible value of the symbol of the metasymbol is determined, the reliability information representing a probability that the value in the original signal corresponding to the symbol of the metasymbol assumed each of the possible values. In another embodiment, error correction coding may be performed by adding redundant data to an original signal prior to transmission by the noise-introducing channel and performing error correction decoding after transmission.
Type: Grant
Filed: June 25, 2004
Date of Patent: September 11, 2007
Assignee: Hewlett-Packard Development Company, L.P.
Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Krishnamurthy Viswanathan

Enhanced denoising system

Publication number: 20050163267
Abstract: A method and apparatus for processing a received digital signal that has been corrupted by a channel is disclosed. The method includes storing the received digital signal and receiving a partially corrected sequence of symbols that includes an output of a preliminary denoising system operating on the received digital signal. Information specifying a signal degradation function that measures the signal degradation that occurs if a symbol having the value I is replaced by a symbol having the value J is utilized to generate a processed digital signal by replacing each symbol having a value I in a context of that symbol in the received digital signal with a symbol having a value J if replacement reduces a measure of overall signal degradation in the processed digital signal relative to the received digital signal as measured by the degradation function and the partially corrected sequence of symbols.
Type: Application
Filed: January 26, 2004
Publication date: July 28, 2005
Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Marcelo Weinberger, Sergio Verdu

Method for correcting noise errors in a digital signal

Publication number: 20050097421
Abstract: An apparatus and method for processing a received signal that has been corrupted by a channel to generate a processed signal having less signal corruption than the received signal is disclosed. The apparatus stores the received signal, information specifying the probability that a symbol having a value I will be converted to a symbol having a value J by the channel, and information specifying a signal degradation function that measures the signal degradation that occurs if a symbol having the value I is replaced by symbol having a value J. The controller replaces each symbol having a value I in a context of that symbol in the received signal with a symbol having a value J that minimizes the overall signal degradation in the processed signal relative to the underlying noise-free signal as estimated via the observed statistics within that context.
Type: Application
Filed: October 17, 2003
Publication date: May 5, 2005
Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Marcelo Weinberger, Sergio Verdu

Books

Related Research Articles

Digital data Discrete, discontinuous representation of information

Digital data, in information theory and information systems, is information represented as a string of discrete symbols each of which can take on one of only a finite number of values from some alphabet, such as letters or digits. An example is a text document, which consists of a string of alphanumeric characters. The most common form of digital data in modern information systems is binary data, which is represented by a string of binary digits (bits) each of which can have one of two values, either 0 or 1.

Information theory is the scientific study of the quantification, storage, and communication of digital information. The field was fundamentally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering, and electrical engineering.

Signal processing Academic subfield of electrical engineering

Signal processing is an electrical engineering subfield that focuses on analysing, modifying, and synthesizing signals such as sound, images, and scientific measurements. Signal processing techniques can be used to improve transmission, storage efficiency and subjective quality and to also emphasize or detect components of interest in a measured signal.

In information theory, the Shannon–Hartley theorem tells the maximum rate at which information can be transmitted over a communications channel of a specified bandwidth in the presence of noise. It is an application of the noisy-channel coding theorem to the archetypal case of a continuous-time analog communications channel subject to Gaussian noise. The theorem establishes Shannon's channel capacity for such a communication link, a bound on the maximum amount of error-free information per time unit that can be transmitted with a specified bandwidth in the presence of the noise interference, assuming that the signal power is bounded, and that the Gaussian noise process is characterized by a known power or power spectral density. The law is named after Claude Shannon and Ralph Hartley.

Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions digital image processing may be modeled in the form of multidimensional systems. The generation and development of digital image processing are mainly affected by three factors: first, the development of computers; second, the development of mathematics ; third, the demand for a wide range of applications in environment, agriculture, military, industry and medical science has increased.

Golomb coding is a lossless data compression method using a family of data compression codes invented by Solomon W. Golomb in the 1960s. Alphabets following a geometric distribution will have a Golomb code as an optimal prefix code, making Golomb coding highly suitable for situations in which the occurrence of small values in the input stream is significantly more likely than large values.

Communication channel transmission channel

A communication channel refers either to a physical transmission medium such as a wire, or to a logical connection over a multiplexed medium such as a radio channel in telecommunications and computer networking. A channel is used to convey an information signal, for example a digital bit stream, from one or several senders to one or several receivers. A channel has a certain capacity for transmitting information, often measured by its bandwidth in Hz or its data rate in bits per second.

Coding theory Study of the properties of codes and their fitness

Coding theory is the study of the properties of codes and their respective fitness for specific applications. Codes are used for data compression, cryptography, error detection and correction, data transmission and data storage. Codes are studied by various scientific disciplines—such as information theory, electrical engineering, mathematics, linguistics, and computer science—for the purpose of designing efficient and reliable data transmission methods. This typically involves the removal of redundancy and the correction or detection of errors in the transmitted data.

Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree.

Discrete wavelet transform transform in numerical harmonic analysis

In numerical analysis and functional analysis, a discrete wavelet transform (DWT) is any wavelet transform for which the wavelets are discretely sampled. As with other wavelet transforms, a key advantage it has over Fourier transforms is temporal resolution: it captures both frequency and location information.

Bit plane

A bit plane of a digital discrete signal is a set of bits corresponding to a given bit position in each of the binary numbers representing the signal.

Peak signal-to-noise ratio (PSNR) is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. Because many signals have a very wide dynamic range, PSNR is usually expressed as a logarithmic quantity using the decibel scale.

Image noise

Image noise is random variation of brightness or color information in images, and is usually an aspect of electronic noise. It can be produced by the image sensor and circuitry of a scanner or digital camera. Image noise can also originate in film grain and in the unavoidable shot noise of an ideal photon detector. Image noise is an undesirable by-product of image capture that obscures the desired information.

Shannon–Weaver model

The Shannon–Weaver model of communication has been called the "mother of all models." Social Scientists use the term to refer to an integrated model of the concepts of information source, message, transmitter, signal, channel, noise, receiver, information destination, probability of error, encoding, decoding, information rate, channel capacity. However, some consider the name to be misleading, asserting that the most significant ideas were developed by Shannon alone.

In sound and music production, sonic artifact, or simply artifact, refers to sonic material that is accidental or unwanted, resulting from the editing or manipulation of a sound.

The decisive event which established the discipline of information theory, and brought it to immediate worldwide attention, was the publication of Claude E. Shannon's classic paper "A Mathematical Theory of Communication" in the Bell System Technical Journal in July and October 1948.

Autoencoder Neural network that learns efficient data encoding in an unsupervised manner

An autoencoder is a type of artificial neural network used to learn efficient codings of unlabeled data. The encoding is validated and refined by attempting to regenerate the input from the encoding. The autoencoder learns a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore insignificant data (“noise”).

Pulse-code modulation (PCM) is a method used to digitally represent sampled analog signals. It is the standard form of digital audio in computers, compact discs, digital telephony and other digital audio applications. In a PCM stream, the amplitude of the analog signal is sampled regularly at uniform intervals, and each sample is quantized to the nearest value within a range of digital steps.

Step detection

In statistics and signal processing, step detection is the process of finding abrupt changes in the mean level of a time series or signal. It is usually considered as a special case of the statistical method known as change detection or change point detection. Often, the step is small and the time series is corrupted by some kind of noise, and this makes the problem challenging because the step may be hidden by the noise. Therefore, statistical and/or signal processing algorithms are often required.

In information theory and signal processing, the Discrete Universal Denoiser (DUDE) is a denoising scheme for recovering sequences over a finite alphabet, which have been corrupted by a discrete memoryless channel. The DUDE was proposed in 2005 by Tsachy Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdú and Marcelo J. Weinberger.

References