Itschak (Tsachy) Weissman | |
---|---|

Alma mater | Technion – Israel Institute of Technology |

Known for | Information Theory and Communications, Statistical Signal Processing, Weissman score |

Scientific career | |

Fields | Information Theory, Digital Communications, Statistical Signal Processing, Applications |

Institutions | Stanford University |

**Tsachy (Itschak) Weissman** is a Professor of Electrical Engineering at Stanford University.^{ [1] } He is the founding director of the Stanford Compression Forum.^{ [1] } His research interests include information theory, statistical signal processing, their applications, with recent emphasis on biological applications, in genomics in particular, lossless compression, lossy compression, delay-constrained and complexity-constrained compression and communication, network information theory, feedback communications, directed information, the interplay between estimation theory and information theory, entropy, noise reduction (denoising), filtering, prediction, sequential decision making, learning, and connections with probability, statistics, and computer science (as listed in Weissman's CV PDF link).^{ [1] }

- Education
- Career
- Patents
- Universal lossy compression methods
- Discrete universal denoising with error correction coding
- Context-based denoiser that simultaneously updates probabilities for multiple contexts
- Denoising video
- Methods for compression using a denoiser
- Method and system for denoising signals
- Method and system for optimizing denoising parameters using compressibility
- Discrete universal denoising with reliability information
- Method and system for producing variable length context models
- Discrete denoising using blended counts
- Enhanced denoising system utilizing incremental parsing
- Method and system for optimizing denoising parameters using compressibility 2
- Discrete universal denoising with reliability information 2
- Enhanced denoising system
- Method for correcting noise errors in a digital signal
- Books
- External links
- References

He was the Senior Technical Advisor to the HBO show * Silicon Valley *, and namesake of the Weissman score therein.^{ [2] } Weissman is the co-inventor of the Discrete Universal Denoiser (DUDE) algorithm.^{ [3] }

On his personal website, Weissman has spoken out against intimidation and sexual harassment in the information theory community.^{ [4] }

Weissman received his Bachelor of Science in Electrical Engineering (Summa Cum Laude) in 1997, and his PhD (2001) from Technion – Israel Institute of Technology.^{ [5] }

In 2002, Weissman joined Hewlett-Packard (HP) Laboratories as a researcher; in 2003, he became a Visiting Scientist at HP.^{ [6] } At HP, he was co-inventor of a denoising algorithm named the Discrete Universal Denoiser (DUDE).

Weissman became Assistant Professor of Electrical Engineering at Stanford University in 2003.^{ [7] }^{ [8] }^{ [9] } He was promoted to Associate Professor in 2010,^{ [10] } and professor in 2015.^{ [11] } He was named Fellow of the Institute of Electrical and Electronics Engineers IEEE in 2013 ^{ [12] }*for contributions to information theory and its applications in signal processing*.

Tsachy Weissman has been granted 15 U.S. patents.^{ [13] }

Patent number: 8320687

Abstract: The present invention provides methods for universal lossy compression that provide performance at or near the rate-distortion limit and that are based on universal, implementable lossy source coding algorithms.

Type: Grant

Filed: February 5, 2010

Date of Patent: November 27, 2012

Assignee: The Board of Trustees of the Leland Stanford Junior University

Inventors: Itschak Weissman, Shirin Jalali

Publication number: 20050289433

Abstract: A method of and system for denoising and decoding a noisy error correction coded signal received through a noise-introducing channel to produce a recovered signal. In one embodiment, noisy message blocks are separated from noisy check blocks in the noisy error correction coded signal. The noisy message blocks are denoised. Error correction decoding is performed on the denoised message blocks using the noisy check blocks to produce the recovered signal.

Type: Application

Filed: June 25, 2004

Publication date: December 29, 2005

Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger

Publication number: 20060070256

Abstract: A discrete, universal denoising method is applied to a noisy signal for which the source alphabet is typically large. The method exploits a priori information regarding expected characteristics of the signal. In particular, using characteristics of a continuous tone image such as continuity and small-scale symmetry allows definition of context classes containing large numbers of image contexts having similar statistical characteristics. Use of the context classes allows extraction of more reliable indications of the characteristic of a clean signal.

Type: Application

Filed: July 12, 2005

Publication date: April 6, 2006

Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Marcelo Weinberger, Sergio Verdu, Giovanni Motta

Patent number: 7420487

Abstract: A denoising process statistically processes a series of frames of a motion picture to construct respective data structures for the frames. Each data structure indicates for each of multiple contexts, occurrences of symbols that have the same context and are in the corresponding one of the frames. The data structures for multiple frames are combined to construct an enhanced data structure for one of the frames, and symbols in that frame are replaced with values determined using the enhanced data structure. Type: Grant

Filed: October 12, 2006

Date of Patent: September 2, 2008

Assignee: Hewlett-Packard Development Company, L.P.

Inventors: Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich, Gadiel Seroussi

Publication number: 20060045360

Abstract: Various embodiments of the present invention provide a compression method and system that compresses received data by first denoising the data and then losslessly compressing the denoised data. Denoising removes high entropy features of the data to produce lower entropy, denoised data that can be efficiently compressed by a lossless compression technique. One embodiment of the invention is a universal lossy compression method obtained by cascading a denoising technique with a universal lossless compression method. Alternative embodiments include methods obtained by cascading a denoising technique with one or more lossy or lossless compression methods.

Type: Application

Filed: September 2, 2004

Publication date: March 2, 2006

Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger

Publication number: 20110274225

Abstract: The application is directed to generally applicable denoising methods and systems for recovering, from a noise-corrupted signal, a cleaned signal equal to, or close to, the original, clean signal that suffered corruption due to one or more noise-inducing processes, devices, or media In a first pass, noise-corrupted-signal-reconstruction systems and methods receive an instance of one of many different types of neighborhood rules and use the received neighborhood rule to acquire statistics from a noisy signal. In a second pass, the noise-corrupted-signal-reconstruction systems and methods receive an instance of one of many different types of denoising rules, and use the received denoising rule to denoise a received, noisy signal in order to produce a cleaned signal.

Type: Application

Filed: July 18, 2011

Publication date: November 10, 2011

Inventor: Itschak Weissman

Publication number: 20060047484

Abstract: In various embodiments of the present invention, a noisy signal denoiser is tuned and optimized by selecting denoiser parameters that provide relatively highly compressible denoiser output. When the original signal can be compared to the output of a denoiser, the denoiser can be accurately tuned and adjusted in order to produce a denoised signal that resembles as closely as possible the clear signal originally transmitted through a noise-introducing channel. However, when the clear signal is not available, as in many communications applications, other methods are needed. By adjusting the parameters to provide a denoised signal that is globally or locally maximally compressible, the denoiser can be optimized despite inaccessibility of the original, clear signal.

Type: Application

Filed: September 2, 2004

Publication date: March 2, 2006

Inventors: Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich

Publication number: 20050289406

Abstract: A method of and system for generating reliability information for a noisy signal received through a noise-introducing channel. In one embodiment, symbol-transition probabilities are determined for the noise-introducing channel. Occurrences of metasymbols in the noisy signal are counted, each metasymbol providing a context for a symbol of the metasymbol. For each metasymbol occurring in the noisy signal, reliability information for each possible value of the symbol of the metasymbol is determined, the reliability information representing a probability that the value in the original signal corresponding to the symbol of the metasymbol assumed each of the possible values. In another embodiment, error correction coding may be performed by adding redundant data to an original signal prior to transmission by the noise-introducing channel and performing error correction decoding after transmission.

Type: Application

Filed: June 25, 2004

Publication date: December 29, 2005

Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Krishnamurthy Viswanathan

Publication number: 20060047501

Abstract: Various embodiments of the present invention provide methods and systems for determining, representing, and using variable-length contexts in a variety of different computational applications. In one embodiment of the present invention, a balanced tree is used to represent all possible contexts of a fixed length, where the depth of the balanced tree is equal to the fixed length of the considered contexts. Then, in the embodiment, a pruning technique is used to sequentially coalesce the children of particular nodes in the tree in order to produce an unbalanced tree representing a set of variable-length contexts. The pruning method is selected, in one embodiment, to coalesce nodes, and, by doing so, to truncate the tree according to statistical considerations in order to produce a representation of a variably sized context model suitable for a particular application.

Type: Application

Filed: September 2, 2004

Publication date: March 2, 2006

Inventors: Gadiell Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich

Publication number: 20060045218

Abstract: Various embodiments of the present invention relate to a discrete denoiser that replaces all of one type of symbol in a received, noisy signal with a replacement symbol in order to produce a recovered signal less distorted with respect to an originally transmitted, clean signal than the received, noisy signal. Certain, initially developed discrete denoisers employ an analysis of the number of occurrences of metasymbols within the received, noisy signal in order to select symbols for replacement, and to select the replacement symbols for the symbols that are replaced. Embodiments of the present invention use blended counts that are combinations of the occurrences of metasymbol families within a noisy signal, rather than counts of individual, single metasymbols, to determine the symbols to be replaced and the replacement symbols corresponding to them.

Type: Application

Filed: September 2, 2004

Publication date: March 2, 2006

Inventors: Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman

Publication number: 20060115017

Abstract: An apparatus for operating on a received signal that includes a noise-free signal that has been corrupted by a channel is disclosed. A memory stores a channel corruption function specifying the probability that a symbol having a value I was converted to a symbol having a value J by the channel, and a degradation function measuring the signal degradation that occurs if a symbol having the value I is replaced by symbol having a value J. The controller parses one of the received signal or the processed signal into phrases, and replaces one of the symbol having a value I in a context of that symbol in the received signal with a symbol having a value J if the replacement would reduce the estimated overall signal degradation in the processed signal. The context of a symbol depends on the phrase associated with the symbol.

Type: Application

Filed: November 29, 2004

Publication date: June 1, 2006

Inventors: Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman

Patent number: 7436969

Abstract: In various embodiments of the present invention, a noisy signal denoiser is tuned and optimized by selecting denoiser parameters that provide relatively highly compressible denoiser output. When the original signal can be compared to the output of a denoiser, the denoiser can be accurately tuned and adjusted in order to produce a denoised signal that resembles as closely as possible the clear signal originally transmitted through a noise-introducing channel. However, when the clear signal is not available, as in many communications applications, other methods are needed. By adjusting the parameters to provide a denoised signal that is globally or locally maximally compressible, the denoiser can be optimized despite inaccessibility of the original, clear signal.

Type: Grant

Filed: September 2, 2004

Date of Patent: October 14, 2008

Assignee: Hewlett-Packard Development Company, L.P.

Inventors: Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Itschak Weissman, Erik Ordentlich

Patent number: 7269781

Abstract: A method of and system for generating reliability information for a noisy signal received through a noise-introducing channel. In one embodiment, symbol-transition probabilities are determined for the noise-introducing channel. Occurrences of metasymbols in the noisy signal are counted, each metasymbol providing a context for a symbol of the metasymbol. For each metasymbol occurring in the noisy signal, reliability information for each possible value of the symbol of the metasymbol is determined, the reliability information representing a probability that the value in the original signal corresponding to the symbol of the metasymbol assumed each of the possible values. In another embodiment, error correction coding may be performed by adding redundant data to an original signal prior to transmission by the noise-introducing channel and performing error correction decoding after transmission.

Type: Grant

Filed: June 25, 2004

Date of Patent: September 11, 2007

Assignee: Hewlett-Packard Development Company, L.P.

Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdu, Marcelo Weinberger, Krishnamurthy Viswanathan

Publication number: 20050163267

Abstract: A method and apparatus for processing a received digital signal that has been corrupted by a channel is disclosed. The method includes storing the received digital signal and receiving a partially corrected sequence of symbols that includes an output of a preliminary denoising system operating on the received digital signal. Information specifying a signal degradation function that measures the signal degradation that occurs if a symbol having the value I is replaced by a symbol having the value J is utilized to generate a processed digital signal by replacing each symbol having a value I in a context of that symbol in the received digital signal with a symbol having a value J if replacement reduces a measure of overall signal degradation in the processed digital signal relative to the received digital signal as measured by the degradation function and the partially corrected sequence of symbols.

Type: Application

Filed: January 26, 2004

Publication date: July 28, 2005

Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Marcelo Weinberger, Sergio Verdu

Publication number: 20050097421

Abstract: An apparatus and method for processing a received signal that has been corrupted by a channel to generate a processed signal having less signal corruption than the received signal is disclosed. The apparatus stores the received signal, information specifying the probability that a symbol having a value I will be converted to a symbol having a value J by the channel, and information specifying a signal degradation function that measures the signal degradation that occurs if a symbol having the value I is replaced by symbol having a value J. The controller replaces each symbol having a value I in a context of that symbol in the received signal with a symbol having a value J that minimizes the overall signal degradation in the processed signal relative to the underlying noise-free signal as estimated via the observed statistics within that context.

Type: Application

Filed: October 17, 2003

Publication date: May 5, 2005

Inventors: Itschak Weissman, Erik Ordentlich, Gadiel Seroussi, Marcelo Weinberger, Sergio Verdu

- B. Marcus, K. Petersen and T. Weissman (eds.),
*Entropy of Hidden Markov Processes and Connections to Dynamical Systems*, Cambridge University Press, July 2011.

**Digital data**, in information theory and information systems, is information represented as a string of discrete symbols each of which can take on one of only a finite number of values from some alphabet, such as letters or digits. An example is a text document, which consists of a string of alphanumeric characters. The most common form of digital data in modern information systems is *binary data*, which is represented by a string of binary digits (bits) each of which can have one of two values, either 0 or 1.

**Information theory** is the scientific study of the quantification, storage, and communication of digital information. The field was fundamentally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. The field is at the intersection of probability theory, statistics, computer science, statistical mechanics, information engineering, and electrical engineering.

**Signal processing** is an electrical engineering subfield that focuses on analysing, modifying, and synthesizing signals such as sound, images, and scientific measurements. Signal processing techniques can be used to improve transmission, storage efficiency and subjective quality and to also emphasize or detect components of interest in a measured signal.

In information theory, the **Shannon–Hartley theorem** tells the maximum rate at which information can be transmitted over a communications channel of a specified bandwidth in the presence of noise. It is an application of the noisy-channel coding theorem to the archetypal case of a continuous-time analog communications channel subject to Gaussian noise. The theorem establishes Shannon's channel capacity for such a communication link, a bound on the maximum amount of error-free information per time unit that can be transmitted with a specified bandwidth in the presence of the noise interference, assuming that the signal power is bounded, and that the Gaussian noise process is characterized by a known power or power spectral density. The law is named after Claude Shannon and Ralph Hartley.

**Digital image processing** is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions digital image processing may be modeled in the form of multidimensional systems. The generation and development of digital image processing are mainly affected by three factors: first, the development of computers; second, the development of mathematics ; third, the demand for a wide range of applications in environment, agriculture, military, industry and medical science has increased.

**Golomb coding** is a lossless data compression method using a family of data compression codes invented by Solomon W. Golomb in the 1960s. Alphabets following a geometric distribution will have a Golomb code as an optimal prefix code, making Golomb coding highly suitable for situations in which the occurrence of small values in the input stream is significantly more likely than large values.

A **communication channel** refers either to a physical transmission medium such as a wire, or to a logical connection over a multiplexed medium such as a radio channel in telecommunications and computer networking. A channel is used to convey an information signal, for example a digital bit stream, from one or several *senders* to one or several *receivers*. A channel has a certain capacity for transmitting information, often measured by its bandwidth in Hz or its data rate in bits per second.

**Coding theory** is the study of the properties of codes and their respective fitness for specific applications. Codes are used for data compression, cryptography, error detection and correction, data transmission and data storage. Codes are studied by various scientific disciplines—such as information theory, electrical engineering, mathematics, linguistics, and computer science—for the purpose of designing efficient and reliable data transmission methods. This typically involves the removal of redundancy and the correction or detection of errors in the transmitted data.

**Noise reduction** is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree.

In numerical analysis and functional analysis, a **discrete wavelet transform** (**DWT**) is any wavelet transform for which the wavelets are discretely sampled. As with other wavelet transforms, a key advantage it has over Fourier transforms is temporal resolution: it captures both frequency *and* location information.

A **bit plane** of a digital discrete signal is a set of bits corresponding to a given bit position in each of the binary numbers representing the signal.

**Peak signal-to-noise ratio** (**PSNR**) is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. Because many signals have a very wide dynamic range, PSNR is usually expressed as a logarithmic quantity using the decibel scale.

**Image noise** is random variation of brightness or color information in images, and is usually an aspect of electronic noise. It can be produced by the image sensor and circuitry of a scanner or digital camera. Image noise can also originate in film grain and in the unavoidable shot noise of an ideal photon detector. Image noise is an undesirable by-product of image capture that obscures the desired information.

The **Shannon–Weaver model of communication** has been called the "mother of all models." Social Scientists use the term to refer to an integrated model of the concepts of information source, message, transmitter, signal, channel, noise, receiver, information destination, probability of error, encoding, decoding, information rate, channel capacity. However, some consider the name to be misleading, asserting that the most significant ideas were developed by Shannon alone.

In sound and music production, **sonic artifact**, or simply **artifact**, refers to sonic material that is accidental or unwanted, resulting from the editing or manipulation of a sound.

The decisive event which established the discipline of **information theory**, and brought it to immediate worldwide attention, was the publication of Claude E. Shannon's classic paper "A Mathematical Theory of Communication" in the *Bell System Technical Journal* in July and October 1948.

An **autoencoder** is a type of artificial neural network used to learn efficient codings of unlabeled data. The encoding is validated and refined by attempting to regenerate the input from the encoding. The autoencoder learns a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore insignificant data (“noise”).

**Pulse-code modulation** (**PCM**) is a method used to digitally represent sampled analog signals. It is the standard form of digital audio in computers, compact discs, digital telephony and other digital audio applications. In a PCM stream, the amplitude of the analog signal is sampled regularly at uniform intervals, and each sample is quantized to the nearest value within a range of digital steps.

In statistics and signal processing, **step detection** is the process of finding abrupt changes in the mean level of a time series or signal. It is usually considered as a special case of the statistical method known as change detection or change point detection. Often, the step is small and the time series is corrupted by some kind of noise, and this makes the problem challenging because the step may be hidden by the noise. Therefore, statistical and/or signal processing algorithms are often required.

In information theory and signal processing, the **Discrete Universal Denoiser** (**DUDE**) is a denoising scheme for recovering sequences over a finite alphabet, which have been corrupted by a discrete memoryless channel. The DUDE was proposed in 2005 by Tsachy Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdú and Marcelo J. Weinberger.

- 1 2 3 Stanford profile, Itschak Weissman
- ↑ IEEE Spectrum, "A Made-For-TV Compression Algorithm", 25 July 2014
- ↑ HP Labs, Discrete Universal Denoiser (DUDE)
- ↑ FAQs
- ↑ Stanford University profile, Tsachy Weissman
- ↑ HP Labs, People
- ↑ Stanford Report, President's Report to the Board of Trustees, 2003
- ↑ Stanford Report, Diverse backgrounds, interests distinguish new faculty on campus, 2004
- ↑ Stanford Report, Report of the President to the Board of Trustees, 2009
- ↑ Stanford Report, Report of the President: Academic Council professoriate appointments, 2010
- ↑ Stanford Report, Report of the President: Academic Council Professoriate appointments, 2015
- ↑ IEEE Information Theory Society Fellows
- ↑ Patents, Itschak Weissman

Text is available under the CC BY-SA 4.0 license; additional terms may apply.

Images, videos and audio are available under their respective licenses.