List: Information theory

by likeorhate

•  Kolmogorov complexity

  In algorithmic information theory (a subfield of computer science), the Kolmogorov complexity (also known as descriptive complexity, Kolmogorov-Chaitin complexity, stochastic complexity, algorithmic entropy, or program-size complexity) of an object such as a piece of text is a measure of the computational resources needed to specify the object.

  

  100.0%
  

  0.0%
  

  0.0%
  
•  Bra-ket notation

  Bra-ket notation is a standard notation for describing quantum states in the theory of quantum mechanics composed of angle brackets and vertical bars. It can also be used to denote abstract vectors and linear functionals in pure mathematics. It is so called because the inner product of two states is denoted by a bracket,, consisting of a left part,, called the bra, and a right part,, called the ket. The notation was introduced in 1939 by Paul Dirac, and is also known as Dirac notation.

  

  0.0%
  

  20.0%
  

  80.0%
  
•  Information theory

  Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Historically, information theory was developed by Claude E. Shannon to find fundamental limits on compressing and reliably storing and communicating data.

  

  14.3%
  

  42.9%
  

  42.9%
  
•  Entropy (information theory)

  In information theory, entropy is a measure of the uncertainty associated with a random variable. The term by itself in this context usually refers to the Shannon entropy, which quantifies, in the sense of an expected value, the information contained in a message, usually in units such as bits. Equivalently, the Shannon entropy is a measure of the average information content one is missing when one does not know the value of the random variable. The concept was introduced by Claude E.

  

  0.0%
  

  50.0%
  

  50.0%
  
•  Quantum computer

  A quantum computer is a device for computation that makes direct use of quantum mechanical phenomena, such as superposition and entanglement, to perform operations on data. The basic principle behind quantum computation is that quantum properties can be used to represent data and perform operations on these data. A theoretical model is the quantum Turing machine, also known as the universal quantum computer.

  

  33.3%
  

  44.4%
  

  22.2%
  
•  Quantum information

  In quantum mechanics, quantum information is physical information that is held in the "state" of a quantum system. The most popular unit of quantum information is the qubit, a two-level quantum system. However, unlike classical digital states (which are discrete), a two-state quantum system can actually be in a superposition of the two states at any given time.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Run-length encoding

  Run-length encoding (RLE) is a very simple form of data compression in which runs of data (that is, sequences in which the same data value occurs in many consecutive data elements) are stored as a single data value and count, rather than as the original run. This is most useful on data that contains many such runs: for example, relatively simple graphic images such as icons, line drawings, and animations.

  

  25.0%
  

  50.0%
  

  25.0%
  
•  Nyquist–Shannon sampling theorem

  The Nyquist–Shannon sampling theorem is a fundamental result in the field of information theory, in particular telecommunications and signal processing. Sampling is the process of converting a signal (for example, a function of continuous time or space) into a numeric sequence (a function of discrete time or space).

  

  0.0%
  

  66.7%
  

  33.3%
  
•  Metcalfe's law

  Metcalfe's law states that the value of a telecommunications network is proportional to the square of the number of connected users of the system (n). First formulated in this form by George Gilder in 1993, and attributed to Robert Metcalfe in regard to Ethernet, Metcalfe's law was originally presented, circa 1980, not in terms of users, but rather of "compatibly communicating devices" (for example, fax machines).

  

  0.0%
  

  33.3%
  

  66.7%
  
•  Shannon–Hartley theorem

  In information theory, the Shannon–Hartley theorem is an application of the noisy channel coding theorem to the archetypal case of a continuous-time analog communications channel subject to Gaussian noise.

  

  100.0%
  

  0.0%
  

  0.0%
  
•  Unicity distance

  Unicity distance is a term used in cryptography referring to the length of an original ciphertext needed to break the cipher by reducing the number of possible spurious keys to zero in a brute force attack. That is, after trying every possible key, there should be just one decipherment that makes sense. Consider an attack on the ciphertext string "WNAIW" encrypted using a Vigenère cipher with a five letter key.

  

  100.0%
  

  0.0%
  

  0.0%
  
•  Harry Nyquist

  Harry Nyquist (né Harry Theodor Nyqvist; pron. , not as often pronounced), (February 7, 1889 – April 4, 1976) was an important contributor to information theory.

  

  44.4%
  

  22.2%
  

  33.3%
  
•  Covert channel

  In Computer Security, a covert channel is a type of computer security attack that creates a capability to transfer information objects between processes that are not supposed to be allowed to communicate by the computer security policy. The term, originated in 1972 by Lampson is defined as "(channels) not intended for information transfer at all, such as the service program's effect on system load. " to distinguish it from Legitimate channels that are subjected to access controls by COMPUSEC.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Rate–distortion theory

  Rate–distortion theory is a major branch of information theory which provides the theoretical foundations for lossy data compression; it addresses the problem of determining the minimal amount of entropy R that should be communicated over a channel, so that the source (input signal) can be approximately reconstructed at the receiver (output signal) without exceeding a given distortion D.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Channel capacity

  In electrical engineering, computer science and information theory, channel capacity is the tightest upper bound on the amount of information that can be reliably transmitted over a communications channel. By the noisy-channel coding theorem, the channel capacity of a given channel is the limiting information rate (in units of information per unit time) that can be achieved with arbitrarily small error probability. Information theory, developed by Claude E.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Physical information

  In physics, physical information refers generally to the information that is contained in a physical system. Its usage in quantum mechanics is important, for example in the concept of quantum entanglement to describe effectively direct or causal relationships between apparently distinct or spatially separated particles. Information itself may be loosely defined as "that which can distinguish one thing from another".

  

  100.0%
  

  0.0%
  

  0.0%
  
•  Asymptotic equipartition property

  In information theory the asymptotic equipartition property (AEP) is a general property of the output samples of a stochastic source. It is fundamental to the concept of typical set used in theories of compression.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Typical set

  In information theory, the typical set is a set of sequences whose probability is close to two raised to the negative power of the entropy of their source distribution. That this set has total probability close to one is a consequence of the asymptotic equipartition property (AEP) which is a kind of law of large numbers.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Structural information theory

  Structural information theory (SIT) is a theory about human perception and, in particular, about perceptual organization, that is, about the way the human visual system organizes a raw visual stimulus into objects and object parts. SIT was initiated, in the 1960s, by Emanuel Leeuwenberg and has been developed further by Hans Buffart, Peter van der Helm, and Rob van Lier.

  

  0.0%
  

  0.0%
  

  100.0%
  
•  Relationship between latency and throughput

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Mutual information

  In probability theory and information theory, the mutual information (sometimes known by the archaic term transinformation) of two random variables is a quantity that measures the mutual dependence of the two variables. The most common unit of measurement of mutual information is the bit, when logarithms to the base 2 are used.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Information geometry

  In mathematics and especially in statistical inference, information geometry is the study of probability and information by way of differential geometry. It reached maturity through the work of Shun'ichi Amari in the 1980s, with what is currently the canonical reference book: Methods of information geometry.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Self-information

  In information theory (elaborated by Claude E. Shannon, 1948), self-information is a measure of the information content associated with the outcome of a random variable. It is expressed in a unit of information, for example bits, nats, or hartleys, depending on the base of the logarithm used in its calculation. The term self-information is also sometimes used as a synonym of entropy, i.e.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  Computational irreducibility

  Computational irreducibility is one of the main ideas proposed by Stephen Wolfram in his book A New Kind of Science.

  

  0.0%
  

  0.0%
  

  0.0%
  
•  List of information theory topics

  

  0.0%
  

  0.0%
  

  0.0%