Claude Shannon

mathematician · 8 mentions across 7 readings

In this course

Claude Shannon founded information theory by introducing entropy as a mathematical measure of uncertainty and information content in 1949, establishing the theoretical framework that underpins all modern digital communication and data compression. His concept of information as a quantifiable property of systems—rather than as semantic meaning—became foundational to both cybernetic feedback systems and machine learning algorithms, enabling the probabilistic and statistical approaches that drive contemporary AI architectures. The course draws on Shannon's formalism to bridge deterministic communication theory with generative models and adversarial learning, showing how quantifying information mathematically opened the conceptual door to treating neural networks and data itself as information-processing systems.

Mentioned in 7 readings

Claudia Jacques

Human-Computer Interaction Design and the Cybersemiotic Experience

2

Looking at information from a systems perspective, Claude Shannon (1949) added entropy to quantify information in any form of communication. “The concept of information applies not to the individual messages (as the concept of meaning would…Shanken, E. (2007). Telematic embrace: Visionary theories of art, technology, and consciousness by Roy Ascott. Berkeley: University of California Press. Shannon, C. (1949). The mathematical theory of communication. In C. Shannon & W. Weave…

I.J. Good

Speculations Concerning the First Ultraintelligent Machine

1

Next suppose that index terms $W1, W2, \dots, Wm$ have been specified. Then we should like to be able to compute the probabilities $P(Dj \mid W1 \cdot W2 \cdot \dots \cdot Wm)$, where the periods denote logical conjunction. One could imagin…

Ian J. Goodfellow, et al.

Generative Adversarial Nets

1

$$C(G) = \maxD V(G, D)$$ $$= \mathbb{E}{\boldsymbol{x} \sim p{\text{data}}} [\log DG^(\boldsymbol{x})] + \mathbb{E}{\boldsymbol{z} \sim p{\boldsymbol{z}}} [\log(1 - DG^(G(\boldsymbol{z})))] \quad (4)$$ $$= \mathbb{E}{\boldsymbol{x} \sim p{\…

O.G. Selfridge

Pandemonium: A Paradigm For Learning

1

take a number of observations, as he suggests, and let the machine learn the code by determining the coefficients A₀ etc. by simple curvilinear regression. The duration of previous and subsequent signals and the interpretation given to prev…

Robert Mercer et al.

A Maximum Likelihood Approach to Continuous Speech Recognition

1

Perplexity is defined in terms of the information theoretic concept of entropy. The tasks used in speech recognition can be adequately modeled as unifilar[^3] Markov sources. Let $P(w|s)$ be the probability that word $w$ will be produced ne…

Timnit Gebru and Émile P. Torres

The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence

1

Dylan Matthews, 2022. “How effective altruism went from a niche movement to a billion-dollar force,” Vox (8 August), at https://www.vox.com/future-perfect/2022/8/8/23150496/effective-altruism-sam-bankman-fried-dustin-moskovitz-billionaire-p…

Tiqqun

The Cybernetic Hypothesis

1

Cybernetics thus emerged as a simple, inoffensive theory of information, a theory for handling information with no precise origin, always potentially present in the environment around any situation. It claims that the control of a system is…

Appears alongside

People mentioned in the same passages — sorted by co-occurrence weight.