Claude Shannon

mathematician · 8 mentions across 7 readings

In this course

Claude Shannon founded information theory by formalizing entropy as a quantitative measure of uncertainty and information content in communication systems, establishing the mathematical foundations that underpin modern signal processing and data transmission. His 1949 work provides the theoretical backbone for understanding how information flows through systems—a concept essential to both cybernetic feedback loops and contemporary machine learning architectures, where probability distributions and information capacity determine what can be learned or transmitted. Shannon's framework appears across these course readings as the conceptual scaffolding enabling everything from document retrieval algorithms to the information-theoretic analysis of generative models and adversarial networks.

Mentioned in 7 readings

Claudia Jacques

Human-Computer Interaction Design and the Cybersemiotic Experience

2

Looking at information from a systems perspective, Claude Shannon (1949) added entropy to quantify information in any form of communication. “The concept of information applies not to the individual messages (as the concept of meaning would…Shanken, E. (2007). Telematic embrace: Visionary theories of art, technology, and consciousness by Roy Ascott. Berkeley: University of California Press. Shannon, C. (1949). The mathematical theory of communication. In C. Shannon & W. Weave…

I.J. Good

Speculations Concerning the First Ultraintelligent Machine

1

Next suppose that index terms $W1, W2, \dots, Wm$ have been specified. Then we should like to be able to compute the probabilities $P(Dj \mid W1 \cdot W2 \cdot \dots \cdot Wm)$, where the periods denote logical conjunction. One could imagin…

Ian J. Goodfellow, et al.

Generative Adversarial Nets

1

$$C(G) = \maxD V(G, D)$$ $$= \mathbb{E}{\boldsymbol{x} \sim p{\text{data}}} [\log DG^(\boldsymbol{x})] + \mathbb{E}{\boldsymbol{z} \sim p{\boldsymbol{z}}} [\log(1 - DG^(G(\boldsymbol{z})))] \quad (4)$$ $$= \mathbb{E}{\boldsymbol{x} \sim p{\…

O.G. Selfridge

Pandemonium: A Paradigm For Learning

1

take a number of observations, as he suggests, and let the machine learn the code by determining the coefficients A₀ etc. by simple curvilinear regression. The duration of previous and subsequent signals and the interpretation given to prev…

Robert Mercer et al.

A Maximum Likelihood Approach to Continuous Speech Recognition

1

Perplexity is defined in terms of the information theoretic concept of entropy. The tasks used in speech recognition can be adequately modeled as unifilar[^3-p10] Markov sources. Let $P(w|s)$ be the probability that word $w$ will be produce…

Timnit Gebru and Emile P. Torres

The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence

1

Dylan Matthews, 2022. “How effective altruism went from a niche movement to a billion-dollar force,” Vox (8 August), at https://www.vox.com/future-perfect/2022/8/8/23150496/effective-altruism-sam-bankman-fried-dustin-moskovitz-billionaire-p…

Tiqqun

The Cybernetic Hypothesis (2001)

1

Cybernetics thus emerged as a simple, inoffensive theory of information, a theory for handling information with no precise origin, always potentially present in the environment around any situation. It claims that the control of a system is…

Appears alongside

People mentioned in the same passages — sorted by co-occurrence weight.