WUSTL ESE 523 - ESE523Lec102013 (30 pages)

Previewing pages 1, 2, 14, 15, 29, 30 of 30 page document View the full content.
View Full Document

ESE523Lec102013



Previewing pages 1, 2, 14, 15, 29, 30 of actual document.

View the full content.
View Full Document
View Full Document

ESE523Lec102013

69 views


Pages:
30
School:
Washington University in St. Louis
Course:
Ese 523 - Information Theory
Information Theory Documents

Unformatted text preview:

ESE 523 Information Theory Lecture 12 Joseph A O Sullivan Samuel C Sachs Professor Electrical and Systems Engineering Washington University 2120E Green Hall 211 Urbauer Hall 314 935 4173 jao wustl edu 10 08 13 J A O Sullivan ESE 523 Lecture 12 1 Information Theory and the News http www insidescience org blog 2013 09 13 information theorycounts size animal languages Information Theory Counts Size Of Animal Languages The frequency and repetition of symbols yield a measure of the information content in human languages The symbols in English are the 26 letters of the alphabet plus a space character For animal modes of communication however figuring out the symbols can be a bit trickier and researchers don t have the benefit of huge animal language libraries that they can mine he uses the term N gram As an independent investigator affiliated with the Citizen Scientists League Smith has previously used statistical methods to probe complex linguistic systems like Meroitic an extinct and undeciphered East African language The dolphin s vocabulary has approximately 36 words while the figure for whales is about 23 the starling song repertoire is estimated at 119 to 202 songs 2 10 08 13 J A O Sullivan ESE 523 Lecture 12 Outline Channel Coding Theorem Concept of perfect communication Information capacity Examples properties Definition of achievability Jointly typical pairs of sequences Set up channel coding theorem Channel coding theorem Proof of converse Fano s Inequality 3 10 08 13 J A O Sullivan ESE 523 Lecture 12 Concept of Perfect Communication Perfect communication is possible achievable over any channel input X output Y I X Y 0 W Encoder Xn information transfer Channel p y x Yn 0 code C of length n such that max Pr error c 10 08 13 c C J A O Sullivan ESE 523 Lecture 12 Decoder W 4 Channels A discrete memoryless channel DMC is a triple X p y x Y The nth extension of a discrete memoryless channel is the channel Xn p yn xn Y n where p yk x k y k 1 p yk xk k 1 2 n n n n



View Full Document

Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view ESE523Lec102013 and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view ESE523Lec102013 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?