Basic Concepts in Information Theory and Coding

Basic Concepts in Information Theory and Coding is an outgrowth of a one- semester introductory course that has been taught at the University of Southern California since the mid-1960s. Lecture notes from that course have evolved in response to student reaction, new technological and theoretical develop- ments, and the insights of faculty members who have taught the course (in- cluding the three of us). In presenting this material, we have made it accessible to a broad audience by limiting prerequisites to basic calculus and the ele- mentary concepts of discrete probability theory. To keep the material suitable for a one-semester course, we have limited its scope to discrete information theory and a general discussion of coding theory without detailed treatment of algorithms for encoding and decoding for various specific code classes. Readers will find that this book offers an unusually thorough treatment of noiseless self-synchronizing codes, as well as the advantage of problem sections that have been honed by reactions and interactions of several gen- erations of bright students, while Agent 00111 provides a context for the discussion of abstract concepts.