Information and Coding Theory

As this Preface is being written, the twentieth century is coming to an end. Many of the first mathematical advances in this area were made in the mid-twentieth century by engineers, often relying on intuition and experience rather than a deep theoretical knowledge to lead them to their discoveries. This book provides an elementary introduction to Information Theory and Coding Theory - two related aspects of the problem of how to transmit information efficiently and accurately. The first part of the book focuses on Information Theory, covering uniquely decodable and instantaneous codes, Huffman coding, entropy, information channels, and Shannon's Fundamental Theorem. In the second part, on Coding Theory, linear algebra is used to construct examples of such codes, such as the Hamming, Hadamard, Golay and Reed-Muller codes.The book emphasises carefully explained proofs and worked examples; exercises (with solutions) are integrated into the text as part of the learning process. Only some basic probability theory and linear algebra, together with a little calculus (as covered in most first-year university syllabuses), is assumed, making it suitable for second- and third-year undergraduates in mathematics, electronics and computer science.