13.704 INFORMATION THEORY AND CODING (T)
LTP : 310 Credits: 4
Course Objectives:
 To understand the concept of information
 To introduce to various aspects of error controlling and coding techniques for communication.
 To have idea on the different coding techniques.

Module I
Introduction to Information Theory. Concept of amount of information, units – entropy, marginal, conditional and joint entropies – relation among entropies – mutual information, information rate. Source coding: Instantaneous codes – construction of instantaneous codes – Kraft‘s inequality, coding efficiency and redundancy, Noiseless coding theorem – construction of basic source codes – Shannon – Fano Algorithm, Huffman coding,
Module II
Channel capacity – redundancy and efficiency of a channel, binary symmetric channel (BSC), Binary erasure channel (BEC) – capacity of band limited Gaussian channels, Shannon – Hartley theorem – bandwidth – SNR trade off – capacity of a channel of infinite bandwidth, Shannon‘s limit.
Information Capacity of Colored noise channel, WaterFilling Interpretation of Information Capacity Theorem, Rate Distortion Theory
Module III
Introduction to rings , fields, and Galois fields. Codes for error detection and correction – parity check coding – linear block codes – error detecting and correcting capabilities – generator and parity check matrices – Standard array and syndrome decoding – perfect codes, Hamming codes – encoding and decoding, cyclic codes – polynomial and matrix descriptions – generation of cyclic codes, decoding of cyclic codes, BCH codes – description and decoding, Reed – Solomon Codes, Burst error correction.
Module IV
Convolutional Codes – encoding – time and frequency domain approaches, State Tree & Trellis diagrams – transfer function and minimum free distance – Maximum likelihood decoding of convolutional codes – The Viterbi Algorithm. Sequential decoding,. Cryptography : Secret key cryptography, block and stream ciphers. DES, Public key cryptography.
References:
 Symon Haykins: Digital Communication Systems , Wiley India, 2013.
 P.S.Sathya Narayana: Concepts of Information Theory & Coding , Dynaram Publications,2005
 Ranjan Bose: Information Theory, Coding and Cryptography, 2/e,TMH, New Delhi ,2008.
 Shu Lin & Daniel J. Costello.Jr., Error Control Coding : Fundamentals and Applications, 2/e,Prentice Hall
 Kulkarni, Shivaprakasha,Information ataheory and Coding,Wiley,2015.
 David J.C Mackay, Information Theory, Inference and Learning Algorithms, Cambridge,2005.
 Paul Garrett, The mathematics of Coding Theory, Prentice Hall, 2004.
 Das Mullick Chatterjee, Principles of Digital communication , Wiley Eastern Ltd.
 Sklar, Ray: Digital Communication, Fundemental and Applications, 2/e Pearson, 2011
 Saha,Manna,Mandal, Information Theory,Pearson, 2013.
Course Outcomes:
 Understand the concept of Information
 Understand the concept of various theorems proposed by Shannon
 Understand the concept of channel capacity
 Understand the idea of groups, rings, field, and codes.
 Understand the different error codes for communication systems.
University Examination Pattern:
Examination duration: 3 hours; Maximum Total Marks: 100
The question paper shall consist of 2 parts.
Part A (20 marks)  Ten Short answer questions of 2 marks each. All questions are compulsory. There should be at least two questions from each module and not more than three questions from any module.
Part B (80 Marks)  Candidates have to answer one full question out of the two from each module. Each question carries 20 marks.
Note: Question paper should contain20% Problems, derivations and proofs.