13.704 INFORMATION THEORY AND CODING (T)
L-T-P : 3-1-0 Credits: 4
- To understand the concept of information
- To introduce to various aspects of error controlling and coding techniques for communication.
- To have idea on the different coding techniques.
Introduction to Information Theory. Concept of amount of information, units – entropy, marginal, conditional and joint entropies – relation among entropies – mutual information, information rate. Source coding: Instantaneous codes – construction of instantaneous codes – Kraft‘s inequality, coding efficiency and redundancy, Noiseless coding theorem – construction of basic source codes – Shannon – Fano Algorithm, Huffman coding,
Channel capacity – redundancy and efficiency of a channel, binary symmetric channel (BSC), Binary erasure channel (BEC) – capacity of band limited Gaussian channels, Shannon – Hartley theorem – bandwidth – SNR trade off – capacity of a channel of infinite bandwidth, Shannon‘s limit.
Information Capacity of Colored noise channel, Water-Filling Interpretation of Information Capacity Theorem, Rate Distortion Theory
Introduction to rings , fields, and Galois fields. Codes for error detection and correction – parity check coding – linear block codes – error detecting and correcting capabilities – generator and parity check matrices – Standard array and syndrome decoding – perfect codes, Hamming codes – encoding and decoding, cyclic codes – polynomial and matrix descriptions – generation of cyclic codes, decoding of cyclic codes, BCH codes – description and decoding, Reed – Solomon Codes, Burst error correction.
Convolutional Codes – encoding – time and frequency domain approaches, State Tree & Trellis diagrams – transfer function and minimum free distance – Maximum likelihood decoding of convolutional codes – The Viterbi Algorithm. Sequential decoding,. Cryptography : Secret key cryptography, block and stream ciphers. DES, Public key cryptography.
- Symon Haykins: Digital Communication Systems , Wiley India, 2013.
- P.S.Sathya Narayana: Concepts of Information Theory & Coding , Dynaram Publications,2005
- Ranjan Bose: Information Theory, Coding and Cryptography, 2/e,TMH, New Delhi ,2008.
- Shu Lin & Daniel J. Costello.Jr., Error Control Coding : Fundamentals and Applications, 2/e,Prentice Hall
- Kulkarni, Shivaprakasha,Information ataheory and Coding,Wiley,2015.
- David J.C Mackay, Information Theory, Inference and Learning Algorithms, Cambridge,2005.
- Paul Garrett, The mathematics of Coding Theory, Prentice Hall, 2004.
- Das Mullick Chatterjee, Principles of Digital communication , Wiley Eastern Ltd.
- Sklar, Ray: Digital Communication, Fundemental and Applications, 2/e Pearson, 2011
- Saha,Manna,Mandal, Information Theory,Pearson, 2013.
- Understand the concept of Information
- Understand the concept of various theorems proposed by Shannon
- Understand the concept of channel capacity
- Understand the idea of groups, rings, field, and codes.
- Understand the different error codes for communication systems.
University Examination Pattern:
Examination duration: 3 hours; Maximum Total Marks: 100
The question paper shall consist of 2 parts.
Part A (20 marks) - Ten Short answer questions of 2 marks each. All questions are compulsory. There should be at least two questions from each module and not more than three questions from any module.
Part B (80 Marks) - Candidates have to answer one full question out of the two from each module. Each question carries 20 marks.
Note: Question paper should contain20% Problems, derivations and proofs.