Vai al contenuto principale

Department of Mathematics "Giuseppe Peano"

# Laurea Magistrale (M.Sc.) in Stochastics and Data Science

Oggetto:
Oggetto:

Oggetto:

## Information theory

Oggetto:

Course ID
MAT0052
Teachers
Matteo Sereno
Valerio Bioglio
Year
2nd year
Teaching period
Second semester
Type
D.M. 270 TAF C - Related or integrative
Credits/Recognition
6
Course disciplinary sector (SSD)
INF/01 - informatics
Delivery
Class Lectures
Language
English
Attendance
Optional
Type of examination
Written and oral
Prerequisites
An undergrauate level class in Probability. Good abilities in elementary probabilistic problem solving are also necessary for the success in this class.
Oggetto:

Oggetto:

## Course objectives

The course represents an introduction to classical results of Shannon information theory.

The lectures will be in presence with exceptions in accordance with university regulations.

Oggetto:

## Results of learning outcomes

Knowledge and understanding: upon completion of this course, students should be able to:

• Calculate the information content of a random variable from its probability distribution.
• Relate the joint, conditional, and marginal entropies of variables in terms of their coupled probabilities.
• Define channel capacities and properties using Shannon's Theorems.
• Construct efficient codes for data on imperfect communication channels.
• Introduce the mathematical ideas underlying the theory of error-detection and error-correction using linear codes.

Applying knowledge and understanding: the student will be able to appreciate the mathematical features underlying the digital communication.

Communication skills: the students will be able to explain to a non expert the acquired concepts by using a formallly correct and rigourous exposition, and to discuss with experts about topics coherent with the course contents.

Oggetto:

## Program

The course is structured in two parts.

1. The first part of the course is devoted to the classical information theory. In particular, the addressed topics are: definition of information and source types, the concept of entropy, source coding, Shannon's first theorem (source coding), uniquely decodable codes, optimality of Huffman coding, models of noisy channels, definition of the channel capacity according to Shannon's theorem (channel coding).
2. The second part of the course is devoted to the study of source coding and channel coding algorithms used in many applications, communication systems and networks. As far as channel coding is regarded the course will introduce linear block codes.
Oggetto:

## Course delivery

The course is composed of 48 hours of class lectures.

The lectures will be in presence with exceptions in accordance with university regulations.

Oggetto:

## Learning assessment methods

Closed-booked written exam (two hours) - (70%). The written exam will consist of a few questions and/or problems. The mark assigned to each question/problem will depend on the level of difficulty.

Oral test - (30%).

To pass the module students must achieve a pass mark of 60% when all elements are combined.

Oggetto:
• Thomas M. Cover, Joy A. Thomas, "Elements of Information Theory, 2nd Edition", ISBN: 978-0-471-24195-9.
• R. W. Yeung, "Information Theory and Network Coding,  ISBN: 978-0-387-79233-0

Oggetto:

## Notes

This course will be delivered at the Computer Science Department

Note: the course web page (on moodle) is at Information theory

Oggetto:

## Class schedule

DaysTimeClassroom
Monday9:00 - 11:00Aula E Dipartimento di Informatica
Wednesday9:00 - 11:00Aula F Dipartimento di Informatica

Lessons: from 20/02/2017 to 26/05/2017

Notes: Class schedule is available at Computer Science Department courses

Enroll
• Closed
Enrollment opening date
01/09/2021 at 00:00
Enrollment closing date
30/06/2022 at 00:00
Oggetto:
Last update: 05/06/2024 12:58
Location: https://www.master-sds.unito.it/robots.html