Siu Wai HO, University of South Australia

Location :

salle de réunion « rose »
5ème étage
INRIA
23 AVENUE D'TALIE
75013 PARIS

Abstract :

In Shannon’s original paper and textbooks in information theory, the entropy of a discrete 
random variable is assumed or shown to be a continuous function. However, we found that all 
Shannon’s information measures including entropy and mutual information are discontinuous in the 
general case that random variables are taking values in possibly countably infinite alphabets. 
This fundamental property explains why strong typicality and Fano’s inequality can only be 
applied on finite alphabets. Note that strong typicality and Fano’s inequality have wide 
applications in information theory so it is very important to generalize them. In this talk, details 
about the discontinuity of all Shannon’s information measures will be given. We will show how these 
results lead to a new definition of typicality and an inequality tighter than Fano’s inequality.  The 
applications in network coding and information theoretic security will be discussed.

Host :

Marc Lelarge and François Baccelli