Information theoretic approach
Rely on the Data Processing Theorem I(X;Y) ¨ I(X;Z): Systems cannot increase a signal’s information content
Difficult to have an empirical theory using mutual information (would need to measure the joint distribution of input and output)
Reframe DPT in terms of Kullback-Leibler distances: D(X(1)|X(0)) ¨ D(Y(1)|Y(0)). Define gX,Y to be the information transfer ratio