Four estimators of the directed information rate between a pair of jointly stationary ergodic
finite-alphabet processes are proposed, based on universal probability assignments. The
first one is a Shannon–McMillan–Breiman-type estimator, similar to those used by Verdú in
2005 and Cai in 2006 for estimation of other information measures. We show the almost
sure and L_1 convergence properties of the estimator for any underlying universal
probability assignment. The other three estimators map universal probability assignments to …

We propose four approaches to estimating the directed information rate between a pair of
jointly stationary ergodic processes with the help of universal probability assignments. The
four approaches yield estimators with different merits such as nonnegativity and
boundedness. We establish consistency of these estimators in various senses and derive
near-optimal rates of convergence in the minimax sense under mild conditions. The
estimators carry over directly to estimating other information measures of stationary ergodic …