Multi-agent learning refers to settings in which multiple agents learn simultaneously. Usually defined in a game theoretic setting, specifically in repeated games or stochastic games, the key feature that distinguishes multi-agent learning from single-agent learning is that in the former the learning of one agent impacts he learning of others. As a result neither the problem definition for mutli-agent learning, nor the algorithms offered, follow in a straightforward way from the single-agent case. In this second of two entries on the subject we focus on algorithms.