In this paper we address the problem of collaborative active state estimation within the framework of multi-robot simultaneous localization and mapping (SLAM). We assume each robot has to autonomously navigate to a pre-specified set of goals in unknown environments and develop an approach that enables the robots to collaborate in order to reduce the uncertainty in their state estimation. We formulate this problem as multi-robot belief space planning, where the belief represents the probability distribution of robot states from the entire group, as well as the mapped environment thus far. Our approach is capable of guiding each robot to reduce its uncertainty by re-observing areas previously observed (only) by other robots. Direct observations between robot states, such as relative-pose measurements, are not required, providing enhanced flexibility for the group as the robots do not have to coordinate rendezvous with each other. Instead, our framework supports indirect constraints between the robots, that are induced by mutual observations of the same area possibly at different time instances, and accounts for these future multi-robot constraints within the planning phase. The proposed approach is evaluated in a simulation study.