The first structures in the Universe formed at z>7, at higher redshift than all currently known galaxies. Since GRBs are brighter than other cosmological sources at high redshift and exhibit simple power-law afterglow spectra that is ideal for absorption studies, they serve as powerful tools for studying the early universe. New facilities planned for the coming decade will be able to obtain a large sample of high-redshift GRBs. Such a sample would constrain the nature of the first stars, galaxies, and the reionization history of the Universe.