Psychiatric research has benefited from attention to measurement theories of reliability, and reliability/agreement statistics for psychopathology ratings and diagnoses are regularly reported in empirical reports. Nevertheless, there are still controversies regarding how reliability should be measured, and the amount of resources that should be spent on studying measurement quality in research programs. These issues are discussed in the context of recent theoretical and technical contributions to the statistical analysis of reliability. Special attention is paid to statistical studies published since Kraemer's 1992 review of reliability methods in this journal.