Despite decades of research, the pathophysiology of bipolar disorder (BD) is still not well understood. Structural brain differences have been associated with BD, but results from neuroimaging studies have been inconsistent. To address this, we performed the largest study to date of cortical gray matter thickness and surface area measures from brain magnetic resonance imaging scans of 6503 individuals including 1837 unrelated adults with BD and 2582 unrelated healthy controls for group differences while also examining the effects of commonly prescribed medications, age of illness onset, history of psychosis, mood state, age and sex differences on cortical regions. In BD, cortical gray matter was thinner in frontal, temporal and parietal regions of both brain hemispheres. BD had the strongest effects on left pars opercularis (Cohen's d=-0.293; P=1.71 × 10^-21), left fusiform gyrus (d=-0.288; P=8.25 × 10^-21) and left rostral middle frontal cortex (d=-0.276; P=2.99 × 10^-19). Longer duration of illness (after accounting for age at the time of scanning) was associated with reduced cortical thickness in frontal, medial parietal and occipital regions. We found that several commonly prescribed medications, including lithium, antiepileptic and antipsychotic treatment showed significant associations with cortical thickness and surface area, even after accounting for patients who received multiple medications. We found evidence of reduced cortical surface area associated with a history of psychosis but no associations with mood state at the time of scanning. Our analysis revealed previously undetected associations and provides an extensive analysis of potential confounding variables in neuroimaging studies of BD.