Evidence strongly suggests that being overweight or obese at midlife confers significantly higher risk for Alzheimer's disease and greater brain atrophy later in life. Few studies, however, examine associations between longitudinal changes in adiposity during early adulthood and later brain morphometry. Measures of body mass index (BMI) were collected in 373 men from the Vietnam Era Twin Study of Aging at average ages 20, 40, 56, and 62 years, yielding 2 BMI trajectories. We then examined associations between BMI phenotypes (trajectories, continuous BMI, obese/nonobese), cortical thickness, and white matter measures from structural magnetic resonance imaging at mean age 62 (time 4, range 56-66 years). Those on the obesity trajectory (N = 171) had a thinner cortex compared with the normal/lean trajectory (N = 202) in multiple frontal and temporal lobe bilateral regions of interest: superior, inferior, middle temporal gyri, temporal pole, fusiform gyrus, banks of the superior temporal sulcus, frontal pole, pars triangularis, caudal and rostral middle frontal gyri (all p < 0.05, false discovery rate corrected). Frontal lobe thinness tended to occur mainly in the right hemisphere. Results were similar for obese versus nonobese adults at age 62. There were no significant differences for white matter volume or abnormalities. Taken in the context of other research, these associations between brain structures and excess BMI at midlife suggest potential for increased risk for cognitive decline in later life.