Significant racial disparities in physical activity-a key protective health factor against obesity and cardiovascular disease-exist in the United States. Using data from the 1999-2004 National Health and Nutrition Examination Survey and the 2000 United States (US) Census, we estimated the impact of race, individual-level poverty, neighborhood-level poverty, and neighborhood racial composition on the odds of being physically active for 19,678 adults. Compared to whites, blacks had lower odds of being physically active. Individual poverty and neighborhood poverty were associated with decreased odds of being physically active among both whites and blacks. These findings underscore the importance of social context in understanding racial disparities in physical activity and suggest the need for future research to determine specific elements of the social context that drive disparities.