Cognitive distortions in gambling are irrational thoughts that cause an individual to overestimate their level of control over the outcome of the game and diminish the role of chance. Due to their strong relation to gambling disorders, they are a particularly important characteristic to assess and understand in gamblers. Although numerous measures of gambling-related cognitive distortions exist, studies assessing criterion validity are scarce. In this study, we develop several tests of the Gamblers Belief Questionnaire (GBQ), a versatile and widely used scale. A sample of 184 U.S. adults was recruited through Amazon Mechanical Turk to complete an online study that included measurement of the GBQ and an assessment of the perceived role of skill and chance in various gambling and non-gambling activities. In addition to a confirmatory factor analysis of the scale, three novel validation tests were developed to understand whether the GBQ subscales can identify and discriminate measures of illusion of control and gambler's fallacy distortions. Our validation tests demonstrate that the scale does measure both distortions, providing information about gamblers' cognition that is unexplained by gambling problems, frequency of play, and demographics. Conversely, our analysis of the factor structure does not show good fit. We conclude that the GBQ measures gambling-related cognitive distortions, but there may be an opportunity to reduce the number of scale items and further refine precision of the two subscales.