Behind the Scenes: Guarding Against Bias

By: Lawrence M. Rudner

When I first took the GMAT exam in 1989, almost all test takers were American or English-speaking Canadian citizens seeking to attend school in North America. Back then, GMAC did not need to be concerned about native English speakers having an unfair advantage. Today, however, slightly more than half of GMAT test takers are non-US citizens, and we recognize the importance of providing a test that is fair, valid and reliable for all test takers, regardless of their native language.

A fair and reliable test means that all individuals of the same ability should get the same score. It does not suggest that the averages by native language should be approximately equal. Sorry, Argentina, your extremely high mean GMAT Total score does not mean that you are smarter than the rest of us or that the GMAT exam is biased in your favor. It may just mean that in Argentina only those very qualified for management study actually take the test.

Yes, the GMAT test is administered in English and is designed for programs that teach in English. But the exam requires just enough English to allow us to adequately and comprehensively assess Verbal reasoning, Quantitative reasoning and Integrated Reasoning skills. Although the GMAT exam is not a grammar test, it is worth noting that grammar tests do not necessarily favor native speakers, because non-native speakers often learn grammar better than native speakers who go by the “sounds right” rule. I was once turned down for a job as an English language instructor in Italy because my spoken English contained grammatical mistakes.

GMAC has systems in place to avoid bias in favor of native speakers, which can happen several ways:

  1. Using vocabulary, idioms, and constructions that are not universal. My favorite example is skimmed milk. In India, toned milk and double toned milk are common, and skimmed milk is rare. Vacation and holiday, and quite and very, are other examples.  Also under this category are double negatives (The GMAT is not unfair), contractions (should’ve), abbreviations (for example, e.g.), imperatives, possessives, and some sentence structures (Do you have a pencil? Have you a pencil?).
  2. Using culturally loaded phrases, idioms, and constructions.  Most people who grew up in North America would probably know what is meant by a Mickey Mouse job (a job that is trivial and a waste of time), the whole nine yards (everything), or skedaddle (depart quickly). Also under this category are military terms (your mission), sports terms (game plan), colloquialisms (lots more), regional terms (pop, hoagie), metaphors (launch an idea), euphemisms (under the weather), clichés (between a rock and a hard place), literary references (wears his heart on his sleeve), US cultural references (Big Mac), acronyms (ASAP), and initials (IRS).
  3. Using culturally offensive phrases, idioms, and constructions. This includes the use of ethnic slurs, derogatory terms, positive and negative stereotypes, and references that rely on or ignore belief systems (drinking alcohol, eating pork). 

Culturally based, loaded, and offensive terms do not always result in differences in performance, but we do not want test takers to be distracted by culturally insensitive questions. We carefully review our questions using criteria defining good item construction. We also compute statistics to assess whether our questions are appropriate across culture groups. We constantly update guidelines for our item writers, including a master list of terms and phrases to avoid, in order to assure cultural fairness.

By using carefully defined and thorough item development and review processes, along with statistical analyses to flag questions with possible cultural bias, we have developed a test that minimizes the impact of culture and language. The GMAT exam is the best objective measure of the likelihood of success in management programs across the globe.

Lawrence M. Rudner, PhD, MBA, is vice president of research and development and chief psychometrician for the Graduate Management Admission Council. This blog post is condensed and re-edited from a Demystifying the GMAT column that originally ran in Graduate Management News in July 2012.

3 Responses

  1. Great Post Lawrence. However I may not fully agree with you. A simple thing for native English speakers might be extremely tough for others.
    Your statement “But the exam requires just enough English” took almost 2yrs of my life to get decent percentile in English (versus a native speaker may need to spend hardly 3-6 months)
    Somehow there are still few top 10 B-Schools giving priority for other factors other than GMAT & hence I ended up doing my MBA in a top 10.
    After doing this MBA, I am more confident in saying that English is not the correct exam for a B-School admissions.
    In a world with so many languages, this is not important to do a business.
    I hope the change will come in GMAT exam too, but might take few yrs as GMAC needs proof. There are successful Leaders emerging even with low GMAT score (esp in English).
    Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 126 other followers

%d bloggers like this: