(Please also see the main blog page, with all entries.)

New results for foreign learners

25 July 2011

We knew that our new exposure would give us lots of data in the U.S. and other English-speaking countries. What we didn't expect was that the quiz would have so much international popularity!

In fact, Russia has generated almost half as much participation as the U.S., Germany has produced nearly 6,000 visits, China over 5,000, and Iran exactly 4,000 as we write this. We've even had solitary visits from Namibia and Zambia.

And probably half the e-mails we've received have been from non-native speakers asking for numbers to compare their personal results with. This presented us with a problem: how to produce numbers that are meaningful enough to compare one's results with?

The obvious answer would be vocabulary correlated to the number of years people have studied English. We tried that, but these numbers have turned out to be all over the place. There are so many different types of courses, and so many ways of informally learning English, that the data has been fairly meaningless so far. Plus, most people taking the test and filling out the survey already have a decent level of English, since the survey itself is in English — so lower levels are probably severly underrepresented.

We'll be surmounting many of these problems with our future launch of this test in Brazil, where styles of English learning are much more uniform, and easier to compare and correlate. (Plus, the interface will be available in Portuguese, so we can include more introductory students.) But we don't want to keep everyone waiting until then.

So we've decided to make available a chart simply showing the overall distribution of vocabulary scores, with the percentage of respondents which fall into the range centered on each score:

The largest proportion of respondents (4.7%) know 4,500 words (or are in the range from 4,250–4,749, technically). Looking at it another way (not displayed on the chart), the median vocabulary size for all respondents is 7,826 — half know more, half know less. But remember that these statistics, while they might be fun to compare yourself against, merely reflect the people who have taken this online quiz from all over the world, and is in no way representative of the world population as a whole. Percentages for people who know less than 1,000 words are not even shown on the chart (the data is too spotty/erratic so far).

However, this doesn't mean we haven't found anything else interesting. To the contrary! Because while our data isn't fine enough to find much correlation between vocabulary size and the number of years of English classes taken, we did find great differences in average vocabulary results for the following questions:

Academic performance: On average, in the English course(s) you took, how would you judge your performance in the classroom, relative to the other students you studied with?

Participation: In class, how much did you participate, talking and asking questions, compared to other students?

Natural ability: In class, compared to other students, how much do you feel, or did you feel, that learning English, and speaking it, was easy or difficult for you?

Outside of class: How much did/do you use English in "real life", learning things outside of the classroom? (Watching TV, listening to songs, writing, travelling, etc.)

Time spent abroad: Have you ever travelled to a country, or to countries, where English is spoken? If the answer is 'yes', how much time did you spend?

So, summing it up for non-native learners of English, what does this mean?

While the charts above should not be interpreted as a scientifically controlled survey, and represent only the voluntary responses of a self-selected Internet survey group, they do suggest a few things:

Academic performance helps, up to doubling your vocabulary size. But that doesn't tell us what helps academic performance.

Classroom participation matters too, but it's not the top factor. It appears to give you up to a 50% boost in vocabulary.

Outside of class is the biggest difference. Students who do "lots" of things in English outside of class have more than twice the vocabulary of those who "don't do much."

Living abroad gets you to and beyond 10,000. Up to one year abroad brings the average student from around 7,000 to around 10,000 words. After that, every year abroad gives you around 850 more words, or around 2.35 per day. (Compare that to the average American adult who learns 0.85 per day.)

But be aware that the results above are suggestive only—we have not separated out the different factors from each other statistically. So, for example, higher vocabulary sizes among people with lots of English activity outside of class might not actually be due to their learning at the time, but the fact that it made them more likely to live abroad afterwards. Or higher vocabulary sizes for top academic performers might simply be due to the fact that they took more years of classes, while others dropped out sooner. Or indeed, causation might run in the opposite direction — students whose English is already better might be more inclined to participate in class, and engage in extracurricular English activities. More research will tell.