How to measure race

[eldan]

More by luck than judgement, I have become Mr. Survey at Sustainable Seattle, which I'm actually rather pleased about. We have this happiness survey that we're trying to get everybody to take (fairly successfully - we had more than 6,000 responses the last time I checked), and I am the main person doing both tech support for the online survey

Comments

[dymaxion]

What if you have a large list of checkboxes, some of them fairly specific and some of them from broader categories, and they can check those of which apply? This means that you can check "jewish", "white non-latino", and "turkish", for instance. Similarly, for gender, checkboxes for identifying as "male", "female", "neutral", etc.

[eldan]

That might be the way to proceed, with the caveat that for race it's turning out to be a real challenge to even come up with an inclusive list for the whole world. I'm hoping we can use the "other" box to iteratively refine that (and that's a big thing I have to keep reminding myself: what we do right now doesn't have to be final).

For gender, what's the list you would use? And should we be concerned about people who still feel that none of the checkboxes presented describe them?

[dymaxion]

For gender, I'd say "female", "male", "neither", "other" as being a pretty good set, and as checkboxes, not radio buttons. It won't get absolutely everyone, but it will mean that most people will be able to find a way of expressing their identity with the specific option to dissent ("other") available, without being too challenging/confusing for more mainstream respondents.

[eldan]

I agree that allowing people to check multiple boxes is a crucial part of getting this right. I think it's more obvious for race, but makes sense for both questions.

[ursule]

The U.N. stats division says there's no good universal method:

http://unstats.un.org/unsd/demographic/sconcerns/popchar/popcharmethods.htm

Looks like the E.U. doesn't have a standard either.

I might lean toward having an ethnicity option, with checkboxes rather than radio bubbles and an "Other: _____" option, and setting which checkboxes appear depending on the country.

[eldan]

Ah, thanks for that link - I hadn't seen anything quite that official. This recommendation, from the page you linked to, is also interesting and may help keep myself sane: "It is also suggested that the primary classification consist of only a few broad categories". Between this iteration of the survey and the next one, I'll also have to sit down and read some of the references from there - one in particular looks very handy: “ETHNIC CLASSIFICATION IN GLOBAL PERSPECTIVE: A Cross-National Survey of the 2000 Census Round” (PDF)We don't want to adjust the checkboxes by country, partly to make the survey as universal as possible and partly because that creates its own minefield of offending whichever countries we haven't tailored the list for yet (I'm guessing it won't be easy to find a standard list for every country). We definitely will allow multiple selection and free text entry for "other" - it's basically a question of trying to give as many people as we can checkboxes that are meaningful to them so the results aren't dominated

[chilimuffin]

I don't know if you can have a universal categorization, especially if what you really want to get at isn't race but racism, because then you have to ask people to fit themselves into the groups their particular culture identifies as self vs other/majority vs minority, and that's incredibly site-specific. If you want to the race of people in the U.S., you'll need a gazillion categories, but if you want to know how racism impacts them, maybe you need an additional question like, "when others look at me, they usually think I am ____" with - at least in the U.S. - some larger generic race boxes. I don't know if that's a validated question or not, but I think the answers could be very interesting and more to your point.

[eldan]

You make a very good point, which I hadn't thought about in quite that way. I'm going to bring this up at our next happiness initiative meeting and see what people think of it. It's far enough outside what I'm used to seeing that I'd want to discuss it at some length before throwing it in the survey, but I could see it generating some really interesting data. Even just comparing that question with the traditionally-worded "which race[s] do you identify as" could be quite a rich vein.

[chilimuffin]

true. I'm sure there's literature on it somewhere in the anthropology world, especially for groups like those from the Caribbean (or Africa) and "black."

On a side note, having taken the survey, now (rather than just pontificating before): I may have read it wrong (and it didn't apply), but in your demographic collecting section, it appeared that the relationship category ran as "Single (never married or in a domestic partnership)." Unless I'm blind (which is very possible), there was no "In a relationship (not a domestic partnership or marriage)" which I think means you're missing a large demographic. I'm not sure it matters, depending on what you need the data for, but I thought it was interesting.

I find it highly interesting how much the stress of my job (high) impacts the satisfaction of my job (also high). I have no doubt it's true, and it was very interesting to see that in a numeric format.

[eldan]

You're right about the relationship status question. It's something we haven't looked at yet, so it came straight from the Canadian project who first put this survey together, and I wonder if it's a bit lost in translation since Canadian law would recognise many more households as a domestic partnership than US law does.

One of the things we're hoping to do in a future, fuller rewrite of the survey is to ask more about happiness at work, which I don't think the current one goes into enough. But first there's a project to identify redundant questions so we can take some out at the same time and not make it even longer....