66 |
Part I: The Basic Basics |
Choosing a Method
Getting an accurate picture of the social world is tricky business, especially because it’s a moving target! Society changes every day — usually not too quickly. Depending on the nature of your question and the resources available to you, you’ll have to make some fundamental decisions about what methodological strategy to use. In this section, I explain what those choices are.
Quantitative vs. qualitative
The most fundamental decision you have to make is whether your study is going to be a quantitative study or a qualitative study. Those words look similar, but they mean completely different things. A quantitative study is a study where data are gathered as, or translated into, numbers. A qualitative study is a study where data are gathered as words, narratives, and impressions that would lose their meaning or value if they were somehow turned into numbers.
When are sociological data too old?
When a colleague and I studied college attainment, we were lucky to have a very large set of survey data collected by the U.S. Department of Education from 1988 to 1994. One day in 2007 we received a call from a New York Times reporter who was writing an article about college attainment. She was very interested in our study — until she learned that our data concerned students who had started college in 1993. Things had changed too much since then, she thought, for our study to be of any interest to her readers.
And yet our study was published in a good sociology journal. Are sociologists less concerned than journalists are in being accurate and up-to-date? Not at all! There are at least two reasons that sociologists found our 14-year-old data of interest when a journalist didn’t.
First, a journalist can write an article based on far less data than you typically need for a sociological study. If I want to publish a newspaper
article on college attainment, I can just summarize the results of one or two recent studies, interview a couple of kids who are on their way to college this fall, and I’m done. But if I want to publish a sociological study of college attainment, I’ll need to do a lot more research. We couldn’t just call up 14,000 kids who were going to college in 2007, such as the government did in the early 1990s. We would certainly have liked to, but a major survey like that can cost tens of thousands — even hundreds of thousands — of dollars.
Also, as I note in Chapter 2, sociologists are interested in the fundamental patterns of social interaction. Even in different social situations, sociologists believe general patterns of human interaction are fairly constant. That’s why a historical sociologist can study something that happened in the 1800s, or even in the 1200s, and discover that a large number of their colleagues are interested if the study is done well. Fourteen years? That’s nothing.
Chapter 4: Research Methods: Because You Can’t Put Society in a Test Tube |
67 |
Quantitative analysis
In a quantitative study, data may be gathered as numbers. Some information about society exists as numbers already. For example:
Annual income, in dollars
Test scores
Unemployment rates
Other information can be fairly easily translated into numbers. For example:
Sex (1=female, 0=male . . . or vice-versa)
Race or ethnicity (1=Latino, 0=non-Latino)
Marital status (1=unmarried, 0=married)
Still other information is trickier to translate into numbers, but it’s possible to at least try. For example:
Self-esteem (How happy are you with yourself? 2=very happy, 1=fairly happy, 0=unhappy)
Cultural tastes (Have you been to a jazz concert in the last year? 1 if yes, 0 if no)
Social networks (Here is a list of people in your company; answer 1 for each person you consider a friend, answer 0 for each person you do not)
The benefit of quantitative research is that it allows you to consider many cases (people or countries or whatever you’re studying) because you don’t need to “get to know” each case — the important facts about each case are right there in the numbers. With statistical analysis, you can easily analyze a huge amount of information. (More on that in the next section.)
Qualitative analysis
In a qualitative study, data are gathered as statements, experiences, or impressions and are usually recorded as words. Examples of qualitative studies include:
An interview study, where you conduct hour-long interviews with 50 people and record their comments.
An ethnography, where you spend weeks, months, or even years in a particular social setting and write about your experiences.
Participant observation, where you join a group of people and make sociological observations even as you walk in your subjects’ shoes.
68 |
Part I: The Basic Basics |
A historical study, where you read and research extensively about a particular place and time with an eye to particular features of the social environment.
The benefit of qualitative research is that it allows for a much more in-depth understanding of a situation. Because of the intense work involved, you can’t study as many cases as you can with a quantitative study — but you’re much less likely to miss something important about any given case.
Cross-sectional vs. longitudinal
Most sociological studies involve questions of causality. In other words, a sociologist asks whether or not one thing causes another. But when you aren’t taking people into a lab and conducting experiments on them, it can be very difficult to sort out what causes what. Certain research methods can make this easier — but at a price. In this section, I explain the difference between a cross-sectional study (where data are gathered at one point in time) and a longitudinal study (where data are gathered from multiple points in time). Longitudinal data allow sociologists to make more confident statements about causality, but they can be difficult to gather.
Say you’re studying the effect of television on violence. Psychologists have the benefit of using a laboratory setting: If you take two groups of randomly selected people and have one group play violent video games while the other group play peaceful video games, and the first group gets more violent than the second, you have a good idea that the games caused the violence. But when studying people in the real world, sociologists can’t just drop video game systems into some houses and not others. If people who are more violent are also more likely to play violent video games, how do we know that the games made them violent — rather than that they chose to play violent games because of the violent nature they were born with? We don’t.
A cross-sectional study is the most common form of sociological study. In a cross-sectional study, data are gathered across multiple groups at one point in time. For example, you may visit 1,000 families from different socioeconomic backgrounds, see what media they have in their homes (video games, TVs, computers, stereos, and so forth), and ask about or observe their children’s behavior. From this information, you do your best to infer what the relationship might be between media use and children’s behavior. If families with violent games tend to have children who are more violent, regardless of other factors (family wealth, school quality, neighborhood), that looks incriminating for the games.
Chapter 4: Research Methods: Because You Can’t Put Society in a Test Tube |
69 |
Even better would be to conduct a longitudinal study, where you follow a group of cases over time. For example, if you revisit those 1,000 families five years later, many aspects of their lives will have changed. Some of the
families that did not have violent games five years ago will have bought them in the meantime, and you can observe whether that’s had an effect on their children’s behavior. If it has, that’s a much more convincing finding than a finding from cross-sectional data.
So why isn’t every sociological study a longitudinal study? Largely because of reasons of time, money, and access. It takes a lot of resources to conduct a sociological study, and conducting a longitudinal study takes at least twice as many resources — plus, of course, you have to wait around while people live their lives. (People can disappear in the meantime — see the final section of this chapter for a discussion of missing data.)
Hybrid methods
Increasingly, sociologists appreciate that the ideal is to use both qualitative and quantitative, and both longitudinal and cross-sectional, methods to answer a question. Because no research method is perfect, you can cover your bases by using multiple methods.
For example, I used all these methods in my study of children’s media. My study had two main parts:
First, I tracked newspaper and magazine articles about children and media over time, assigning numerical codes to the articles to indicate which themes they addressed. That was a quantitative, longitudinal study: I retrospectively “followed” a number of publications over time, and translated their content into numbers on a spreadsheet.
Then, I sat down with teachers and parents at two schools — one urban and one suburban — to interview them about their views on children and media. That was a qualitative, cross-sectional study: I talked with people from two different social groups, and wrote about what they said without turning it into numbers.
That was a useful approach, but in the end I decided that the issues were just too complex to be distilled into numbers, and I presented a qualitative analysis rather than a quantitative analysis of the newspaper and magazine articles.
Conducting a sociological study is kind of like fixing a car: You’d better bring your whole toolbox because you don’t know ahead of time what (methodological) tool is going to be the best one for the job.