Analyzing Analytical Tools

every high school student in the country. You’ll have to make do with a sample of high school students.

Maybe you know that there were 15 boys and 15 girls in your freshman homeroom, and that by senior year 2 boys and 1 girl had dropped out of school. Should you then conclude that in your country, boys drop out of high school twice as often as girls? Of course not! That sample is way too small. So imagine you find data on your entire school, and learn that of 354 boys and 373 girls who started high school with you, 32 boys and 20 girls dropped out. Now you can be more confident that boys drop out more often than girls . . .

but just how confident? Maybe you can find data on your whole school district, and learn that of 4,909 boys and 5,012 girls who started freshman year, 489 boys and 318 girls failed to graduate. Now you can be even more confident that if you were to look at all high schoolers in your country, you’d consistently find boys dropping out in greater proportion than girls.

But where do you stop gathering data? When can you be confident enough? Here’s where statistics come in. A statistical test would tell you that if your district wide sample is representative, we can be 99.99 percent confident that, in fact, in the general population of high school students, boys drop out more often than girls. That’s pretty confident.

That’s just a very simple statistical test; programs that can be run on an ordinary personal computer are capable of performing much more complex analyses on huge sets of data. The general principle, though, remains: Statistical analyses tell you how confident you can be that a pattern you observe in your sample holds in a general population. You can never be 100 percent confident, but with a few thousand cases you can often be 90 percent, or even 99 percent, confident.

To be valid, statistical tests rely on a number of assumptions — and the more complex the tests get, the more assumptions they rely on. A crucial assumption is that you’re testing a representative sample of the population. In the previous example, what if there’s something unusual about your school district? What if it’s unusually wealthy, or unusually poor? If that’s the case (and it probably is), your district is not really representative of your country’s high schools. You would need to take a broader sample, from a wide range of different school districts, to get a representative sample. The thing to remember — about this example and about sociological studies generally — is that if you don’t have a representative sample, your statistics program doesn’t know that. It will happily carry on applying tests and reporting results, trusting you to interpret those results appropriately. See the next section for more on this problem and other things that can go wrong with sociological studies.

Some computer programs are now available to help researchers analyze qualitative data. These programs can help tabulate uses of key words or phrases and let researchers highlight and tag themes to make it easier to see where and when they show up in the data. These programs can’t conduct your analysis for you, but they can help you work more quickly and efficiently as well as help you to collaborate with colleagues who may be looking at the same data.

Preparing For Potential Pitfalls

It’s exciting when you look at your data and see a pattern you hadn’t expected, or realize that you have an eye-opening finding to share. But again, you need to be cautious lest you fall into one of these traps.

Data/theory mismatch

As I mentioned earlier in this chapter, in a perfect world you’d go out and gather precisely the data you need to answer your question. But in the imperfect world we actually inhabit, you often need to settle for the best data available. This creates the dangerous possibility of using inappropriate data: data that don’t actually answer your question. Your data (the information you gather) need to match your theory (the question you’re asking, and your hypothesis about it).

For example, when my colleague and I studied the effect of kids’ high school activities on college attainment, as our outcome we took a measure of whether or not kids enrolled in college. What if, instead, we had taken

a measure of whether they had graduated from college? That would have been a data/theory mismatch because it would have missed all the kids who enrolled in college and then dropped out. Our question was whether or not the students had enrolled in college, and we had to be sure our data actually spoke to that question. What makes students more or less likely to graduate from college after they enroll is an interesting question — but it’s a different question from the one we were asking.

Getting overzealous

Statistical analysis can tell you how confident you can be that a pattern observed in your sample is typical of the general population — but that applies only to the general population of people like those you sampled, not necessarily the whole population of everyone in the whole wide world. It can

be tempting to suggest that your findings are relevant to a broader range of situations than you’ve actually observed — and often, in fact, they are! Still, you need to be cautious when it comes to telling the world what you’ve discovered and what its significance is. You don’t want to overgeneralize.

Another danger is oversimplifying: presenting your findings as being simpler than they actually are. Often, the most interesting result of a study will be

a kernel of data that jumps out of one of several regression analyses; you have a responsibility to explain to your readers and listeners that the effect you’ve discovered may vary depending on what analysis you conduct or the presence of a certain situation. If the wording of a survey question may have influenced respondents’ answers, you need to explain that. The world is a complicated place, and it’s okay to acknowledge that when you present the results of your study.

This is a valuable lesson you can take away from the study of sociology: If you take some time to look closely at sociological research findings, you’ll see that the research process that leads to a discovery is long and complex. Although this doesn’t make the discovery invalid — indeed, if a researcher is properly attentive to details and analyses, it makes the discovery especially valid — it does mean that any quick-and-dirty summary of the findings, such as you might find in the media, is bound to leave out important details.

For example, say you conduct a qualitative study of parents at an elementary school, and several of your respondents mention that they feel intimidated by the school staff and hesitant to call their children’s teachers to discuss concerns they have. You notice that the children of these parents, on average, earn unusually low grades given their standardized test scores. You also notice that of the several respondents who mention this, most are nonwhite. In your paper, you write:

These findings suggest that parents who vocally advocate for their children may reap a reward in terms of higher grades awarded to those children. The findings further suggest that this effect may particularly disadvantage minority children. Further research should focus on the content of parent-teacher conversations and the context in which they occur.

The journal in which you publish this research is excited by your finding, and they send a press release summarizing your work. Your research is then reported in a two-paragraph newspaper blurb with the headline sociologist: minority parents hurt their kids by refusing to grade-grub. That headline is not exactly inaccurate, but it seriously oversimplifies your findings. That kind of thing happens frequently in media coverage of scientific findings, and scientists sometimes play into it by presenting their work in such a way as to make their findings sound especially “sexy” — that is, intriguing and important in a way that even a layman can understand. Responsible sociologists try to avoid this.

Traditionally, surveys have been conducted by phone. Sociologists call a bunch of numbers pulled randomly from the phone book and record the responses they receive. This has never been a perfect method because often people aren’t home or, if they do answer the phone, they refuse to participate. As long as that happens only randomly, it’s fine — but what if it doesn’t? What if the people who are not home or refuse to participate tend to be relatively poor? That means that your data overrepresent the wealthier people in a society. Today, this problem is compounded by the problem that

more and more people are trading their landlines for cell phones, which don’t appear in the phone book — so a sample of people who are home, have a phone, answer the phone, and agree to participate in a survey is looking less and less like a sample of the general population.

Then there is the problem of missing data on particular variables. What if someone takes a 100-question survey and refuses or forgets to answer 5 questions that figure in your analysis? Do you throw out that person’s response entirely? Do you try to guess what they might have answered, based on what other people like them answered? There’s no obvious answer as to the right way to handle that situation, but on many surveys, a huge fraction — even most — of the respondents have missing data on some variables, so it’s potentially a big problem.

Missing variables

This is even a trickier problem. What if there are important questions that you simply forget to ask people? Usually, you just don’t know. That may lead you to believe that something you do know is more important than it is in actuality.

For example, say you’re called in to consult for a company that wants to improve its employees’ reliability. The company gives you a large set of data on its employees: their work responsibilities, their salaries, their ages, their performance ratings . . . and how often each employee misses work. You run a multiple regression analysis on these data and discover that the most significant variable is age: The younger an employee is, the more often he or

she misses work. You report this to the company, which assumes its younger employees are simply partying too much and instructs the human resources department to screen more carefully for serious-minded applicants.

But what don’t you know about those employees? You don’t know where they live or how they get to work. What if younger employees are less likely to own cars and more likely to rely on public transportation, which may be unreliable. How do you know that’s not the problem? You don’t, because information on transportation is simply missing from your analysis. Again, there’s no obvious way to avoid this problem — you need to be mindful of the possibility that there’s important information you don’t have, which may cause your analysis to suggest that some variables are more important than they actually are.

Part II

Seeing Society

Like a Sociologist

In this part . . .

One of the great discoveries of sociology is that there are some common processes and similar challenges across a range of social phenomena. No matter what you

study in society, you have to know what culture is (and what it isn’t), how to connect big social structures with everyday person-to-person interaction, and how to make sense of social networks. That’s what this part is all about.

Chapter 5

Socialization: What is “Culture,” and Where Can I Get Some?

In This Chapter

Defining culture

Separating the buyers from the sellers

Understanding socialization

Identifying culture and conflict

When I say that people are in a society together, it generally means that they interact in some way, either directly with one another or by

interacting with the same social institutions (the government, for example). It also means that they share a common culture.

In this chapter, I tell you what culture is and how sociologists study it. I explain how sociologists have developed strategies for studying everything from hip-hop music to fashion to first names to our deepest values and unquestioned assumptions about the world. I also explain how culture is spread, and how a person learns about culture from the first moment he or she opens his or her eyes and ears to the world.

Understanding how this process works means understanding that you’re not born into a single culture, but actually into many cultures: the microculture of your family; the intersecting cultures of your neighborhood, church, and school; the broader cultures of your city, region, and country; and even the global culture shared, in some part, by nearly every single person on Earth.

Socialization is the process through which you learn this culture. Some socialization happens through the media and people you encounter at work and school, but the most important way you learn about culture — and your place in culture — is at home. I explain the concept of primary group as sociologists use it, and as you can use it to understand your own socialization.

Tastes in music, movies, books, and art: Do you prefer classical music or rock music? Who’s your favorite actor? Who’s your favorite author? (Records, movies, books, and paintings themselves are referred to as “cultural products.”)

Religious views: Are you Catholic? Protestant? Muslim? Hindu? A Jew? Do you believe there is a holy text that should be read and followed?

Political views: Are you a Republican, a Democrat, or neither? Do you think we should raise taxes or cut them?

Moral values: Is it right or wrong to eat animals? What are people’s moral responsibilities to one another?

Although the average person doesn’t normally change political views or musical tastes every day, people can and do change these views without their everyday lives necessarily being affected very much.

Of course, if these elements of culture can change so easily and vary so widely, you may wonder if they even matter. Some sociologists say yes, absolutely, they make a huge difference; other sociologists say no, not really, they’re just window dressing. Even moral values, which are deeply important to each individual, don’t directly affect the overall organization of society. What everyone agrees on is the importance of structure. For more on the debate, see Chapter 3.

Adding to the potential confusion of understanding culture is the fact that sociologists sometimes draw a distinction between “real culture” and “ideal culture.” When those terms are used, ideal culture refers to the values that a society professes — for example, that college students shouldn’t drink alcohol — and real culture refers to the values that a society actually acts on — in this example, that drinking is generally understood to be a normal part of the college experience. Cultural values aren’t always consistent, even in the same society.

Breaking down structure

In sociology, the word “structure” (or “social structure”) refers to the fundamental organization of society. The overall structure of your society determines what statuses are available and how easy (or difficult) it is to move from one status to another. Your status in the social structure determines what rights and responsibilities you have.

The government: Democracies are organized a certain way; communist governments are organized another way; and military dictatorships are organized yet another way. A government’s organization affects the lives of all its citizens. The statuses available to people in the Soviet

Union were very different than those available to people in Russia today. Compared to when they lived in a communist society, Russians today have more freedom to make significant amounts of money — but they also run a greater risk of being unemployed and impoverished.

The military: Those who have access to weapons and command over armies can often force their will on others. A gun may be fired for reasons having to do with culture, but whatever culture you belong to, a bullet is a bullet. A group or an individual with enough military power can overthrow a government and bring about a new way of life (for better or for worse) for millions of people.

In a society we agree on what our structural situation is, and it is not easy for us to change — sometimes it changes in ways we can control, and sometimes it changes in ways beyond our control. Technological revolutions, economic upheavals, and military coups can transform social organization in ways that may or may not be welcome, but are hard to control or predict.

Examining the culture-structure continuum

To say that structure is more stable than culture is not to say that structure never changes, or never varies. It’s helpful to make a distinction between culture and structure, but keep in mind that the word “culture” can be used to describe even the kind of basic understandings that change slowly and vary relatively little.

As an example, look at our economic system. The thousand-dollar bills in my wallet are real, but their value is a social construction — they have value only because the people in my society agree that they have value. If I visited England, I would need to trade my dollars for pounds before I could buy anything — and if I visited a completely different society, such as an isolated tribe in the South American rain forest, my dollars would be completely worthless. Our economic system is a basic component of our social structure, but still, in some ways it behaves like the things we call “culture.”

Because the line between “culture” and “structure” can be drawn in different places — and is, in fact, drawn in different places by different sociologists (confusing, I know!) — it’s helpful to think about the distinction between “culture” and “structure” as a continuum rather than a split between two fixed categories (see Figure 5-1). Some sociologists consider “culture” to be only relatively frivolous things like fashion and style; other sociologists consider “culture” to include everything that’s in your head. You can define “culture” as broadly as you want, but the more broadly you define it, the bigger the picture you have to look at if you want to see significant change and variation.