– By Chris Wagner
©2005, The Center for Parent/Youth Understanding
At CPYU, we depend a lot on surveys. They’re useful for getting feedback and suggestions, and can provide insight on a targeted group of people. We also monitor and review the results of surveys from a variety of different and reliable sources and pass this information on to you. We inform you of the highlights, the statistics we think are important for you to understand, and provide some commentary and analysis on what the results mean in the context of today’s youth culture, teens and your ministry opportunities with them.
Surveys and their results can often be confusing, misleading and frustrating. Why is this? There are a number of reasons. Perhaps the first is because surveys deal with the messy world of human behavior. Secondly, not all surveys are created equal. There are many variables and factors for this: the sample size, margin of error, etc. Others have to do with how representative the sample was of the intended population or demographic. Then there’s the issue of bias. Who was conducting the survey and why? Were the questions asked in such a way as to solicit a certain response? Were the questions misleading? Was the study systematically designed to favor certain outcomes? The book Introduction to the Practice of Statistics by David Moore and George McCabe sums this up by saying, “In human populations, even probability samples can suffer from bias due to undercoverage or nonresponse, from response bias due to the behavior of the interviewer or the respondent, or from misleading results due to poorly worded questions.” Some surveys are simply conducted better than others.
At one time or another, you have probably read the results of a survey and thought “that doesn’t seem right” or “how in the world did they get those results?” Perhaps you’ve found that the results of two similar surveys vary greatly. Misleading information can often turn people off to surveys, disregarding them as a useful way to summarize and discover human trends and behavior. For those of you who may feel this way at times, I can empathize with you. You see, I am (or at least I used to be) a numbers guy. Before I transferred schools to study and graduate with a youth ministry degree, I was … a statistics major. You read correctly, for a short while I was perhaps the first/only/last student to be doing double duty with a youth min and math degree. Thank God, (truthfully) I eventually figured out that numbers weren’t really everything.
There’s still a part of me that wishes I had The Beautiful Mind of John Nash—sans the schizophrenic tendencies. That said, allow me to do something crazy. I’m going to throw away the numbers for a moment. I’m not saying surveys and their numerically summarized results are useless or ineffective. I believe they are a very valuable tool. We owe a lot of thanks to organizations such as the Barna and Gallup groups who do all the dirty work for us. Nearly every other article I’ve written pulls statistics from recent surveys (92% percent of them to be exact … just kidding). But for now, let’s set aside the numbers and discover how we can effectively use surveys in our ministry settings, without getting out the calculator.
At CPYU we hope to equip every parent, youth worker and educator to know and understand the changing world of the teens in their lives better than the teens know and understand that world themselves. Hands down, the best way to do this is to enter their world and spend time with them, all the while keeping a sharp eye on how cultural influences shape and direct them. That said, it is impossible to know all the styles, attitudes, behaviors, preferences and habits of every individual in your ministry setting. Nationally conducted surveys can provide a broad picture, but how can you be sure they offer a glimpse into the hearts and minds of the teens you work with each and every day? To catch such a glimpse, why not conduct your own survey?
When you conduct a survey within your own group, you get to choose the questions, you get to sort through the answers, you get to adapt the survey in such a way to meet your needs. I’m not speaking of bias here. I’m simply saying you know your group best, and should structure the survey accordingly. Are they shy? Will they be honest? Would they be more comfortable choosing multiple choice answers or are they open enough to provide truthful written responses? You know best, so you decide. Plus if you want to find out specific info, include questions that will probe into the areas of their lives that you are interested in.
How do you get started? Decide what it is you want to know. Then take a good look at the group you’ll be surveying. Would it be best to issue this survey to guys, girls or both? Should you include all the kids in your youth group, break it down into grades or perhaps create a separate set of age-appropriate questions to include only in the junior/senior high survey? Decide what is best for your ministry setting.
Last year, CPYU created a media and music survey for teens and made it available for them to take online. Later on I’ll share a little bit about what we found. But first let me give you some pointers based on what we’ve learned. Looking through the results has taught us a little bit more about surveys and how they can be used most effectively, so here are a few tips, reminders and suggestions to help you along the way as you begin coming up with your survey questions:
-
Many teens don’t like taking surveys, so make them fun—and keep them short.
-
Teens are likely to be more honest when the surveys are taken anonymously. Make sure the teens know that you’re not going to use the results to “tattle” on them or to pass on their secrets to mom and dad.
-
Teens will see right through agendas, so don’t insult them by asking biased questions.
-
If you ask a yes or no question, you’ll get a yes or no answer. If you’re looking for more details, come up with creative ways to get the teens to interact with the survey.
-
Remember, word your questions carefully. The Introduction to the Practice of Statistics points out that, “The wording of questions is the most important influence on the answers given to a sample survey.” Let your questions be honest, so the answers are honest.
-
Typically, if a teen is ashamed of the answer they’re giving, they will include some sort of excuse to justify their actions.
-
Finally, teens will lie. Don’t take every answer at face value.
This may seem like a lot of work, but don’t worry, one less game of Snood (or Halo 2, for you more adventurous type) each day and you’ll have a great survey created in no time. Seriously, with a little bit of time and effort, you will have a valuable tool to provide you with insight into the lives of the teens you work with. The hard part will be what comes after the survey results are sorted through. You’ll have to decide what you’re going to do with the results. Having the teens take a survey is pointless unless it’s going to impact your ministry in some way.
Now, doing something with the results doesn’t mean submitting a numerical analysis to the church board. We’re not talking about telling all the parents that only 32 percent of the teens in youth group are reading their Bibles regularly. (Remember, we’ve thrown away the numbers.) In some cases, it’s not going to be necessary or appropriate to tell others what you’ve found—although the girls in your group might find it helpful to learn that eight out of 10 boys in the youth group brush their teeth an average of one or fewer times a day—but then again, that would involve numbers. Doing something with the results also doesn’t mean being judgmental if you discover the teens who took the survey reveal things about themselves of which you don’t approve. In taking the survey, they were putting trust in you, and coming down on them will cause that trust to be lost quickly.
Instead, use the findings of your survey to shape and direct your ministry. You may find that students are struggling in certain areas more than others. Focus your efforts accordingly. If you are concerned about their media choices, discuss with them how to make biblically informed decisions about their media intake. You get the picture? Using the results in this way (rather than creating bar graphs and pie charts) will help you connect with your teens and allow you to truly invest more in their lives.
Before you get started with your own survey, the following are some of CPYU’s key findings from the online responses we generated from our media and music survey. Thanks to all the youth pastors and others across the country that helped us out by telling their youth groups about it via the Web and e-mail. Keep that in mind as you read the results, most of the respondents were your “typical youth group kid.”
-
Teens see TONS of movies.
-
Teens love their television. Atop the list of shows most mentioned were many from the WB and ABC Family such as “One Tree Hill,” “Gilmore Girls,” “7th Heaven” and “Whose Line Is It Anyway.” Teens also can’t get enough of Cory and Topanga as “Boy Meets World” was mentioned as often as teen hit “The O.C.”
-
No two teens like the same exact music. In fact, either they don’t know what kind of music they prefer, or they’re very confused. It was very common for their favorite CD/song to be outside their favorite genre.
-
A lot of teens admit to knowing their parents have set media guidelines for them, but these guidelines include many gray areas.
-
Nobody can spell Relient K correctly.
-
Everybody uses IM (Instant Messaging). The overwhelming response as to why: “To keep in touch with friends and be able to talk to many people at the same time.”
-
Teens visit online game sites frequently.
-
With the shifting programming of MTV to reality-based shows, it seems many teens pay little or no attention to music videos. Those who do, access the videos online.
-
Video games are definitely a growing aspect of adolescent life.
-
Teen girl magazines still dominate other teen-related titles.
-
Most teens now have their own cell phone, and their parents are paying the bill.
That’s just a portion of what we discovered as a result of asking! Hopefully you will find these generalizations helpful in impacting your ministry. But to find out more about the teens you care about, make an effort to create a useful and interesting survey for your group. If you still need help getting started, visit our media and music survey online at www.cpyu.org and adapt it to your needs. As for me, I’m off to figure out how to use that old abacus.
The Center for Parent/Youth Understanding grants permission for this article to be copied in its entirety, provided the copies are distributed free of charge and the copies indicate the source as the Center for Parent/Youth Understanding.
For more information on resources to help you understand today’s rapidly changing youth culture, contact the Center for Parent/Youth Understanding.