There is a great
article in this month's Christianity Today about how Evangelical Christians have misused statistics and survey results - sometimes maliciously and other times simply in error. As I read the article, I discovered that some of the information I believed was true was, in fact, not. It goes to show that it's important we all do our homework and not just believe what someone else says...
Curing Christians' Stats Abuse
The statistics we most love to repeat may be leading us to make bad choices about the church.
Ed Stetzer | posted 1/15/2010 09:44AM
The reporter's question was one of the best I had ever been asked. "Why do you evangelicals love to make up and say such bad things about yourselves?"
Great question, I thought. But I'm here to talk about social science research, not abnormal psychology.
I was facing a room full of reporters in a Religion Newswriters Association session at the Washington Post building in D.C. They had invited me to explain the difference between good religious research and bad. It's a real problem. News reports are always batting around some new bit of bad research. And sometimes a snippet from good research gets pulled out of context, then mangled, garbled, and spewed all over.
Research Gone WildOnce a choice morsel of misinformation gets out, it multiplies faster than dandelions in the spring. We have all heard these soul-seizing yet false factoids. Some of us have even repeated them:
"Christianity will die out in this generation unless we do something now."
"Only 4 percent of this generation is Christian."
"Ninety-four percent of teenagers drop out of church, never to return again."
And perhaps my favorite: "With its 195 million unchurched people, America has become the new mission field. America has more unchurched people than the entire populations of all but 11 of the world's 194 nations." The "195 million unchurched people" statistic is all over the place—from books to blogs to church bulletins. And those who quote it often attribute it to researcher George Barna.
The problem is, it isn't true. That's not what the research showed, and Barna wasn't the one who conducted the study.
The original stat came out of a project I was a part of while working with the Southern Baptist Convention's North American Mission Board (NAMB). We researched the number of unbelievers in the U.S., not the number of unchurched people. But someone somewhere changed the language, and thus the meaning.
Three years ago, Christianity Today sister magazine Books & Culture carried a provocative article by Christian Smith entitled, "Evangelicals Behaving Badly with Statistics." Smith, a highly respected professor of sociology at the University of Notre Dame, declared that American evangelicals "are among the worst abusers of simple descriptive statistics."
He went on to dissect an advertisement for a summit that declared, "Christianity in America won't survive another decade unless we do something now." The summit organizers claimed that only 4 percent of today's teenagers would be evangelical believers by the time they became adults. "We are on the verge of a catastrophe!" the advertisement screamed.
As it turns out, that 4 percent statistic comes from an informal survey of 211 young people in three states conducted by a seminary professor nine years earlier. Smith affirmed the professor's approach but explained that an unwarranted inference was drawn from a small, non-representative sample to reach conclusions about the future faith conditions of entire generations.
Smith wrote, "Why do evangelicals recurrently abuse statistics? My observation is that they are usually trying desperately to attract attention and raise people's concern in order to mobilize resources and action for some cause …. Evangelical leaders and organizations routinely use descriptive statistics in sloppy, unwarranted, misrepresenting, and sometimes absolutely preposterous ways, usually to get attention and sound alarms, at least some of which are false alarms."
My friends at NAMB and my boss, Thom Rainer (the originator of the 4 percent statistic), accurately reported their methods and conclusions. But the research took on a life of its own. Unfortunately, good people who are trying to help the church change its bad habits in order to reach a lost world often misappropriate the research. Evangelical Christianity in the U.S. undoubtedly faces serious challenges, but hyperventilating doesn't help—even when the statistics are accurate. Crying, "The sky is falling!" might sell books, but it never fixes problems.
I suspect that we are attracted to bad statistics mostly for motivation. We need a personal push and hope to give one to our churches as well. But bad stats can feed self-loathing and lethargy as much as they can encourage steadfastness in mission.
We will Always have the ARISAnother way we misrepresent the state of American Christianity is by seizing upon one startling aspect of a study and pulling it out of context, sometimes ignoring other findings in the study. This happened with the American Religious Identification Survey (ARIS), released in March 2009. It found a significant decline in religion and a rise in secularism that set news outlets buzzing.
"The percentage of Americans who call themselves Christians has dropped dramatically over the past two decades, and those who do are increasingly identifying themselves without traditional denomination labels," wrote Michelle Boorstein of Washington Post. Cathy Lynn Grossman at USA Today reported, "The percentage of people who call themselves in some way Christian has dropped more than 11 percent in a generation. The faithful have scattered out of their traditional bases: The Bible Belt is less Baptist. The Rust Belt is less Catholic. And everywhere, more people are exploring spiritual frontiers—or falling off the faith map completely."
The ARIS correctly observed that denominations and denominationalism are in decline, the cultural influence of Christianity continues to slip, more people are describing themselves as nonreligious (now 15 percent), and minority religions like Islam and Wicca are becoming more popular.
Some observers also latched onto the ARIS, finding that nonreligious Americans have dramatically increased in number, and concluded that evangelical Christianity in the U.S. is on the verge of annihilation. But the ARIS finding was only half the equation.
The ARIS also reveals a simultaneous increase in the number of Americans who self-identify as evangelical Christians. There is a lot in the ARIS that should be disturbing to the church, but the decline in the percentage of self-identified Christians that the ARIS found falls far short of a great crackup in evangelicalism.
What does the Good Research say?One of the first things you learn in research is that there are many ways to look at things. Every time we have new data from a well-designed study, it helps us. But one conclusion from one study is no foundation for a theory on the future of a society.
To get the whole picture, responsible researchers look at various studies, their methodologies, and their results. We must understand the parts in light of the whole. We should interpret each finding in light of the full study, and interpret each study in light of other studies. We reach bad conclusions when we latch onto one finding of one study, drag it out of context, and proclaim it from the rooftops without knowing whether our interpretation is justified.
The ARIS: Perhaps its most interesting finding was the stunning decline of Catholicism in the Northeast. In addition to that trend, 15 percent of Americans claimed no religion at all. The growth that did occur in the Christian population was among those who would identify only as "Christian," "evangelical/born again," or "nondenominational Christian." These groups, 5 percent of the population in 1990, stood at 11.8 percent in 2008.
General Social Survey (GSS): One relevant aspect of the GSS (conducted every other year since 1972) is the snapshot it provides about worship service attendance. My book Lost and Found included a 1972-2006 GSS chart that showed that the percentage of 20-somethings attending weekly worship services has been rising since 2000, after a serious dip in the mid-1990s. My co-authors and I admitted that only time would tell if the rising trajectory would continue. Since then, the 2008 data showed another uptick, bringing attendance among evangelical 20-somethings back to what it was in 1972. Among non-evangelicals there was indeed a decline: Just fewer than 25 percent attended weekly in 1972. In 2008, it was just over 12 percent. Listening to some commentators, you might conclude that young adults had left the church. But that is not what the data tell us.
Gallup: In April 2009, Gallup reported that church attendance among Protestant young adults rebounded in the 1980s and is now close to the level it was in the 1950s. Of course, "Protestant" covers a wide range of denominations, some of which have experienced high membership growth, others of which have declined. Gallup generally hasn't shown any change in reported church attendance, much less a massive decline among 20-somethings.
To be fair, these numbers are likely influenced by the "halo effect," where people are more likely to say they go to church than to actually go. People always inflate reports of behaviors when they perceive that the behavior in question is desirable. It's significant, though, that the numbers on weekly church attendance haven't changed substantially. If Christianity were in steep decline, the social desirability of it would be, too—which would reduce the halo effect.
Baylor Religion Survey: The 2007 Baylor Religion Survey, reported in Rodney Stark's book What Americans Really Believe (2008), found that, of the 11 percent of Americans who said they had "no religion," two-thirds expressed some belief in God. In addition, many were not "irreligious" but merely "unchurched." More to the point, the study found that people do not use activities of the "scattered" church—religious activities not affiliated with or sponsored by a congregation—as a substitute for participation in the "gathered" church.
The study exposed as a myth a widespread dissatisfaction with organized religion. Of those who are engaged in "scattered" activities, such as prayer and Bible study groups, 80 percent frequently attend church. While many Americans are disillusioned with organized religion, theirs is by no means the prevailing attitude. Organized churches face serious issues, but the facts simply do not support the mantra that churches are dying out.
Pew Forum on Religion and Public Life: A 2008 study conducted by the Pew Forum found that 40 percent of religiously unaffiliated people say religion is still important in their lives. Further, the research found that the "unaffiliated" group does a lousy job of retaining adherents. Pew found that 39 percent of those who grew up unaffiliated are now Protestant, most of them evangelical, while another 15 percent now affiliate with Catholicism or another faith. Though we hear a great deal about young people leaving the church, we hear few reports about the stream of young people coming into the church.
What are the Concerns?It's hard to generalize about American Christianity. The scene is just too diverse. But the most reputable studies give us certain indicators about particular denominations and the spiritual lives of U.S. adults. Mainline denominations are no longer bleeding; they are hemorrhaging. Increasingly, they are simply managing their decline. For evangelicals, the picture is better, but only in comparison to the mainline churches. Southern Baptists, composing the largest Protestant denomination in the U.S., have apparently peaked and are trending toward decline. The same is true of most evangelical denominations. Only 2 of the top 25 Christian denominations are growing: the Assemblies of God and the Church of God (Cleveland, Tennessee). Both are Pentecostal.
Still, those worried about church decline are worried about data beyond the simple "more" or "less" numbers. The bigger concern is that people who identify themselves as Christians (and even evangelicals) do not evidence the beliefs historically held by Christians.
The Shape of Faith to Come, a 2008 book by Brad Waggoner (and based on a LifeWay Research study), evaluated seven domains of spiritual formation: learning truth, obeying God and denying self, serving God and others, sharing Christ, exercising faith, seeking God, and building relationships. It found that only 17 percent of Protestant churchgoers in America scored the equivalent of 80 percent or higher in those key areas of Christian discipleship. A full 57 percent of respondents said they had not once explained to another person in the past six months how to become a Christian. Then, over the course of the next year, only 3.5 percent showed a net increase in spiritual growth. These data are cause for concern, for sure. The church cannot grow if Christians are not actively discipling new believers.
American Christianity is not DeadReports of Christianity's demise in America have been greatly exaggerated. While the main thrust of good research does indicate that the percentage of Americans who self-identify as Christians is declining, these data are not necessarily a bad thing. If three out of four Americans call themselves Christians, we are in big trouble. Three out of four Americans certainly do not live like Christians. Christianity becomes confused when everyone is a Christian but no one is following Christ. We evangelicals believe that most Americans do not have a relationship with Jesus Christ.
There is little doubt in my mind that the cultural expression of Christianity in America is declining. True, Christianity is losing its "home-field advantage" in North America. At the same time, some trends tell us we are seeing the growth of a more robust Christian faith and commitment. We are seeing some abandon nominal Christianity, and many others retain an authentic Christian faith. Christianity in North America is not going to die out in this generation or any other, even though it is going through an identity crisis of sorts.
In the meantime, bad and misinterpreted data must not convince us that organized Christianity in America is dead and gone. Facts are our friends. The facts tell us that the church in North America is struggling but also, in many places, growing. Discerning research can help us diagnose our condition. It may even help the church find strategic means to address the mission field right outside our doors. And ultimately, we all can agree that a declining church needs the unchanging gospel.
Ed Stetzer is president of LifeWay Research and co-author of Lost and Found: The Younger Unchurched and the Churches That Reach Them (Broadman and Holman).
Copyright © 2010 Christianity Today. Click for reprint information.
