By Paul A. Djupe and Ryan P. Burge

In our recent post, we checked to see whether divergent measurement schemes for mainline Protestants returned similar estimates of their political affiliations and opinions. TL;dr: They did. While that may be reassuring to many researchers (ahem, us) who have relied on a born-again identity rather than a denominational approach to differentiate Protestants, it raises some uncomfortable questions about American religious classification. That is, roughly 25 percent of the people identified under each scheme are not shared with the other. So, while most are shared — they identify in a way that is consistent with their affiliation as defined by the “reltrad scheme” — many are not, which suggests yet again that religious traditions are quite heterogeneous entities.

An exchange with Clyde Wilcox on Facebook this morning reminded us that he was publishing on religion in the 1980s. Well, not just that, but that he (and others) have been pointing out that these problems with religious measurements have been systematic and may be organized spatially. Here’s an excerpt from his (read free online!) 1986 Journal of Politics article:

Early research operationalized fundamentalists by their denominational affiliation, while most later research has focused on doctrinal beliefs. This analysis suggests that the groups identified by these two strategies are different in demographic characteristics, political beliefs, and political behaviors. The fundamentalists identified by the doctrinal definition have higher levels of socioeconomic status, and are less Southern and rural than those identified by the denominational definition. These doctrinal fundamentalists are more conservative on women’s issues and economic issues, and are more politically active and Republican. The analysis suggests that attempts to integrate literature that has employed these different operational definitions should be made cautiously.

Datasets were small back then but our measurement problems have not changed. Large datasets with multiple measurement schemes can help us see if religious misclassification is systematically related to the religious culture in the geographic area — the state. The two versions of misclassification we examine among white Americans are (1) mainline Protestant affiliates who identify as born-again evangelicals and (2) evangelical Protestant affiliates who do not identify as born-again evangelicals. We use the enormous sample that comprises the 2016 CCES, which has sizable state samples, to see how this is linked to the state portion of born-again identifiers in the first case and non-born again identifiers in the second case.

As shown in the figure below, mainline misclassification ranges from the single digits in the Northeast to near a majority in the deep South. That alone tells us that regional cultures affect how people perceive their religious identities. Knowing how born-again the state is helps us understand 73 percent of the variation in how mainliners identify themselves — this is a very strong relationship. It is interesting to see what states fall off that prediction line. States in the Midwest with a moralistic political culture (e.g., the Dakotas) have higher rates of born-again mainliners than we might predict, and states in the West with an individualist political culture (e.g., Idaho) tend to have lower rates than we might otherwise predict. There may be something about the local norms regarding conformity that influence how people adopt identities.

The figure below shows misclassification rates among evangelical denominational affiliates and how they respond to the state non-born again context. It is interesting to note that misclassification among evangelicals has a higher set point — the lowest is just under 20 percent and the highest is over 60%. This is ironic given that the born-again/evangelical measurement scheme has always been criticized for being biased toward the evangelical subculture. Well, there’s clearly slippage in 2016! Knowing the portion of the state that is not born again helps explain 79 percent of evangelical misclassification. Again, this is a very strong relationship linking the state religious culture to individual religious identification.

Notably, the relationship deteriorates at the very high end where evangelicals are not plentiful. Those sample sizes, therefore, are small and the estimates are more error prone. Still, they are in range of what we would predict — very high rates of misclassification among white evangelicals.

To many of us, none of this is surprising, though we honestly did not quite expect to see such large variation in misclassification nor how tightly it would be linked to the state. What this adds up to is an explanation for why both of these religious measurement schemes return equivalent estimates of the politics of mainline Protestants — given the influence of the state context, these measures are proxies for it.

Sociologists have been demonstrating for years that religion is a social institution and individuals learn how to be religious from their peers in the pews. This is clearly at a different level of aggregation, but the message is the same. And given the high level of variation and tight connection to geography, it begs the question why common measurement schemes are trying to bulldoze over this variation.

Paul A. Djupe, Denison University Political Science, is an affiliated scholar with PRRI, the series editor of Religious Engagement in Democratic Politics (Temple), and co-creator of religioninpublic.blog (see his list of posts here). Further information about his work can be found at his website and on Twitter.

Ryan P. Burge teaches at Eastern Illinois University in Charleston, Illinois. He can be contacted via Twitter or his personal website.

The syntax for this analysis available on Github.