By Ryan P. Burge and Paul A. Djupe

It started when we were writing out the code to classify American’s religious identity in a new survey and quickly became frustrated that there were so many missing cases. If anyone has coded before, it seemed obvious that our code had some small but consequential error and so began the hunt. When everything checked out, we found the problem that was hiding in plain sight – the coding rules themselves had a small hole that has grown much larger with time. In our new paper forthcoming in the Review of Religious Research, we examine the logic of this coding decision with data and then recommend new coding rules going forward.

Figure 1 below shows the growth in those left “unclassified” in the RELTRAD scheme laid out by Steensland, Park, Regnerus, Robinson, Wilcox, and Woodberry in their influential 2000 article in Social Forces. Though not the only classification scheme out there, it is commonly used and widely cited. The perennial problem in classifying American religion is what to do with the hundreds of Protestant denominations. RELTRAD creates the mainline and evangelical Protestant categories, but struggles with what to do with those who don’t claim a denomination. The solution was to cut non-denominationals who reported attending worship at low levels under the theory that they were not showing valid signs of being sufficiently religious to be counted. This is the only time in the RELTRAD scheme where a worship attendance filter is applied. But, because attendance is declining across the board and the share of non-denominationals is rising, the filter was catching more and more people across time. Those left “unclassified” by the scheme were only 2 percent in 1972, but had more than tripled to 6.5 percent by 2018. That’s a lot of people cut from an analysis using this coding scheme. Who are they?

Figure 1 – The Growth in the “Unclassifieds” in the RELTRAD Scheme from 1972-2018 (General Social Survey)

But, is kicking out these low attending non-denominational justified? Do they look different from denominational identifiers? Figure 2 starts a comparison of the worship attendance levels of non-denominationals with Southern Baptist and United Methodist affiliates. If we simply followed their denominational examples, we would expect the same proportions of non-denominationals to attend at low levels. And that’s what we see – slightly more non-denominationals are low attenders (33%), but not too many more than in the SBC (26%) and UMC (29%).

Figure 2 – Comparing Worship Attendance Levels of SBC, UMC, and Non-denominational Affiliates

Do the non-denominational respondents who attend at low levels demonstrate religious beliefs and political attachments that signal distinctiveness? In Figure 3, we break out low and high attenders and compare the three groups. High attenders are much more similar to one another, whereas low attenders show greater spread in these attributes, on average. But, in almost all cases, the non-denominationals lie between SBC and UMC affiliates. That is, low attending non-denominationals do not appear to drop off a religious cliff, but show religious and political patterns in line with their evangelical and mainline denominational cousins.

Figure 3 – Comparing Low and High Attending Protestants in their Religious Beliefs and Political Attachments

So, what should we do about this? We believe we have good evidence (there’s more in the article) to show that using an attendance filter is not appropriate to be used on one group. Instead, group members systematically differ internally based on how frequently they attend worship services. That is, non-denominational low attenders are as different from high attending non-denominationals as, for example, low attending Catholics (and others) are to those who attend mass regularly. The attendance filter should be used across the board in a consistent way – either universally or not at all.

Instead, we advocate for the principle that classification should do one job, not two. It should classify people and leave levels of attachment, identity, involvement, etc. to other variables. Toward that end, we argue that RELTRAD should incorporate a new category: non-denominational. They are not mainline nor evangelical and their existence outside a denominational structure makes them distinctive. Moreover, adding the non-denom category would enable us to pursue new sets of questions worth asking (as we’ve found in a book in development).

The upshot of doing so is to recognize that non-denoms are an enormous and quickly growing group, larger than mainline Protestants and the fourth largest religious group behind the nones, Catholics, and denominational evangelicals. Naturally, the evangelical category (including all races) shrinks with this coding decision, pegged in 2018 at about 16 percent, down from 23 percent when it included high attending non-denoms. Obviously, the number of unclassifieds drops too, down from almost 7 percent to a much more acceptable 2.

Figure 4 – Our Solution is to Recognize Non-Denominationals as Their Own Category.

Classifying religion is an important task and requires vigilance over a constantly churning religious economy that is adapting or even changing identities in order to better compete for members. For instance, because of the damage of associating so closely with Trump, some are calling for distance from the label “evangelical.” Therefore, one implication is that researchers should express extreme caution when employing classification in a causal framework. Classification schemes, like RELTRAD, are academic abstractions to enable analysis of survey data with small samples. Instead, it is ideal to get a handle on just what individuals believe and what they are exposed to within their particular houses of worship, which has shown to be quite different given the community, mix of congregants, and religious leadership. Still, classification schemes are useful in the description of American religion and we recommend this fix to adapt RELTRAD to a changing religious economy.

The new article is:

Burge, Ryan P. and Paul A. Djupe. 2021(forthcoming). “Falling Through the Cracks: Dealing with the Problem of the Unclassifieds in RELTRAD.” Review of Religious Research DOI: 10.1007/s13644-020-00441-y

Ryan P. Burge teaches political science at Eastern Illinois University in Charleston, Illinois. He can be contacted via Twitter or his personal website. Syntax for this post can be found here.

Paul A. Djupe, Denison University, is an affiliated scholar with PRRI, the series editor of Religious Engagement in Democratic Politics (Temple), and co-creator of religioninpublic.blog (check out his posts). Further information about his work can be found at his website and on Twitter.