I’ve been thinking about the different groups in the Critical Social Justice Attitude Scale (CSJAS) studies and the scale’s ability to measure what the participants mean when they say they could be described as “woke” by their friend. These participants account for a third of the sample. 18% of men and 48% of women self-reported as “woke”. The CSJAS was validated in part by looking at its correlation with this self-reported “wokeness”.
In medicine, a test’s sensitivity and specificity mean its ability to confirm real positives and negatives and discard false ones. Used as a binary test, the CSJAS can predict whether women are “woke” or not with roughly 80% sensitivity and specificity, and both go up for Gen Z women. It does a particularly good job in predicting which men are not “woke”, with specificity of 97%. Women who say they are “woke” agree with nearly the entirety of the Critical Social Justice Attitude Scale. Women who say they are not “woke” reject everything on the scale except for one item. And men who say they are not “woke” want nothing to do with the statements. There is one group however, who say they are “woke” but clearly mean something else by it than the rest of the sample.
Men in the study who say they are “woke” present a conundrum. The scale doesn’t predict their “wokeness” well as sensitivity is essentially a coin toss with 54.5%. That means that out of the 18% or so of men who say they are “woke”, only little over half actually agree with the CSJAS items more than they disagree with them. Overall, this group doesn’t endorse a single scale item. This statistic sits well with my impression of occasionally seeing some men disparage the scale items as “nonsensical” online. However, “woke” Gen Z women, on average, agreed with every single item on the scale and scale scores somewhat reliably predict “wokeness” in the other 80-90% of study participants.
I’ll repeat the last part: young women who say they are “woke” agree with every single item on the Critical Social Justice Attitudes Scale. And the items can be used to predict whether they self-identify as “woke” or not, as it can be used to predict which men and women self-identify as not “woke”. What do the men who self-identify as “woke”, believe then?
Out of the entire battery of 20 test items there are two items they (barely) endorse: “Transwomen are women.” and “We should have more safe spaces in society.” Barely, as the means fall in the middle of “not agree, not disagree” and “somewhat agree”. What is meant by “woke” then, for this group, seems to be a lukewarm assessment of the entire battery of critical social justice beliefs, that their female counterparts take onboard, and an unenthusiastic endorsement of “transwomen are women” and adding safe spaces.
Seen through this lense, I can see why the CSJAS may appear ludicrous to some men in this group. If you’re not enthusiastic about combating microaggressions or cultural appropriation you are perhaps not going to like a scale that assesses your “wokeness” using these indices. The simplistic formulations may feel like they add insult to injury. Here, I feel like I should repeat the caveat that these items were designed more to capture an attitude, and less to be nuanced truth claims, though it would be great if they did both.
Psychology questionnaire items can sometimes feel frustrating in this way – they can feel like they are belittling the complexity of the measured phenomenon and your rich experience of it. They are just trying to do the least bad job of balancing nuance with succinctness. I’m sure CSJAS is guilty of this. I’ve seen a fair bit of criticism about how the items lack nuance or can be interpreted in different ways, but few have been able to suggest improvements to them, let alone better items. I’ve made a further round of possible improvements on them after the study.
However, as mentioned, the other 80-90% of the study sample seem to be mostly in agreement that yes, what they mean by “woke” is close enough to what the CSJAS has on offer. There is thus a rift between genders in this too (as there is in the prevalence of CSJA between genders) – men self-report as “woke” even when their idea of what that means is very different from women and “non-woke” men. “Wokeness” for “woke” men struggles to acknowledge that “transwomen are women” but for women encompasses a host of beliefs ranging from wanting skin color to be discussed more to decolonizing reading lists and combating microaggressions.
What should a measure maker do with this information? First, I wonder whether this finding is particular to Finland or if it replicates elsewhere. The measure has already been translated to other languages and other studies are planned, so more data may elucidate the question. If this finding is universal, then either critical social justice attitudes are to be defined differently for different groups or we have to wait for the different understandings of the term to converge between, say, left-leaning men and young progressive women.