The sting from a sting operation and the Trojan horse of ideology

The sting from a sting operation (and the Trojan horse of ideology)
Griff is a dancer in a troupe of dancers but recent criticism has suggested Griff cannot dance. This
criticism, rightly construed, questions the membership of Griff in the troupe. Does Griff deserve to
be considered a dancer? But in response, Griff has suggested that the criticism aims to undermine
the whole troupe, it aims to delegitimize dancing itself. Simultaneously, complaints are made that
the dance competencies of other members of the troupe have not been criticized equally, and that
criticizing Griff will play into the agenda of another troupe. This move seeks to draw cover from the
group while spreading concerns across the group. But none of it addresses the question of whether
Griff can dance?
Dancing was not the focus of the 18-month sting operation by Helen Pluckrose, Peter Boghossian
and James Lindsay (PBL). It was academicing. And the criticisms weren’t aimed at a person but the
methodological approach/epistemological lens (i.e. applied postmodernism) prevalent in a subset of
disciplines that are members of the Humanities and Social Sciences. Academicing (or Research and
Scholarship) is not about spit balling ideas and publishing brain farts in the hope that they generate
interesting and constructive conversations. I mean, the spit balling and the brainstorming are a part
of it, they are important in the generation and selection of ideas to study. They are important for
academic inquiry, they are the core of the interesting conversations we have about the bounds of
our ignorance. But they are not the scholarly outputs of an academic process. They are not the
publishable artefacts, the possible end-points of a study. Academic scholarship is not just saying
thought provoking things. Research and scholarship involves evaluating and constructing reasoned
arguments, building from first principles or from the shoulders of giants, and where possible
supporting those arguments with data so empirical claims can be made – and then tested, supported
or refuted by others/other data. Philosophy (or philosophizing) is maybe closest to intellectual spit
balling but even then rigor and reason is a bare minimum. Good academic work involves a lot of
critical thinking, but a lot of thinking is not necessarily good academic work. There is an attempt to
re-define academicing as waxing lyrical (sans reason, logic, coherence, and empirical data) once the
melody inspires some response from others. This is obviously too broad a re-definition. It would
make Jesus, or Hitler, or Lennon and McCartney academics; even Milo Yiannopoulos would be an
academic. So, the sting operation attempted to shine a spotlight on an area of study that PBL believe
is masquerading as academia.
Though the investigation was cut short, the endeavor (now referred to as #SokalSquared) was
reported by PBL in Academic Grievance Studies and the Corruption of Scholarship. It is clear from the
title alone that their concern was with the academic credentials of disciplines in this field of
grievance studies. It is a focused critique. It should not be considered an attack on scholarship – it is
the very opposite, a defense of scholarship from re-definition, or corruption.
But their critique was quickly straw manned as an attack on Humanities scholarship generally. Their
exposé was even framed as a coordinated right-wing attack (even though all are left leaning); their
characters and their motives have been maligned. The charlatan disciplines that were the focus of
the sting operation hide in the open, behind the fears of neighboring disciplines (that they’ll be
next). They coast on -- not under the coat tails of the legitimate social sciences. They hide in plain
sight with the Teflon defense of constructivism, reconstructing the hoax papers that exposed their
lack of academic reason and rigor as an unethical, poorly conducted experiment, and a mean one at
that. Where was the control? Where was the ethical approval? The critics of the critique claim it only

exposes a more general problem with peer review. The oddest defense? Maybe some of their
papers actually made sense. There is overlap between all of these and most, if not all, are examples
of whataboutism.
Criticism of the work as lacking experimental control
The study is not a good experiment because it was not an experiment. In the same vein, a criticism
of their critique is a poorly conducted experiment -- where are the other critiques to act as control?
Even by PBL’s description (reflexive ethnography), their work could not be considered as having an
experimental design. But I would argue it wasn’t even academic research, it was investigative
journalism or academic policing. Just as a sting operation to expose a criminal gang would involve
hours of preparation of backstories and character work, so that a journalist or agent could pose as a
member of the gang, PBL put hours of work into the submitted papers. The articles were the
undercover agents. Dressing the articles in the target journals’ ideology provided the disguise. There
was no pressure for the journals to accept the bogus papers so if you follow the analogy here,
entrapment is not a defense. The expectation that the study (or investigation) should have included
a control is a standard of evidence not demanded of other claims.
Critic: ‘You lie’
Liar: ‘Who doesn’t?’
This is a red herring, a specific form of ad hominem. To be clear, their work has exposed an issue in
absolute not relative standards. The claims are specific to these disciplines, not comparative across
disciplines. Asking for a control or a comparison is whataboutery. Their exposé not extending to
other academic fields should not be an impediment to dealing with the lack of standards, i.e. the
corruption of scholarship in these disciplines. When government corruption is reported on, a
comparison table is not necessary to bolster the evidence. When financial corruption is exposed in
business, people tend not to ask to see another industry as ‘control’. When moral corruption is
exposed in the Catholic Church, people do not to ask ‘relative to what?’ When academic corruption
has been shown in the past, the defense is never ‘did you check other researchers?’
Criticism of the work as research and the research as lacking ethical approval
If you position any of the above exposures of corruption, or any piece of investigative journalism as
research then you have many points of attack. You can criticize the breadth and depth of the
literature review that informs the research; the rationale for and clarity of the hypotheses; the
appropriateness of the research design; the representativeness and size of the sample; the
psychometric properties of the measures used etc. You can do a lot to ignore the point. After an
investigation, Boghossian’s own institution, Portland State University (PSU) have determined the
work constituted human-subjects research. The requirement for prior ethical approval from an
institutional review board (IRB) rests on whether the work qualifies as research, based on the federal
definition (known as the Common Rule). This definition of research that requires IRB approval raises
serious questions about academic freedom and freedom of speech and the press. The countless
Twitter polls conducted by academics online seem to meet the criteria for needing prior IRB
approval under the Common Rule i.e. Research means a systematic investigation, including research
development, testing, and evaluation, designed to develop or contribute to generalizable
knowledge. These polls ‘obtain information’ from ‘living individuals’ and ‘publish the results’.
Whether results are published / intended to be published is a common interpretation of what
constitutes ‘generalizable knowledge’. Interestingly, scholarly and journalistic activities are deemed
not to be research under the Common Rule. This is unlikely to excuse Twitter polls, but it should

exempt SokalSquared from the need for IRB oversight. As an exposé it is best considered a
journalistic endeavour. I find it shameful to throw the (ethics) book at whistle-blowers. PSU’s
reasoning on this case will be interesting to see in full, as the criteria they are attempting to hang
Boghossian with should also condemn the university itself for conducting human-subjects research
without, I assume, prior IRB approval. Here is a PSU Twitter poll. It obtains information, from living
individuals and its results are published.
This new development, which brings academic oversight and IRB approval into the scandal, invites a
serious discussion about what constitutes human-subjects research
As Columbia Law Professor Phillip Hamburger warned in 2007, ‘when a mere colloquy gets
characterized as “human subjects research,” or when a mere reading project or written survey
becomes an “experiment,” does the researcher really become a nascent predator, and his “human
subjects,” potential victims? This tendency to generalize about researchers and human subjects is
profoundly dangerous, for it lends credence to laws that treat researchers and subjects as incapable
of exercising individual authority. Researchers come to seem incapable of responsibility toward
others, and their subjects come to seem incapable of responsibility for themselves—thus making it
necessary for IRBs to license the researchers in their curiosity and the subjects in their consent. For
example, the commission that established the foundations for what became the Common Rule
argued that “investigators” must be deprived of their individual authority on account of their
curiosity’ (p. 474)
PBL have most consistently referred to their work as an audit so I will use that term going forward
although I see it more as an act of whistle-blowing. The gold standard of whistle-blowing is to
attempt to resolve things internally first and if that is not possible to expose the corruption by going
public. PBL have been accused of a lack of genuine engagement with scholars from these disciplines.
They have tried to engage. They’ve tried to generate conversations (see here). They invited debate
and discussion (see here, here, here, and their Twitter histories). If the audit is to be seen as
traditional research, then ethical approval should have been obtained. It is an important point but it
does not change the outcomes. If the study is an act of investigative journalism, a practice of
exposing wrongdoing, then perhaps some ethical concerns remain but IRB approval was likely
unnecessary. Would IRB approval be necessary for a food safety inspector checking standards in a
selection of restaurants, or a food critic writing on their experiences. Would IRB approval be
necessary for a teaching inspector checking the teaching quality across courses or modules, or a
mystery shopper reporting on the customer service received at branch outlets? Is the movie review
site Rotten Tomatoes in need of ethical approval? Bret Weinstein’s tweet was prescient. Any way
the study is positioned, the ideological blinders should not be ignored. The blind leading the blind is
not a good look for academia.
Criticism of the work as simply revealing a general problem with peer review
The concerns that SokalSquared undermines academia tend to spread the muck, and do the
undermining themselves. They equate these disciplines with other academic disciplines; the criticism
of the individual dancer becomes framed as an attempt to undermine dancing. Presenting the
findings of this audit as an example of a flaw in peer-review more generally, conflates issues and
generalizes the specific concerns of PBL about the quality of scholarship in these areas to academia.
After performing this hasty generalization themselves, critics then attribute the ulterior motive of
delegitimizing academia to PBL. This is not to ignore general issues with peer-review and the
‘publish or perish’ pressures of academia. There are issues of precarity and workload that likely

impact quality of academic work and the quality of the review process. And there are many
examples of bad research getting past reviewers.
However, there is a difference here that needs
attention. The expectation of honesty and
integrity is a part of the review process. If I am
making an empirical claim, the data I use to
ground that claim, if faked, can be used as a
Trojan horse to get it past quality control. And
technical aspects of statistical analyses in
quantitative research can bamboozle timepoor
reviewers and errors can go unnoticed.
But in order for my claim to be warranted
based on my data, the line of reasoning that
links the two should be sufficiently logical and
coherent. Poor data and poor argumentation
are two different things. Dishonesty can hide
poor (or fake) data, and compelling data can sometimes disguise poor reasoning. But dishonesty
cannot disguise reasoning directly as the reasoning should be laid bare in the article. Dishonesty is
not the Trojan horse that is exposed in SokalSquared. Everyone agrees dishonesty hurts the quality
and reputation of academic work (Note: dishonesty is different from deception – i.e. goals matter).
Ideology as a Trojan horse
Toulmin’s model of argumentation. Reproduced from Wikipedia.
The Trojan horse here is the ideological bias of the journals (and the grievance studies disciplines
they represent). These were not obscure journals; they were in some cases, the leading journal for
that academic area. The purposely broken reasoning of the PBL articles was disguised with the
‘correct’ ideology. It is belief bias in action (see also motivated reasoning and confirmation bias). The
sting highlights the need for greater institutionalized disconfirmation and viewpoint diversity. So if
there is a prevalent mechanism that will disguise (or cloud) shoddy reasoning, while this does not
prevent well-reasoned arguments being made, it certainly limits the ability of scholars in these areas
to tell good from bad.
The point of the audit was not to suggest all of the articles from these disciplines are garbage but to
highlight the detrimental impact ideological blinders have on identifying poor methodology and
argumentation. And in these disciplines ideology is pervasive to the point of being an orthodoxy. The
difference is dishonesty and data fakery (a Trojan horse in and beyond the social sciences) is not
orthodox behavior. It is not promoted, it is actively exposed. Exposure of that Trojan horse is not
met with whataboutery. To expand the problems identified in SokalSquared to all peer-review is
false equivalence.
The audit suggests the lack of a ‘logic’ barometer in grievance studies (to differentiate A and C in the
graphic below), but this does not mean reasonable scholarship has not been done (A and B).
However, while good scholarship is sensitive to a difference across rows, good propaganda is
sensitive to a difference across columns.

Reasoning
Good
Bad
Correct
A
C
Ideology
Incorrect
B
D
Graphic representing the correct
orientation of scholarship focused on
distinguishing value of an argument based
on reasoning (rows) vs a corruption of
scholarship focused on distinguishing
value of an argument based on ideology
(columns)
To best steel man these disciplines, skeptical academics should scrutinize the best examples of
scholarship nominated by experts in these fields (A). Defenders of these disciplines could also post
examples of counter-ideology narratives that have been published and engaged with (B). The
original rejections of PBL nonsense papers that were not disguised with ideology (D) seemed to be
correctly rejected. While the original rejections presented ideologically-neutral nonsense and not
‘incorrect ideology’, the rejections would suggest that it was the correct ideology that provided the
disguise and that a paper draped with incorrect ideology would also be rejected. So not including
‘right-wing papers’, as a control, is not a major limitation to SokalSquared. Referee bias in
manuscript review is not a new finding. Ideological bias likely affects disciplines beyond the scope of
SokalSquared. Critics of the audit can diffuse responsibility that way if they like. I was one of the first
international members of the Heterodox Academy because I feel the evidence and arguments are
compelling, so I recognize the general problem with ideological bias in academia – are these critics
admitting as much now? But where is the greatest pushback against ideological diversity and open
dialogue coming from?
Criticism of the work as mean spirited
Open dialogue and academic freedom are important components for good scholarship. Civility is
also a net contributor to any academic dialogue. Some feel this audit was mean in its deceptive
nature, and mean in its focus on and framing of the basket of disciplines labelled grievance studies.
It is not about the disciplines. You do not need to feel aggrieved if you are a scholar in these
disciplines. It is a critique of methodology. Finding fault in methods with the aim of improvement
gives those reliant on those methods the opportunity to counter the criticism or to take it on board
and adapt. It is not mean but a vital stress test for any academic discipline. It is far meaner to
question motives and insinuate ill intent. ‘You are doing bad research and here is why’ (i.e. criticizing
behavior) is not as mean as ‘You are a bad person for asking this question’ (i.e. criticizing
character/intent). Some have chosen to respond to SokalSquared by employing the latter tactic.
Conclusion
Finally, to disabuse any readers of the belief that I want to aid the Right in attacking academia, I am a
vocal defender of public education and the HE sector, I am left wing, I am an academic, a social
scientist. I recognise the ideological and methodological problems in my own discipline (psychology),
and I try to engage in conversations to improve the empirical and humanistic elements of our
research and practice. I see psychology straddling science and the Humanities, and I want it to stay
there. I value good scholarship. I believe good research and scholarship has been, is being and can
be conducted on questions of identity and culture. Like PBL, I ‘think studying topics like gender, race
and sexuality is worthwhile and getting it right is extremely important'. A postmodernist lens can be
useful. We need to be more concerned about whether ‘applied’ postmodernism, as an approach on
deconstructing narratives, gets it right, or just spins the right narrative.
Crucially, more interdisciplinary conversations are needed.