Within Utts Review
How Small Effects Became a Big Argument
Utts' case depended on repeated small shifts from chance, not dramatic hits in every session.
On this page
- What the reported effect sizes meant
- Why average performance mattered more than single hits
- How small effects shaped the controversy
Page outline Jump by section
Introduction
One of the most misunderstood aspects of Jessica Utts’ evaluation of the U.S. government’s remote-viewing research is that it did not depend on spectacular individual successes. Instead, her statistical argument rested on a much subtler claim: across many carefully controlled experiments, participants appeared to perform slightly better than chance often enough that the average deviation became statistically difficult to dismiss as random.[UC Irvine Bren School]ics.uci.eduIn these experiments, a viewer attempts to…Read more…
That distinction shaped the entire debate. Supporters argued that science often discovers genuine phenomena through small but repeatable effects rather than dramatic demonstrations. Critics accepted that some statistical analyses showed departures from chance but questioned whether those departures reflected paranormal perception rather than unnoticed methodological problems, bias, or weaknesses in experimental design. The controversy therefore centred less on whether a few impressive sessions existed than on what persistent, modest statistical effects actually meant.[National Security Archive]nsarchive2.gwu.edudoc 57National Security ArchiveAn Evaluation of Remote Viewing: Research and…by MD Mumford · 1995 · Cited by 75 — Professor Utts uses the ga…
What the reported effect sizes meant
An effect size measures how large an observed difference is, independent of sample size. Statistical significance answers one question—whether an effect is unlikely to have arisen by chance—while effect size addresses another: how large the effect actually is.
Utts repeatedly emphasised that the reported remote-viewing results were not enormous. In her review she characterised the findings as falling between what behavioural scientists typically describe as a small and medium effect. Such effects are rarely obvious in individual observations but can become detectable when many trials are combined. She argued that this pattern resembled numerous accepted findings in psychology and medicine, where individual outcomes are noisy but group averages consistently shift in one direction.[UC Irvine Bren School]ics.uci.eduIn these experiments, a viewer attempts to…Read more…
This point is often lost in popular descriptions of remote viewing. The statistical case did not require viewers to produce detailed, unmistakably correct descriptions every time. Instead, it proposed that correct targets appeared slightly more often than expected under chance when evaluated across many independent sessions using predefined scoring procedures.[UC Irvine Bren School]ics.uci.eduIn these experiments, a viewer attempts to…Read more…
Why average performance mattered more than single hits
The later SRI and SAIC experiments commonly used blind judging methods such as rank-order comparisons. Rather than asking whether one session looked astonishing, judges compared a viewer’s description with several possible targets and ranked the best match without knowing which target was correct.
Under this design, chance predicts a particular average ranking over many trials. If the correct target repeatedly receives better-than-expected rankings, even by a modest margin, the cumulative statistics may become significant despite considerable variability from one session to the next. Utts argued that this aggregate performance, not isolated anecdotes, represented the appropriate scientific evidence.[CIA]cia.govAN EVALUATION OF REMOTE VIEWINGSubsequently, a judge evaluates this description by rank ordering the set of locations against the desc…
This approach reflects a common principle in experimental science:
- Individual observations can be inaccurate or ambiguous.
- Random variation produces both unusually good and unusually poor performances.
- Reliable conclusions depend on long-run averages rather than memorable examples.
- Replication across multiple experiments is generally more informative than any single dramatic result.
Consequently, Utts discouraged evaluating remote viewing as though it were a stage demonstration. In her view, expecting perfect accuracy in every session misunderstood how weak behavioural effects are normally investigated statistically.[UC Irvine Bren School]ics.uci.eduIn these experiments, a viewer attempts to…Read more…
How small effects shaped the controversy
Ironically, the modest size of the reported effects became both the strongest and weakest part of the case.
Supporters argued that precisely because the effects were small, rigorous statistical analysis was essential. They maintained that subtle but consistent departures from chance are common throughout science and should not be dismissed merely because they are not visually dramatic. Utts therefore viewed repeated replication under improved laboratory controls as stronger evidence than isolated spectacular claims.[UC Irvine Bren School]ics.uci.eduIn these experiments, a viewer attempts to…Read more…
Critics saw the same modest effect sizes differently. Ray Hyman accepted that some of the statistical outcomes were too consistent to ignore casually and agreed that later experiments had improved substantially over earlier work. However, he argued that small effects are especially vulnerable to hidden methodological influences. When the observed advantage is slight, even minor biases in judging, randomisation, experimental procedures or publication practices can create an apparent signal without requiring any paranormal explanation. For Hyman, statistical significance alone could not identify the cause of the effect.[National Security Archive]nsarchive2.gwu.edudoc 57National Security ArchiveAn Evaluation of Remote Viewing: Research and…by MD Mumford · 1995 · Cited by 75 — Professor Utts uses the ga…
The American Institutes for Research ultimately reflected this divide. The review acknowledged statistically significant laboratory findings while concluding that the evidence did not establish remote viewing as a reliable operational intelligence tool. The disagreement therefore shifted from whether unusual statistical results existed to how those results should be interpreted.[National Security Archive]nsarchive2.gwu.edudoc 57National Security ArchiveAn Evaluation of Remote Viewing: Research and…by MD Mumford · 1995 · Cited by 75 — Professor Utts uses the ga…
Why small effects remained scientifically important
The debate surrounding remote viewing illustrates a broader principle of scientific reasoning. Small, repeatable effects can represent genuine discoveries, but they also require unusually careful experimental control because they are easier to distort through subtle sources of error.
Utts regarded the reported effect sizes as sufficiently consistent to justify concluding that an anomalous phenomenon existed. Hyman regarded the same evidence as interesting but insufficient to eliminate conventional explanations. Their disagreement was therefore not primarily about arithmetic; it concerned the standard of evidence required before attributing a persistent statistical pattern to a previously unknown human ability.[UC Irvine Bren School]ics.uci.eduIn these experiments, a viewer attempts to…Read more…
For that reason, the most influential numbers in the remote-viewing literature were never spectacular accuracy rates or dramatic individual “hits”. They were modest average departures from chance accumulated across many experiments—small enough to demand sophisticated statistical analysis, yet large enough to fuel one of the longest-running disputes in parapsychology.[UC Irvine Bren School]ics.uci.eduIn these experiments, a viewer attempts to…Read more…
Amazon book picks
Further Reading
Books and field guides related to How Small Effects Became a Big Argument. Use these as the next step if you want deeper reading beyond the article.
The Men Who Stare At Goats
Rating: 3.5/5 from 11 Google Books ratings
Provides broader military context.
The Signal and the Noise
Provides context for interpreting weak but repeatable evidence.
The cult of statistical significance
First published 2008. Subjects: Statistical methods, Statistical hypothesis testing, Social aspects, Economics, Social aspects of Statist...
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
1.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180005-5.pdf
Source snippet
AN EVALUATION OF THE REMOTE VIEWING PROGRAMTo evaluate the research program, a "blue ribbon" panel was assembled. The panel included t...
2.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180006-4.pdf
Source snippet
AN EVALUATION OF REMOTE VIEWINGSubsequently, a judge evaluates this description by rank ordering the set of locations against the desc...
3.
Source: ics.uci.edu
Link:https://www.ics.uci.edu/~jutts/air.pdf
Source snippet
In these experiments, a viewer attempts to...Read more...
4.
Source: nsarchive2.gwu.edu
Title: doc 57
Link:https://nsarchive2.gwu.edu/NSAEBB/NSAEBB438/docs/doc_57.pdf
Source snippet
National Security ArchiveAn Evaluation of Remote Viewing: Research and...by MD Mumford · 1995 · Cited by 75 — Professor Utts uses the ga...
5.
Source: Wikipedia
Title: Ray Hyman
Link:https://en.wikipedia.org/wiki/Ray_Hyman
6.
Source: scribd.com
Title: Jessica Utts
Link:https://www.scribd.com/document/78122493/Jessica-Utts-The-Significance-of-Statistics-in-Mind-Matter-Research
Source snippet
governments remote viewing program, wrote: Only parapsychology claims to be a science on the basis of...Read more...
7.
Source: psi-encyclopedia.spr.ac.uk
Title: spr.ac.uk Remote Viewing
Link:https://psi-encyclopedia.spr.ac.uk/articles/remote-viewing/
Source snippet
Viewing - Psi EncyclopediaJan 13, 2017 — Remote viewing replaced repetitive forced-choice tasks with free-response protocols designed to...
Additional References
8.
Source: researchgate.net
Link:https://www.researchgate.net/publication/333228024_An_Assessment_of_the_Evidence_for_Psychic_Functioning
Source snippet
An Assessment of the Evidence for Psychic FunctioningResearch on psychic functioning, conducted over a two-decade period, is examined to...
9.
Source: nectar.northampton.ac.uk
Link:https://nectar.northampton.ac.uk/id/eprint/12472/1/Krippner_etal_ELS_2018_Remote_viewing_of_concealed_target_pictures_under_light_and_dark_conditions.pdf
Source snippet
Hyman (1995) replied in agreement to Utts that certain RV effect sizes reported were “too large and consistent to be dismissed as...Re...
10.
Source: journals.lub.lu.se
Title: se [Anomalous Cognition]({{ ‘not-clairvoyance/’ | relative_url }}): An Umbrella Review of the Meta
Link:https://journals.lub.lu.se/jaex/article/download/23206/20893/58111
Source snippet
Cognition: An Umbrella Review of the Meta-...Abstract: Objective: The aim of this study was to assess the results of all meta-analy- ses...
11.
Source: koestlerunit.wordpress.com
Link:https://koestlerunit.wordpress.com/wp-content/uploads/2015/06/wiseman-milton-1998.pdf
Source snippet
One of the SAIC Remote Viewing Programby R Wiseman · Cited by 24 — The study chosen was Experiment One, which had obtained statistically...
12.
Source: ucdavis.edu
Title: psychic spying research produces credible evidence
Link:https://www.ucdavis.edu/news/psychic-spying-research-produces-credible-evidence
Source snippet
'Psychic Spying' Research Produces Credible Evidence28 Nov 1995 — Utts found the results were consistent with the small- to medium-sized...
13.
Source: youtube.com
Title: Training Anomalous Cognition with Edwin C. May
Link:https://www.youtube.com/watch?v=MAnEiItzEXk
Source snippet
This video exploring Statistics in Parapsychology with Jessica Utts provides essential context regarding how statistical significance and...
14.
Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10275521/
Source snippet
Central Intelligence Agency's (CIA...by Á Escolà‐Gascón · 2023 · Cited by 10 — What is remote viewing? RV is an experiential technique f...
15.
Source: ciaotest.cc.columbia.edu
Title: sa jan02srm01
Link:https://ciaotest.cc.columbia.edu/olj/sa/sa_jan02srm01.html
Source snippet
Remote Viewing: The US Sponsored Psychic...This paper deals with experiments conducted in USA in which certain individuals were trained...
16.
Source: youtube.com
Title: Feeling The Future: Is ESP real (according to science)?
Link:https://www.youtube.com/watch?v=qM1xv4a6Ls8
Source snippet
Training Anomalous Cognition with Edwin C. May...
17.
Source: youtube.com
Title: CONTROLLED REMOTE VIEWING
Link:https://www.youtube.com/watch?v=wWqldsr_zIg
Source snippet
Feeling The Future: Is ESP real (according to science)?...
Topic Tree