Within SAIC Tests
When One Lab Setup Does Too Much Work
The strongest SAIC results raised a hard question: did the effect come from remote viewing, or from a repeated experimental setup?
On this page
- The repeated viewers, judge, targets, and scoring
- Why repetition can mimic robustness
- What independent replication would need to change
Page outline Jump by section
Introduction
The strongest results from the Science Applications International Corporation (SAIC) remote-viewing experiments created a difficult interpretive problem that went beyond simple questions of statistical significance. Even critics such as psychologist Ray Hyman acknowledged that the later studies had addressed many of the obvious flaws found in earlier work. However, he argued that the central positive findings still came from a remarkably consistent experimental ecosystem: many of the same viewers, the same principal researchers, similar target pools, the same judging approach, and closely related scoring methods. That raised the possibility of monomethod bias—the risk that an apparent effect reflects one stable experimental system rather than a phenomenon that survives changes in people, procedures and settings.[UC Irvine Bren School]ics.uci.eduUC Irvine Bren SchoolEvaluation of Program on Anomalous Mental PhenomenaIn evaluating the SAIC research, Utts points to the consistency o…
Within the debate over remote viewing, this became one of the most important methodological questions. If an anomaly only appears when a particular laboratory repeatedly uses the same combination of participants and procedures, it becomes difficult to determine whether researchers have isolated a genuine psychological phenomenon or merely identified a method that consistently produces unusual statistical results.
The Repeated Viewers, Judge, Targets and Scoring
The SAIC programme did not rely on a constantly changing pool of participants. Instead, many of its strongest results came from a small number of experienced viewers who had participated in remote-viewing research for years. Supporters regarded this as sensible because expertise might improve performance. Critics argued that concentrating evidence in a handful of highly practised participants reduced confidence that the findings would generalise beyond that specific group.[UC Irvine Bren School]ics.uci.eduUC Irvine Bren SchoolEvaluation of Program on Anomalous Mental PhenomenaIn evaluating the SAIC research, Utts points to the consistency o…
The concern extended beyond the viewers themselves.
Several elements remained relatively stable across the strongest studies:
- experienced remote viewers who had worked within the programme for long periods;
- similar target-selection procedures;
- free-response descriptions rather than forced-choice tasks;
- rank-order judging in which a judge compared one transcript with several candidate targets;
- closely related statistical analyses from one experiment to the next.[CIA]cia.govAN EVALUATION OF REMOTE VIEWING: RESEARCH…A remote viewer is asked to visualize a place, location, or object being viewed by a "bea…
None of these features is inherently flawed. Scientific research often benefits from standardised methods. The difficulty arises when nearly every important component remains constant simultaneously. If positive findings continue under those conditions, it becomes unclear which part of the system is responsible.
A further issue noted by Hyman was the concentration of responsibility within the research programme. In some SAIC work, the principal investigator also played important roles in judging or methodological development. Even where formal blinding was maintained, combining multiple responsibilities increased concern about unconscious expectations influencing design choices or interpretation. Critics viewed this as another reason to seek replication by independent laboratories using different personnel.[UC Irvine Bren School]ics.uci.eduUC Irvine Bren SchoolEvaluation of Program on Anomalous Mental PhenomenaIn evaluating the SAIC research, Utts points to the consistency o…
Why Repetition Can Mimic Robustness
Monomethod bias does not imply fraud or careless experimentation. Instead, it describes a situation in which repeated use of one successful methodology creates an appearance of broad confirmation even though the evidence comes from variations of essentially the same experiment.
This matters because laboratory findings can become statistically impressive without demonstrating broad reproducibility.
For example, repeated success could theoretically arise from:
- subtle characteristics of one target collection;
- consistent judging preferences developed over years;
- interactions between highly experienced viewers and familiar procedures;
- analytical decisions that are individually reasonable but collectively favour one outcome;
- unnoticed laboratory-specific influences that disappear elsewhere.
None of these possibilities requires deliberate misconduct. They simply illustrate why methodological independence is valued so highly in experimental science.
Hyman accepted that the SAIC statistics deserved attention but argued that statistical significance alone could not distinguish between a genuine anomalous process and a persistent feature of one laboratory system. His central objection was therefore interpretive rather than mathematical. The reported effect sizes could be real while their underlying cause remained uncertain.[UC Irvine Bren School]ics.uci.eduUC Irvine Bren SchoolEvaluation of Program on Anomalous Mental PhenomenaIn evaluating the SAIC research, Utts points to the consistency o…
Jessica Utts took a different view. She argued that the consistency of statistically significant results across related studies, together with improved controls against sensory leakage, made ordinary methodological explanations increasingly implausible. In her assessment, the remaining task was to understand the mechanism rather than to establish whether an anomaly existed at all.[UC Davis]ucdavis.eduUC Davis'Psychic Spying' Research Produces Credible EvidenceNovember 28, 1995 — 28 Nov 1995 — Remote viewing has been conceptually replicated across a number of laboratories, she says, helping to r…
The disagreement therefore centred less on whether unusual statistics existed than on how much confidence should be placed in results generated within one largely consistent research framework.
Why Independent Replication Would Need to Change More Than One Variable
The monomethod critique implies that successful replication requires more than repeating the same protocol with new data. To demonstrate that an effect is independent of one laboratory system, researchers would ideally vary several components at once while preserving the underlying hypothesis.
Meaningful independent replication would include changes such as:
- different laboratories with no organisational connection to the original investigators;
- new experimenters designing and running the studies;
- independent judges unfamiliar with previous outcomes;
- fresh target databases assembled by unrelated researchers;
- different participant pools rather than the same expert viewers;
- preregistered statistical analyses specified before data collection.
Changing only one feature—for example, recruiting new viewers while retaining the same judging system and investigators—would leave much of the original methodological network intact.
This emphasis on independent variation reflects a general principle of experimental science. Confidence grows most rapidly when an effect survives changes in personnel, equipment, analytical approaches and institutional setting, because each successful replication rules out another class of laboratory-specific explanation.
Why the Issue Remains Central to the SAIC Debate
The monomethod argument helps explain why the SAIC experiments continue to divide commentators despite widespread agreement about some basic facts.
Both Utts and Hyman accepted that the strongest SAIC studies produced statistically unusual results. Where they diverged was in deciding what those results justified. Utts interpreted the persistence of the findings as evidence for an unexplained information-acquisition process. Hyman argued that the evidence had not escaped dependence on one closely connected research tradition and therefore had not crossed the threshold expected for extraordinary scientific claims.[UC Irvine Bren School]ics.uci.eduUC Irvine Bren SchoolEvaluation of Program on Anomalous Mental PhenomenaIn evaluating the SAIC research, Utts points to the consistency o…
Subsequent supporters have pointed to broader remote-viewing literature and later meta-analyses as evidence that positive effects extend beyond SAIC, while critics have continued to argue that the most influential findings still rely heavily on related methods, overlapping researchers or insufficiently independent replications.[ResearchGate+2PMC]researchgate.net369604750 Remote Viewing a 1974 2022 systematic review and meta analysisResearchGate(PDF) Remote Viewing: a 1974-2022 systematic review…March 20, 2023 — This is the first meta-analysis of all studies relate…
For that reason, monomethod bias remains one of the most persistent interpretive challenges surrounding the strongest SAIC findings. The question is not whether the laboratory produced statistically interesting data, but whether those data demonstrate a phenomenon that survives once the familiar viewers, judges, procedures and institutional context are replaced.
Amazon book picks
Further Reading
Books and field guides related to When One Lab Setup Does Too Much Work. Use these as the next step if you want deeper reading beyond the article.
The conscious universe
First published 1997. Subjects: Parapsychology, Case studies, Cas, Études de, Paranormale verschijnselen, Parapsychologie.
Research Design
First published 1994. Subjects: Statistical methods, Research, Methodology, Social sciences, Social sciences - research - methodology.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
1.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180005-5.pdf
Source snippet
sor of. Statistics at the University of California/Davis, and Dr.Read more...
2.
Source: cia.gov
Link:https://www.cia.gov/readingroom/document/cia-rdp96-00791r000200180006-4
Source snippet
AN EVALUATION OF REMOTE VIEWING: RESEARCH...A remote viewer is asked to visualize a place, location, or object being viewed by a "bea...
3.
Source: researchgate.net
Title: 369604750 Remote Viewing a 1974 2022 systematic review and [meta analysis]({{ ‘meta-analysis/’ | relative_url }})
Link:https://www.researchgate.net/publication/369604750_Remote_Viewing_a_1974-2022_systematic_review_and_meta-analysis
Source snippet
ResearchGate(PDF) Remote Viewing: a 1974-2022 systematic review...March 20, 2023 — This is the first meta-analysis of all studies relate...
Published: March 20, 2023
4.
Source: pmc.ncbi.nlm.nih.gov
Title: PMCFollow‐up on the U.S
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10275521/
Source snippet
Central Intelligence Agency's (CIA...by Á Escolà‐Gascón · 2023 · Cited by 10 — We offer new statistical evidence suggesting a new approa...
5.
Source: ics.uci.edu
Link:https://www.ics.uci.edu/~jutts/hyman.html
Source snippet
UC Irvine Bren SchoolEvaluation of Program on Anomalous Mental PhenomenaIn evaluating the SAIC research, Utts points to the consistency o...
6.
Source: Wikipedia
Title: Remote viewing
Link:https://en.wikipedia.org/wiki/Remote_viewing
Source snippet
Remote viewingReviewers included Ray Hyman and Jessica Utts. Utts maintained that there had been a statistically significant positive...
7.
Source: ucdavis.edu
Title: UC Davis’[Psychic Spying]({{ ‘psychic-spying/’ | relative_url }})’ Research Produces Credible Evidence
Link:https://www.ucdavis.edu/news/psychic-spying-research-produces-credible-evidence
Source snippet
November 28, 1995 — 28 Nov 1995 — Remote viewing has been conceptually replicated across a number of laboratories, she says, helping to r...
Published: November 28, 1995
Additional References
8.
Source: psi-encyclopedia.spr.ac.uk
Title: Psi Encyclopedia Remote Viewing
Link:https://psi-encyclopedia.spr.ac.uk/articles/remote-viewing/
Source snippet
Psi EncyclopediaRemote Viewing - Psi Encyclopedia13 Jan 2017 — Jessica Utts judged the evidence persuasive, and even Ray Hyman conceded m...
9.
Source: youtube.com
Link:http://www.youtube.com/watch?v=Ln47DanodE4
Source snippet
Jessica Utts Ray Hyman remote viewing debate IRVA 2006 Jessica Utts, Ph.D. - Remote Viewing: It Works, But HOW?...
10.
Source: koestlerunit.wordpress.com
Title: wiseman milton 1998
Link:https://koestlerunit.wordpress.com/wp-content/uploads/2015/06/wiseman-milton-1998.pdf
Source snippet
One of the SAIC Remote Viewing Programby R Wiseman · Cited by 24 — The panel included two reviewers chosen for their expertise in parapsy...
11.
Source: youtube.com
Link:http://www.youtube.com/watch?v=m33bEGmMOKc
Source snippet
Project [Stargate]({{ 'stargate/' | relative_url }}): The CIA's Secret Psychic Spies (1972-1995)...
12.
Source: youtube.com
Title: EXAMINING THE EVIDENCE FOR PSI PHENOMENA
Link:http://www.youtube.com/watch?v=UbXcCBPHUn0
Source snippet
The CIA's Psychic Spies: The Declassified History of Remote Viewing (Project STAR GATE)...
13.
Source: youtube.com
Link:http://www.youtube.com/watch?v=wpJTGrIL4xA
Source snippet
EXAMINING THE EVIDENCE FOR PSI PHENOMENA - Dean Radin PHD #40...
14.
Source: youtube.com
Title: Remote Viewing and Statistical Validation
Link:http://www.youtube.com/watch?v=YrwAiU2g5RU
Source snippet
IRVA 2006 Jessica Utts, Ph.D. - Remote Viewing: It Works, But HOW?...
Topic Tree