Within Hyman Review
What Would Count as Real Replication?
For Hyman, real replication meant changing the lab, judges, targets, procedures, and viewer pool without losing the effect.
On this page
- Why internal repetition was not enough
- How independent labs would change the test
- What success across new conditions would mean
Page outline Jump by section
Introduction
For Ray Hyman, the central replication question in remote-viewing research was not whether one laboratory could repeatedly obtain statistically unusual results. It was whether those results would survive when the entire research environment changed. A convincing demonstration would require different laboratories, different investigators, different judges, different target collections, different participants and independently developed procedures to produce the same effect. Until that happened, Hyman argued, it remained difficult to distinguish a genuine phenomenon from subtle features of one research programme.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
This standard became one of the defining disagreements in the 1995 review of the U.S. government’s remote-viewing programme. While Hyman acknowledged that the later Science Applications International Corporation (SAIC) experiments were better controlled than much of the earlier work, he maintained that repeated success within the same research lineage did not amount to the kind of independent replication normally expected before an extraordinary scientific claim is accepted.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
Why Internal Repetition Was Not Enough
One of Hyman’s most important methodological arguments was that repeated experiments can appear more convincing than they really are if they continue to share the same underlying ingredients.
In his review, he noted that the principal SAIC studies were conducted:
- in the same laboratory;
- with essentially the same experimental protocol;
- using many of the same experienced remote viewers;
- drawing from the same target collections;
- under the direction of the same investigators.
Rather than treating this consistency as a strength, Hyman argued that it could preserve any unnoticed weakness built into the research system. If an unidentified source of bias existed, repeating the same design would reproduce that bias just as effectively as it would reproduce a genuine phenomenon.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
This reflects a standard principle throughout experimental science: replication is strongest when it tests whether a finding survives changes that ought to eliminate laboratory-specific influences.
The Judge Was Part of the Experiment
Hyman regarded judging as especially important because remote-viewing experiments typically involve matching open-ended descriptions against possible targets.
In the SAIC free-response studies, Edwin May frequently served as the sole judge. May argued that his familiarity with individual viewers helped him interpret recurring personal styles and symbols. Hyman accepted that this rationale might improve matching accuracy, but argued that it created a different scientific problem.
If successful scoring depends on one person’s accumulated knowledge of particular viewers, then the evidence cannot easily be separated from that individual’s expertise. Scientific claims become much stronger when different judges—working independently and without privileged familiarity—obtain comparable results. Hyman therefore argued that judges should ideally be blind not only to the correct target but also to the identity of the viewer.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
In other words, the question was no longer simply whether one experienced evaluator could recognise successful descriptions, but whether many independent evaluators could do so under equally controlled conditions.
How Independent Laboratories Would Change the Test
Hyman’s preferred standard effectively required remote viewing to leave its original institutional environment.
A persuasive replication programme would include changes such as:
- new laboratories with no connection to the original research team;
- independent experimenters designing and conducting the studies;
- new pools of participants rather than repeatedly using experienced viewers;
- fresh target sets unknown to previous investigators;
- independent judges applying pre-specified scoring rules;
- public methods allowing criticism before conclusions became established.
Each of these changes removes one potential explanation based on laboratory culture, experimenter expectations or accumulated procedural habits.
The more independent variables that change while the reported effect remains stable, the harder it becomes to argue that success depends upon a particular group of researchers rather than upon the claimed phenomenon itself. This logic is widely used across experimental psychology and biomedical research, where multi-site replication is generally regarded as stronger evidence than repeated success within a single laboratory.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
Why Secrecy Made Replication Harder
Hyman also believed that the classified nature of much of the Stargate programme delayed genuine scientific evaluation.
Because many experiments remained secret for years, outside researchers could not inspect protocols, identify weaknesses or attempt independent replications while the work was still developing. Hyman argued that this lack of open criticism prevented the normal process by which experimental methods improve through external challenge.
He therefore regarded secrecy as more than a historical inconvenience. It meant that the research programme had fewer opportunities to discover hidden methodological problems before large bodies of apparently consistent evidence accumulated.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
What Success Across New Conditions Would Mean
Importantly, Hyman did not argue that independent replication would merely increase confidence statistically. He viewed it as changing the interpretation of the evidence.
If remote viewing produced comparable results despite changes in:
- laboratory,
- personnel,
- judging,
- participant population,
- target materials,
- and experimental implementation,
then many ordinary methodological explanations would become progressively less plausible.
Such findings would not automatically identify the mechanism involved, but they would substantially strengthen the claim that the observed effect reflected something more general than the practices of one research group. This distinction explains why Hyman repeatedly separated the existence of statistically significant results from proof of psychic functioning.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
What Happened Afterwards
The debate over independent replication did not end with the AIR review. Later discussions of remote-viewing evidence continued to return to the same issue: whether positive findings had travelled successfully beyond the original investigators and institutions.
Critics argued that the strongest reported effects remained concentrated within closely related research programmes and that later reassessments continued to question whether apparently independent studies had achieved sufficient methodological separation. Supporters pointed to broader parapsychology literature and meta-analytic evidence, but disagreement persisted over whether those studies met the level of independence Hyman considered necessary.[Wikipedia+2Wikipedia]WikipediaStargate Project (U.S. Army unitStargate Project (U.S. Army unit
As a result, Hyman’s replication criterion became one of the lasting legacies of the 1995 review. The key issue was never simply whether remote-viewing experiments could be repeated, but whether they could be repeated after removing the people, procedures and institutional continuity that originally produced the reported effects.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H…
Amazon book picks
Further Reading
Books and field guides related to What Would Count as Real Replication?. Use these as the next step if you want deeper reading beyond the article.
The seven deadly sins of psychology
First published 2017. Subjects: Methodology, Psychology, Research, Psychology, research, Psychology, methodology.
Statistics Done Wrong
First published 2015. Subjects: Missing observations (Statistics), Methodology, Statistics.
Research Design
First published 1994. Subjects: Statistical methods, Research, Methodology, Social sciences, Social sciences - research - methodology.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
1.
Source: Wikipedia
Title: Ray Hyman
Link:https://en.wikipedia.org/wiki/Ray_Hyman
2.
Source: Wikipedia
Title: Stargate Project (U.S. Army unit)
Link:https://en.wikipedia.org/wiki/Stargate_Project_%28U.S._Army_unit%29
3.
Source: Wikipedia
Title: Parapsychology research at SRI
Link:https://en.wikipedia.org/wiki/Parapsychology_research_at_SRI
4.
Source: ia600501.us.archive.org
Link:https://ia600501.us.archive.org/20/items/FritjofCapraTheTurningPoint/Dean%20Radin%20-%20The%20Conscious%20Universe%20-%20The%20Scientific%20Truth%20of%20Psychic%20Phenomena%20%5BOCR%5D.pdf
Source snippet
CONSCIOUS UNIVERSEIn 1995, the CIA commissioned a review of the government-sponsored remote-viewing research. Dr. Ray Hyman (whom we have...
5.
Source: youtube.com
Title: Remote Viewing and Statistical Validation
Link:https://www.youtube.com/watch?v=YrwAiU2g5RU
Source snippet
Ray Hyman - The Life of an Expert Skeptic, Part 1...
6.
Source: youtube.com
Title: Ray Hyman
Link:https://www.youtube.com/watch?v=-8VAOzcXGc4
Source snippet
Lecture 10: Independent Replicability, [Meta-analysis]({{ 'meta-analysis/' | relative_url }}) and Pseudo-Replicability...
7.
Source: nsarchive2.gwu.edu
Link:https://nsarchive2.gwu.edu/NSAEBB/NSAEBB438/docs/doc_57.pdf
Source snippet
National Security ArchiveAn Evaluation of Remote Viewing: Research and...March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Raymond H...
Published: March 13, 2015
8.
Source: scribd.com
Title: Debating Psychic Experiences
Link:https://www.scribd.com/document/94892851/Debating-Psychic-Experiences
Source snippet
PDF | ParapsychologyIn 1999 Wiseman and Julie Milton challenged Honortons claims of replication with the autoganzfeld. Hyman and Honortons...
Additional References
9.
Source: skepticalinquirer.org
Link:https://skepticalinquirer.org/wp-content/uploads/sites/29/2019/03/Issue-02-23.pdf
Source snippet
MID parapsychologyThe CIA recently released details of more than twenty years of research into remote viewing and a new debate erupted ov...
10.
Source: researchgate.net
Title: remote viewing pilot study using a [ganzfeld]({{ ‘ganzfeld/’ | relative_url }}) induction technique
Link:https://www.researchgate.net/publication/287831983_Alien_visitation_extra-terrestrial_life_and_paranormal_beliefs
Source snippet
Alien visitation, extra-terrestrial life, and paranormal beliefsRay Hyman argued that the initial rate of successful replication may have...
11.
Source: newdualism.org
Title: Consciousness and the Physical World by Douglas M
Link:https://www.newdualism.org/papers/D.Stokes/ConsciousnessandthePhysicalWorld-book.htm
Source snippet
StokesIt draws conclusions about the self and its relations to the physical body and the physical world that the reader may find unorthod...
12.
Source: pmc.ncbi.nlm.nih.gov
Title: PMCFollow‐up on the U.S
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10275521/
Source snippet
Central Intelligence Agency's (CIA...by Á Escolà‐Gascón · 2023 · Cited by 10 — the U.S. Central Intelligence Agency (CIA) commissioned s...
13.
Source: dokumen.pub
Link:https://dokumen.pub/transcendent-mind-rethinking-the-science-of-consciousness-9781433822773-1433822776-9781433822780-1433822784.html
Source snippet
ussion at the cutting edge of consciousness research.Read more...
14.
Source: youtube.com
Title: Lecture 10: Independent Replicability, Meta-analysis and Pseudo-Replicability
Link:https://www.youtube.com/watch?v=arxJ8huoXy0
Source snippet
The Replication Crisis: Why Studies Fail and How Psychology Is Fixing Itself...
15.
Source: youtube.com
Title: The Replication Crisis: Why Studies Fail and How Psychology Is Fixing Itself
Link:https://www.youtube.com/watch?v=YpizifCQ96Y
Source snippet
Questioning Psychology's Findings: A Real Crisis...
16.
Source: philpapers.org
Title: BULPEL 2
Link:https://philpapers.org/archive/BULPEL-2.pdf
Source snippet
Utts and Hyman were given access to the same data—the declassified SAIC reports and a selection of peer-reviewed...
17.
Source: youtube.com
Title: Questioning Psychology’s Findings: A Real Crisis
Link:https://www.youtube.com/watch?v=FXIFtWH0GQM
Topic Tree