When One Method Can Fool a Whole Program

Introduction

Ray Hyman’s concern about monomethod bias in the Science Applications International Corporation (SAIC) remote-viewing programme was not that any single experiment was obviously flawed. Instead, he argued that a research programme can appear consistently successful simply because it repeatedly uses the same experimental system. If the same viewers, target-selection procedures, judging methods, scoring rules and research team are reused, any hidden bias within that system may also be repeated, creating the appearance of a stable phenomenon rather than revealing a genuine paranormal ability. This concern became an important part of Hyman’s broader argument that the SAIC findings, although statistically interesting, had not yet demonstrated remote viewing independently of the methods used to measure it.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — In the ea…Published: March 13, 2015

What monomethod bias means in remote viewing

In research methodology, monomethod bias refers to the risk that an apparent effect is tied to a particular measurement approach rather than to the underlying phenomenon being studied. The concept originates from broader work on construct validity by Campbell and Fiske, who argued that strong scientific claims require confirmation using different methods rather than repeated success with only one.[three-mode.leidenuniv.nl]three-mode.leidenuniv.nlAssessing Method Variance in Multitrait-MultimethodSeptember 14, 2001 — by RP Bagozzi · Cited by 889 — Campbell and Fiske's (1959) criterion for conver- gent validity is that the monotrait…Published: September 14, 2001

Applied to remote viewing, the concern is straightforward. Suppose one laboratory repeatedly obtains positive results using:

Even if every experiment reaches statistical significance, the common element across them is not necessarily remote viewing itself. It could instead be an unnoticed feature of the shared methodology.

Hyman argued that the SAIC experiments had become increasingly careful compared with earlier work at the Stanford Research Institute, but he questioned whether they had escaped this broader methodological dependence. A successful programme conducted almost entirely within one research culture cannot easily separate a genuine effect from characteristics unique to that laboratory.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — In the ea…Published: March 13, 2015

Why shared viewers, targets, judges and scoring mattered

The SAIC studies reviewed by the American Institutes for Research were largely conducted by the same research group over a relatively short period during the early 1990s. Hyman viewed this continuity as both a strength and a weakness.

It was a strength because procedures had become more disciplined and internally consistent. However, it also meant that many parts of the experimental system remained connected.

Several recurring features were especially important.

Experienced viewers. Many experiments relied on a limited number of participants who had worked extensively with the researchers. Their familiarity with laboratory routines could improve performance for ordinary psychological reasons, but it also made it harder to determine whether success would generalise to new participants.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — In the ea…Published: March 13, 2015

Target pools. Although targets were randomly assigned, they often came from related collections developed within the same research programme. If target characteristics interacted with judging procedures in subtle ways, those interactions could persist across multiple studies.

Judging procedures. Remote-viewing experiments typically depend on human judges comparing a viewer’s description with several possible targets. Even under blind conditions, this process involves interpretation rather than simple right-or-wrong scoring. Hyman repeatedly noted that subjective judging creates opportunities for systematic preferences that may remain invisible if the same judging framework is reused.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — In the ea…Published: March 13, 2015

Scoring rules. Once a particular scoring approach becomes standard within one laboratory, any unnoticed assumptions built into that approach also become standard. Consistent positive outcomes therefore demonstrate consistency within that scoring system but not necessarily robustness across alternative methods.

None of these factors automatically invalidate the experiments. Hyman’s point was that they could all produce the same pattern of apparent success without proving that remote viewing itself had been isolated.

Why consistency inside one laboratory is not enough

One of Hyman’s central arguments was that repeated success inside a single programme is not equivalent to independent replication.

Independent replication means more than repeating an experimental protocol. It requires that different laboratories, different investigators and ideally different participant pools obtain comparable results without relying on the same research traditions or assumptions.

From this perspective, the SAIC programme resembled a highly refined but largely self-contained experimental ecosystem. Every successful experiment increased confidence that the laboratory could reproduce its own findings, but it did not necessarily show that the phenomenon would survive outside that environment.

This distinction explains why Hyman did not dismiss the statistical evidence while still refusing to accept it as proof of psychic functioning. Statistical significance answers whether results are unlikely under a specified chance model. It does not automatically identify the source of those results.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — In the ea…Published: March 13, 2015

How method variation could test the pattern

Hyman’s concern naturally suggested ways of testing whether the reported effects reflected remote viewing or the experimental system itself.

Useful tests would include:

If positive results continued across these different methods, the case against monomethod bias would become substantially stronger because the common factor would increasingly be the alleged phenomenon rather than the research procedure.

This reasoning reflects a standard principle across behavioural science: claims become more convincing when they survive changes in measurement, personnel and setting rather than only repeated application of one successful protocol.[three-mode.leidenuniv.nl+2conjointly.com]three-mode.leidenuniv.nlAssessing Method Variance in Multitrait-MultimethodSeptember 14, 2001 — by RP Bagozzi · Cited by 889 — Campbell and Fiske's (1959) criterion for conver- gent validity is that the monotrait…Published: September 14, 2001

Why the issue remained important

Monomethod bias remains one of the more nuanced criticisms of the SAIC remote-viewing experiments because it does not depend on identifying a single procedural mistake. Instead, it questions whether an entire sequence of apparently successful studies may have inherited the same hidden strengths and weaknesses.

Later commentators continued to revisit this issue. Some argued that the SAIC programme represented a methodological improvement over earlier remote-viewing work, while others maintained that independent reproduction and greater methodological diversity were still lacking. Reanalyses of individual SAIC experiments likewise focused on whether specific design features or information leakage could account for reported effects, reinforcing Hyman’s broader concern that results should be tested under genuinely independent conditions before being treated as evidence for paranormal functioning. researchprofiles.herts.ac.uk+2researchprofiles.herts.ac.uk[researchprofiles.herts.ac.uk]researchprofiles.herts.ac.ukexperiment one of the saic remote viewing program a critical re eExperiment One of the SAIC Remote Viewing Programby R Wiseman · 1999 · Cited by 24 — This paper first outlines xx a key study in the repo…

Amazon book picks

Further Reading

Books and field guides related to When One Method Can Fool a Whole Program. Use these as the next step if you want deeper reading beyond the article.

Book

The Demon-Haunted World

By Carl Sagan

Promotes skeptical evaluation of extraordinary claims.

See on Amazon

Book

Science and pseudoscience in clinical psychology

By Steven J. Lynn, Scott O. Lilienfeld et al.

First published 2002. Subjects: Psychiatry, Clinical psychology, Psychotherapy, Medical misconceptions, Research.

See on Amazon

Book

Research Design

By John W. Creswell, J. David Creswell

First published 1994. Subjects: Statistical methods, Research, Methodology, Social sciences, Social sciences - research - methodology.

See on Amazon

Book

Parapsychology

By Etzel Cardeña, John Palmer et al.

First published 2015. Subjects: Parapsychology.

See on Amazon

Browse more on Amazon:The Demon Haunted WorldScience pseudoscience clinical psychologyResearch Design

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Example eBay listing

Canyon View Through Arch Outside Mo Framed Wall Art Poster Canvas Print Picture

SearcheBay.co.uk: remote viewing poster

Browse similar oneBay.co.uk

Example eBay listing

Photo:Steele Missouri 1938 Children Viewing Movie Posters Outside Theater

SearcheBay.co.uk: remote viewing poster

Browse similar oneBay.co.uk

Example eBay listing

Boulevard Of Lights At Dusk Eiffel Remote View Wall Home Decor - POSTER 20x30

SearcheBay.co.uk: remote viewing poster

Browse similar oneBay.co.uk

Example eBay listing

PSIONIC REMOTE VIEWING Wall Art, Psychic Telepathy Medium Empath Meditate Poster

SearcheBay.co.uk: remote viewing poster

Browse similar oneBay.co.uk

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: three-mode.leidenuniv.nl
Title: Assessing Method Variance in Multitrait-Multimethod
Link:https://three-mode.leidenuniv.nl/pdf/b/bagozzi1990jap.pdf

Published: September 14, 2001

2. Source: conjointly.com
Link:https://conjointly.com/kb/construct-validity-threats/

3. Source: researchprofiles.herts.ac.uk
Title: experiment one of the saic remote viewing program a critical re e
Link:https://researchprofiles.herts.ac.uk/en/publications/experiment-one-of-the-saic-remote-viewing-program-a-critical-re-e

5. Source: youtube.com
Title: Researching [Anomalous Cognition]({{ ‘not-clairvoyance/’ | relative_url }}) with Edwin C. May
Link:https://www.youtube.com/watch?v=sT_tbmB4J5I

6. Source: youtube.com
Title: Ray Hyman
Link:https://www.youtube.com/watch?v=-8VAOzcXGc4

7. Source: nsarchive2.gwu.edu
Link:https://nsarchive2.gwu.edu/NSAEBB/NSAEBB438/docs/doc_57.pdf

Published: March 13, 2015

8. Source: Wikipedia
Title: Remote viewing
Link:https://en.wikipedia.org/wiki/Remote_viewing

9. Source: psi-encyclopedia.spr.ac.uk
Title: spr.ac.uk Remote Viewing
Link:https://psi-encyclopedia.spr.ac.uk/articles/remote-viewing/

10. Source: flicks.com.au
Title: Ray | Where to watch streaming and online in Australia Jamie Foxx
Link:https://www.flicks.com.au/movie/ray/

Additional References

11. Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180005-5.pdf

12. Source: merriam-webster.com
Link:https://www.merriam-webster.com/dictionary/ray

13. Source: raythebrand.com
Link:https://raythebrand.com/?srsltid=AfmBOoo04wS54zNQ9AAfBNrvmD8nLMIvIb3-iYMtN4xYuxYyPzz8cofM

14. Source: netflix.com
Link:https://www.netflix.com/title/70001994

15. Source: reddit.com
Link:https://www.reddit.com/r/remoteviewing/comments/1dazs09/creation_of_study_on_statistical_evidence_of/

16. Source: rottentomatoes.com
Link:https://www.rottentomatoes.com/m/ray

17. Source: sbs.com.au
Link:https://www.sbs.com.au/whats-on/article/ray-review/n0ztsa5q1

18. Source: researchgate.net
Title: 374881423 Remote Viewing A 1974 2022 Systematic Review and [Meta Analysis]({{ ‘meta-analysis/’ | relative_url }})
Link:https://www.researchgate.net/publication/374881423_Remote_Viewing_A_1974-2022_Systematic_Review_and_Meta-Analysis

19. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/1579653/

20. Source: ics.uci.edu
Link:https://www.ics.uci.edu/~jutts/hyman.html