Within Replication
Why Positive Numbers Were Not Enough
The CIA-commissioned AIR review shows why positive statistics did not settle whether remote viewing had been demonstrated as paranormal.
On this page
- What the AIR panel was asked to evaluate
- Why Utts and Hyman disagreed
- How the conclusion shaped mainstream caution
Page outline Jump by section
Introduction
The 1995 American Institutes for Research (AIR) evaluation of the US government’s remote-viewing programme remains one of the most important assessments of whether reported statistical successes amounted to scientific evidence for paranormal perception. Rather than asking only whether some experiments produced results above chance, the review examined whether those findings justified concluding that remote viewing had been demonstrated as a genuine phenomenon and whether it had practical value for intelligence work. The outcome illustrates a key principle of mainstream science: statistically significant results are necessary but not sufficient. The AIR panel agreed that some laboratory data deserved attention, yet disagreed sharply on what those numbers meant and whether they met the evidential standards required for an extraordinary claim.[CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…
What the AIR panel was asked to evaluate
The review was commissioned after the US intelligence community transferred oversight of the long-running Stargate programme to the CIA. AIR was asked to assess both the scientific quality of the research and the operational usefulness of remote viewing for intelligence purposes. To provide contrasting perspectives, AIR appointed statistician Jessica Utts, who had previously published on parapsychological research, and psychologist Ray Hyman, a prominent critic of paranormal claims. Both reviewed the same body of evidence but were expected to reach independent conclusions.[CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…
The scientific review concentrated heavily on the more recent experiments conducted at Science Applications International Corporation (SAIC), because these were generally considered methodologically stronger than many earlier studies. Earlier work had attracted criticism over weaknesses such as inadequate blinding, subjective judging and possible information leakage. Later experiments attempted to address several of these concerns through improved experimental controls, making them a better test of whether a genuine effect remained once obvious flaws were reduced.[CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…
The review also distinguished between two separate questions that are often conflated:
- Did some experiments produce statistically unusual outcomes?
- Did those outcomes demonstrate paranormal remote viewing under ordinary scientific standards?
The panel agreed that these were not identical questions.
Why Utts and Hyman disagreed
The disagreement between Utts and Hyman did not arise because one accepted the data and the other rejected it outright. Both acknowledged that several experiments produced statistical results that were difficult to dismiss as simple random chance.
Utts argued that the cumulative statistical evidence justified concluding that an anomalous information-transfer effect had been demonstrated under controlled laboratory conditions. In her view, the consistency of significant departures from chance across multiple studies meant that the existence of an effect should be accepted, even if its mechanism remained unknown. She argued that future work should focus less on proving existence and more on understanding the phenomenon and identifying the conditions under which it occurred.[CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…
Hyman accepted that some reported effect sizes were larger and more consistent than would normally be expected from random variation alone. However, he argued that statistical significance alone could not distinguish between a genuine paranormal effect and subtle methodological problems that had not yet been identified. He emphasised that extraordinary claims require confidence that all plausible ordinary explanations have been excluded, something he believed the available evidence had not yet achieved.[Center for Inquiry]cdn.centerforinquiry.orgCenter for InquiryEvaluation of the Military's Twenty-Year Program on…by RAY HYMAN · 1996 · Cited by 8 — The remote-viewing experiment…
His position rested on several concerns:
- residual opportunities for sensory leakage or inadvertent cueing;
- reliance on subjective judging procedures;
- uncertainty about whether analytical decisions could influence results;
- insufficient independent replication by researchers outside the original programmes.
In other words, Hyman questioned whether the observed statistics uniquely supported a paranormal explanation rather than simply indicating that something in the experimental system remained imperfectly understood.[Center for Inquiry]cdn.centerforinquiry.orgCenter for InquiryEvaluation of the Military's Twenty-Year Program on…by RAY HYMAN · 1996 · Cited by 8 — The remote-viewing experiment…
Why positive statistics were not enough
The AIR review became influential because it highlighted an issue that extends well beyond remote viewing. A statistically significant result answers only a limited question: if there were truly no effect, how surprising would the observed data be? It does not establish why the result occurred.
Several features of remote-viewing experiments complicated interpretation.[cia.gov]cia.govAN EVALUATION OF REMOTE VIEWINGUtts has published articles that view paranormal interpretations positively, while Dr. Hyman was select…
First, many experiments depended on judges matching narrative descriptions to possible targets. Even under blind procedures, judging involves interpretation, and different scoring methods can influence measured success rates.
Second, laboratory effects were often modest rather than dramatic. Small statistical departures from chance require especially rigorous controls because even subtle procedural artefacts can produce misleading patterns across many trials.
Third, there was no accepted theoretical framework explaining how remote viewing could occur. In established sciences, statistical evidence is often evaluated alongside independently verified mechanisms and converging evidence from multiple methods. Remote viewing lacked this broader supporting structure, making methodological certainty even more important.[CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…
The AIR review therefore illustrated a broader scientific principle: statistical significance is one component of evidence, not the endpoint of scientific demonstration.
How the conclusion shaped mainstream caution
Although Utts and Hyman disagreed about interpretation, the overall AIR evaluation did not recommend treating remote viewing as an established intelligence capability.
The review found little documented evidence that remote-viewing sessions had produced operationally useful intelligence that could be independently verified or routinely relied upon. Laboratory findings, even if statistically interesting, had not translated into a dependable intelligence tool under real-world conditions. This distinction between laboratory anomalies and practical application was central to the CIA’s subsequent decision to discontinue the programme.[CIA]cia.govAN EVALUATION OF REMOTE VIEWINGUtts has published articles that view paranormal interpretations positively, while Dr. Hyman was select…
Within mainstream science, the AIR review reinforced a cautious position that continues to influence discussions of remote viewing. Researchers who remain sceptical often point to the review as evidence that promising statistical outcomes must ultimately be supported by independent replication, transparent methodology and reproducible results across laboratories before extraordinary conclusions are accepted. Supporters, by contrast, frequently cite Utts’ assessment as evidence that statistical findings deserve more serious attention than they have traditionally received.[CIA+2Center for Inquiry]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…
The lasting importance of the AIR review lies precisely in this unresolved tension. It demonstrated that a body of research can contain statistically significant findings while still failing to persuade the wider scientific community that a paranormal phenomenon has been established. For mainstream science, the decisive question was never simply whether some numbers exceeded chance, but whether those numbers could survive every reasonable methodological challenge and remain independently reproducible.
Amazon book picks
Further Reading
Books and field guides related to Why Positive Numbers Were Not Enough. Use these as the next step if you want deeper reading beyond the article.
The Demon-Haunted World
Explains mainstream scientific standards for extraordinary claims.
The conscious universe
First published 1997. Subjects: Parapsychology, Case studies, Cas, Études de, Paranormale verschijnselen, Parapsychologie.
Phenomena
First published 2017. Subjects: Military research, Parapsychology, Extrasensory perception, Psychokinesis, History.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
1.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180005-5.pdf
Source snippet
AN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev...
2.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180006-4.pdf
Source snippet
AN EVALUATION OF REMOTE VIEWINGUtts has published articles that view paranormal interpretations positively, while Dr. Hyman was select...
3.
Source: cdn.centerforinquiry.org
Link:https://cdn.centerforinquiry.org/wp-content/uploads/sites/29/1996/03/22165045/p21.pdf
Source snippet
Center for InquiryEvaluation of the Military's Twenty-Year Program on...by RAY HYMAN · 1996 · Cited by 8 — The remote-viewing experiment...
4.
Source: Wikipedia
Title: Remote viewing
Link:https://en.wikipedia.org/wiki/Remote_viewing
Source snippet
Remote viewingThe program ran from 1975 to 1995 and ended after evaluators concluded that remote viewers consistently failed to produc...
Additional References
5.
Source: philpapers.org
Link:https://philpapers.org/rec/HYMEOP
Source snippet
Evaluation of Program on Anomalous Mental Phenomena.by R Hyman · Cited by 74 — Professor Jessica Utts and I were given the task of evalua...
6.
Source: irp.fas.org
Link:https://irp.fas.org/program/collect/air1995.pdf
Source snippet
reportJavaScript is disabled. In order to continue, we need to verify that you're not a robot. This requires JavaScript. Enable JavaScrip...
7.
Source: bazaarmodel.net
Link:https://bazaarmodel.net/Onderwerpen/remoteviewingCIA/CIA-InitiatedRV.html
Source snippet
CIA-Initiated Remote Viewing At Stanford Research Instituteby HE Puthoff · Cited by 2 — Abstract - In July 1995 the CIA declassified, and...
Published: July 1995
8.
Source: researchgate.net
Title: 369604750 Remote Viewing a 1974 2022 systematic review and [meta analysis]({{ ‘meta-analysis/’ | relative_url }})
Link:https://www.researchgate.net/publication/369604750_Remote_Viewing_a_1974-2022_systematic_review_and_meta-analysis
Source snippet
(PDF) Remote Viewing: a 1974-2022 systematic review...Both frequentist and Bayesian meta-analyzes revealed a strong average effect size...
9.
Source: researchprofiles.herts.ac.uk
Title: experiment one of the saic remote viewing program a critical re e
Link:https://researchprofiles.herts.ac.uk/en/publications/experiment-one-of-the-saic-remote-viewing-program-a-critical-re-e
Source snippet
Hertfordshire Research ProfilesExperiment One of the SAIC Remote Viewing Programby R Wiseman · 1999 · Cited by 24 — This paper first outl...
10.
Source: alice.id.tue.nl
Link:https://www.alice.id.tue.nl/references/mumford-rose-goslin-1995.pdf
Source snippet
An Evaluation of Remote Viewing: Research and...by MD Mumford · 1995 · Cited by 74 — The CIA asked AIR to address a number of key o...
11.
Source: researchgate.net
Title: An Assessment of the Evidence for Psychic Functioning
Link:https://www.researchgate.net/publication/333228024_An_Assessment_of_the_Evidence_for_Psychic_Functioning
Source snippet
Besides the ganzfield, the most prominent psi [protocol]({{ 'protocol/' | relative_url }}) is "remote viewing." As described by Utts (2019) -in a 1995 report for the US Cong...
12.
Source: slideshare.net
Link:https://www.slideshare.net/slideshow/an-evaluation-of-remote-viewing-research-and-applications-air1995pdf/257460594
Source snippet
atory research, it was unclear if this was due to paranormal ability.Read more...
13.
Source: researchgate.net
Link:https://www.researchgate.net/publication/267978941Evaluation_of_Program_on%27Anomalous_Mental_Phenomena%27
Source snippet
related phenomena which was carried out at Stanford Re-search Institute...
14.
Source: researchonline.ljmu.ac.uk
Title: ljmu.ac.uk Follow‐up on the U.S
Link:https://researchonline.ljmu.ac.uk/id/eprint/23584/1/Follow-up%20on%20the%20U.S.%20Central%20Intelligence%20Agency%27s%20%28CIA%29%20remote%20viewing%20experiments%E2%98%86.pdf
Source snippet
Central Intelligence Agency's (CIA)...by Á Escolà-Gascón · 2023 · Cited by 10 — 1.1. What is remote viewing? RV is an experiential techn...
Topic Tree