Can Pooled Results Prove the Effect?

Introduction

Meta-analyses are often presented as the strongest statistical evidence in favour of remote viewing because they combine results from many experiments rather than relying on a handful of striking studies. Supporters argue that, although individual experiments may be small or noisy, pooling them reveals a consistent signal that is unlikely to arise by chance. Critics respond that combining studies does not automatically produce stronger evidence if the underlying literature contains methodological weaknesses, selective reporting or inconsistent experimental designs. The central question is therefore not whether a meta-analysis can detect an average effect, but whether that average represents a genuine phenomenon or an artefact of the published record.[CIA+2National Security Archive]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…

Within the broader debate about repeatability and scientific standards, meta-analysis occupies an unusual position. It can increase statistical power and identify broad patterns across decades of research, yet it is also especially vulnerable to publication bias, heterogeneous methods and subjective decisions about which studies to include. For remote viewing, these issues are inseparable from the interpretation of the evidence itself.

What supporters take from positive meta-analyses

Advocates of remote viewing argue that the accumulated statistical record is more persuasive than any individual experiment. A recent systematic review and meta-analysis covering studies from 1974 to 2022 identified 36 studies comprising 40 effect sizes. After excluding statistical outliers, the authors reported an average effect size of approximately 0.34 and concluded that several statistical tests did not indicate substantial publication bias. They also reported only a modest decline in effect size over time, interpreting this as evidence that the reported anomaly has remained relatively stable across decades.[Journal of Scientific Exploration]journalofscientificexploration.orgJournal of Scientific ExplorationRemote Viewing: A 1974-2022 Systematic Review and…19 Oct 2023 — This is the first meta-analysis of al…

From this perspective, the key argument is cumulative consistency rather than spectacular demonstrations. Individual studies may fail because of inexperienced participants, procedural variation or simple statistical noise, but if positive effects repeatedly emerge across independent datasets, supporters argue that the combined evidence deserves attention.

Proponents also note that statistical significance alone is not their only claim. Modern meta-analyses increasingly apply multiple approaches—including Bayesian analyses, selection models and sensitivity analyses—to assess whether apparent effects survive alternative statistical assumptions. They argue that the persistence of a positive average under several analytical methods strengthens the inference that something beyond random chance may be present.[Journal of Scientific Exploration]journalofscientificexploration.orgJournal of Scientific ExplorationRemote Viewing: A 1974-2022 Systematic Review and…19 Oct 2023 — This is the first meta-analysis of al…

Why small-study and file-drawer effects matter

The principal criticism is that a meta-analysis cannot correct for systematic problems already embedded within the literature it summarises. If positive studies are more likely to be published than null studies, the combined estimate may exaggerate the true effect.

This concern is known as the file-drawer problem. Studies producing null or disappointing results may never appear in journals, remain unpublished in institutional archives or simply never be written up. Because meta-analyses generally depend on available studies, missing negative findings can inflate the apparent success rate. Publication bias has become a recognised concern across many scientific disciplines rather than an issue unique to paranormal research.[Wikipedia+2arXiv]WikipediaOpen source on wikipedia.org.

Remote-viewing research presents particular challenges:[cia.gov]cia.govAN EVALUATION OF REMOTE VIEWINGThe first component was a review of the research program. The second component was a review of the operati…

These features make it inherently difficult to estimate how representative the published record actually is.

Supporters reply that modern statistical techniques can detect some forms of publication bias through funnel plots, selection models and Bayesian approaches. The recent 1974–2022 meta-analysis concluded that its data showed little evidence of such bias. Critics caution, however, that statistical tests for publication bias have limited power when the number of available studies is modest or when studies differ substantially in design. Consequently, the absence of statistical evidence for publication bias is not equivalent to proving that publication bias does not exist.[Journal of Scientific Exploration+2arXiv]journalofscientificexploration.orgJournal of Scientific ExplorationRemote Viewing: A 1974-2022 Systematic Review and…19 Oct 2023 — This is the first meta-analysis of al…

How mixed methods complicate one average effect

Another difficulty is that “remote-viewing study” does not refer to one standardised experimental protocol.

Across five decades, investigators have used differing combinations of:

• participant selection and training;
• target types, including photographs, geographical locations and objects;
• outbound-agent and no-agent designs;
• precognition versus contemporaneous viewing tasks;
• different judging procedures;
• varying statistical outcome measures.

Pooling such studies produces a single average effect size, but that average may conceal substantial methodological diversity. A positive combined estimate could reflect a genuine effect across many methods, or it could arise because certain protocols produce stronger outcomes than others while weaker protocols contribute little. Conversely, a real but highly specific effect could become diluted when merged with substantially different experimental designs.

The recent meta-analysis attempted moderator analyses to examine factors such as participant selection, task type and publication status. Although some trends were reported—for example, stronger average effects in certain protocol variants—many comparisons had wide confidence intervals, making firm conclusions difficult. High statistical heterogeneity remained a notable feature of the dataset.[Journal of Scientific Exploration]journalofscientificexploration.orgJournal of Scientific ExplorationRemote Viewing: A 1974-2022 Systematic Review and…19 Oct 2023 — This is the first meta-analysis of al…

Why sceptics and supporters interpret the same statistics differently

The long-running exchange between psychologist Jessica Utts and psychologist Ray Hyman illustrates why meta-analysis alone has not resolved the debate.

Utts argued that the accumulated statistical evidence exceeded what would normally be dismissed as chance and that the remaining task was to explain the underlying mechanism. Hyman accepted that some datasets contained statistically unusual results but questioned whether methodological weaknesses, selective reporting and imperfect experimental controls had been sufficiently eliminated to justify concluding that paranormal perception had been demonstrated. Their differing interpretations were incorporated into the independent review commissioned for the United States government in 1995.[CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this rev…

Importantly, the review distinguished between evidence of statistical anomalies and evidence for the claimed phenomenon itself. It acknowledged that some laboratory results appeared statistically significant while concluding that the available research did not provide an unambiguous demonstration of remote viewing as a paranormal capability.[National Security Archive]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…by MD Mumford · 1995 · Cited by 76 — Utts and Hyman were asked…

Can pooled results prove the effect?

Most researchers, including many who value meta-analysis, would answer no. A meta-analysis is designed to estimate what existing studies collectively suggest, not to establish that the underlying studies are free from bias or methodological flaws.

For remote viewing, the strongest interpretation is therefore conditional. Positive meta-analyses indicate that reported experiments have often produced above-chance outcomes when considered together. They do not, by themselves, determine whether those outcomes reflect a genuine anomalous ability, subtle methodological artefacts, selective publication or some combination of these factors.

The decisive evidence would come from a growing body of preregistered, independently replicated, transparently reported experiments using standardised methods and complete reporting of both successful and unsuccessful trials. Until such evidence accumulates, meta-analyses remain important pieces of the discussion, but they cannot by themselves settle the publication-bias question or establish broad scientific acceptance. National Security Archive+2Journal of Scientific Exploration[nsarchive2.gwu.edu]nsarchive2.gwu.eduNational Security ArchiveAn Evaluation of Remote Viewing: Research and…by MD Mumford · 1995 · Cited by 76 — Utts and Hyman were asked…

Amazon book picks

Further Reading

Books and field guides related to Can Pooled Results Prove the Effect?. Use these as the next step if you want deeper reading beyond the article.

Book

Bad Science

By Ben Goldacre

Discusses publication bias and interpretation of evidence.

See on Amazon

Book

Calling Bullshit

By Carl T. Bergstrom, Jevin D. West

Explains statistical claims and misleading evidence.

See on Amazon

Book

The Demon-Haunted World

By Carl Sagan

Promotes careful evaluation of scientific evidence.

See on Amazon

Book

Phenomena

By Annie Jacobsen

First published 2017. Subjects: Military research, Parapsychology, Extrasensory perception, Psychokinesis, History.

See on Amazon

Browse more on Amazon:Bad ScienceCalling BullshitThe Demon Haunted World

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Example eBay listing

Periodic Table of Elements Poster For Kids, LAMINATED, Science & Chemistry Chart

SearcheBay.co.uk: science poster

Browse similar oneBay.co.uk

Example eBay listing

Periodic table with elements poster Science Wall Chart UPDATED Education Poster

SearcheBay.co.uk: science poster

Browse similar oneBay.co.uk

Example eBay listing

PERIODIC TABLE Real Elements A3 Laminated Chemistry Science Educational Poster

SearcheBay.co.uk: science poster

Browse similar oneBay.co.uk

Example eBay listing

Funny Science Poster Can I Lick It Periodic Table Print Chemistry Wall Art Nerd

SearcheBay.co.uk: science poster

Browse similar oneBay.co.uk

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180005-5.pdf

2. Source: Wikipedia
Link:https://en.wikipedia.org/wiki/Meta-analysis

3. Source: arxiv.org
Link:https://arxiv.org/abs/2208.12334

4. Source: arxiv.org
Title: arXiv Testing for publication bias in meta-analysis under Copas selection model
Link:https://arxiv.org/abs/2007.00836

5. Source: arxiv.org
Title: arXiv Modelling publication bias and p-hacking
Link:https://arxiv.org/abs/1911.12445

6. Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180006-4.pdf

7. Source: nsarchive2.gwu.edu
Link:https://nsarchive2.gwu.edu/NSAEBB/NSAEBB438/docs/doc_57.pdf

8. Source: journalofscientificexploration.org
Link:https://journalofscientificexploration.org/index.php/jse/article/view/2931

Additional References

9. Source: reddit.com
Link:https://www.reddit.com/r/remoteviewing/comments/1dazs09/creation_of_study_on_statistical_evidence_of/

10. Source: researchgate.net
Title: 374881423 Remote Viewing A 1974 2022 Systematic Review and Meta Analysis
Link:https://www.researchgate.net/publication/374881423_Remote_Viewing_A_1974-2022_Systematic_Review_and_Meta-Analysis

11. Source: academia.edu
Title: Remote Viewing A 1974 2022 Systematic Review and Meta Analysis
Link:https://www.academia.edu/108445581/Remote_Viewing_A_1974_2022_Systematic_Review_and_Meta_Analysis

12. Source: youtube.com
Title: The method that can “prove” almost anything
Link:https://www.youtube.com/watch?v=i60wwZDA1CI

13. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/20565166/

14. Source: youtube.com
Title: Publication Bias in Meta-Analysis (and Other Vexing Issues)
Link:https://www.youtube.com/watch?v=UhxRR7qnMxM

15. Source: youtube.com
Title: P-Curve: P-Hacking Your Way To Fame | Part 6 of 6
Link:https://www.youtube.com/watch?v=faOGQxtNlYg

16. Source: youtube.com
Title: Publication Bias in Meta-Analysis
Link:https://www.youtube.com/watch?v=38AAVQoXaBE

17. Source: youtube.com
Title: Part 1
Link:https://www.youtube.com/watch?v=lt0kHGRguUU

18. Source: philpapers.org
Link:https://philpapers.org/rec/ETRRVA