Within Replication
When Vague Hits Look Too Good
Blind judging can still overstate success when vague impressions are matched too generously to a small set of possible targets.
On this page
- Why fragmentary descriptions invite generous matching
- How multiple blind judges test reliability
- What objective scoring can and cannot fix
Page outline Jump by section
Introduction
Independent blind judging is one of the most important safeguards in remote-viewing experiments, but it is not a complete solution to the problem of interpretation. The central concern is known as over-matching: a remote-viewing transcript may contain broad, fragmentary or ambiguous descriptions that appear impressive only after a judge links them to a specific target. If enough flexibility exists in the matching process, even sincere and well-blinded judges can identify patterns that arise by chance rather than from genuine information transfer.
This issue sits at the heart of mainstream scientific criticism of remote-viewing evidence. The question is not simply whether judges are blind to the correct answer, but whether the judging procedure itself allows vague statements to be interpreted too generously. For this reason, modern experimental design places increasing emphasis on multiple independent judges, predefined scoring rules and objective statistical analysis alongside blinding. These measures reduce, but do not entirely eliminate, the possibility that ambiguous descriptions will be interpreted as successful “hits”.[National Security Archive]nsarchive2.gwu.eduIn this review, they were to cover four general topics: •.Read moreNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Utts and…
When Vague Hits Look Too Good
Remote-viewing sessions rarely produce precise statements such as “a red suspension bridge with two towers”. More commonly they generate collections of impressions: “water”, “metal”, “curved”, “open”, “bright”, “movement”, or rough sketches of shapes. Such descriptions are often compatible with many different targets.
The over-matching problem arises because humans are naturally skilled at recognising patterns after the fact. When presented with a limited set of possible targets, a judge can often find enough similarities between an ambiguous transcript and one candidate to make the match seem persuasive. At the same time, mismatches may receive less attention, while compatible fragments are emphasised.
A simple illustration demonstrates the problem. Imagine a transcript containing:
- “large structure”
- “water nearby”
- “hard surfaces”
- “vertical elements”
These observations could plausibly fit a lighthouse, harbour, bridge, dam, coastal fortification or waterfront skyscraper. If only four candidate photographs are presented, one may appear to fit noticeably better than the others even if the original description contains nothing uniquely identifying. The apparent success therefore depends partly on the available alternatives rather than on the specificity of the transcript itself.
This distinction between recognisable after matching and predictive before matching has long been emphasised by critics of remote-viewing methodology.[National Security Archive]nsarchive2.gwu.eduIn this review, they were to cover four general topics: •.Read moreNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Utts and…
Why Fragmentary Descriptions Invite Generous Matching
Over-matching is driven by several interacting mechanisms rather than a single flaw.
First, many remote-viewing reports intentionally encourage viewers to describe sensory impressions instead of naming the target directly. While this approach is intended to avoid premature guessing, it also produces descriptions with substantial interpretive flexibility.
Second, judges know that genuine correspondences may be incomplete. They therefore tend to evaluate overall resemblance instead of requiring exact agreement. This introduces unavoidable judgement calls about which similarities matter most.
Third, targets themselves often share common features. Outdoor scenes may all contain vegetation, sky, buildings or water. Industrial sites may all include concrete, machinery or geometric forms. Even careful target selection cannot entirely remove overlapping characteristics.
Finally, psychological factors affect interpretation. Cognitive research has repeatedly shown that people naturally detect meaningful patterns in noisy or ambiguous information. Independent judges are not immune to these tendencies simply because they are blind to the correct answer.
The result is that apparent success can be inflated without deliberate bias or fraud. The concern is methodological rather than accusatory: honest judges may still reach optimistic conclusions if the scoring system allows considerable interpretive freedom.[National Security Archive]nsarchive2.gwu.eduIn this review, they were to cover four general topics: •.Read moreNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Utts and…
How Multiple Blind Judges Test Reliability
One response is to replace a single evaluator with several independent blind judges.
In a stronger protocol:
- each judge receives exactly the same transcript;
- none knows which target is correct;
- judges work independently without discussion;
- each ranks or scores the candidate targets using predefined rules;
- agreement between judges is analysed statistically.
This approach tests whether the correspondence is robust enough that different evaluators reach similar conclusions. If several independent judges consistently identify the same target, confidence increases that the transcript contains information distinguishable from chance interpretation.
Conversely, if judges frequently disagree, the result suggests that the transcript supports multiple plausible interpretations. In that case, the apparent success of any one judge may reflect subjective matching rather than clear evidence.
Later remote-viewing research increasingly adopted such procedures in an effort to address earlier criticisms. The 1995 American Institutes for Research evaluation noted methodological improvements including stronger blinding and more formal judging procedures, while also concluding that judging and interpretation remained important issues in assessing the evidence.[National Security Archive]nsarchive2.gwu.eduIn this review, they were to cover four general topics: •.Read moreNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Utts and…
Research on associative remote viewing has also examined agreement among independent raters as an explicit variable, recognising that judging reliability itself deserves empirical measurement rather than assumption.[Koestler Parapsychology Unit]koestler-parapsychology.psy.ed.ac.ukKPU 1034 Published ResultsKoestler Parapsychology UnitAssociative remote viewing projects: Assessing rater…by DL KATZ · 2021 · Cited by 10 — Three teams of judg…
What Objective Scoring Can and Cannot Fix
To reduce subjective interpretation further, many experiments replace free-form judgement with predetermined scoring systems.
Examples include:
- forcing judges to rank several possible targets from best to worst match;
- using numerical similarity scales defined before judging begins;
- analysing only whether the correct target receives the highest rank;
- preregistering statistical methods before data collection.
These procedures reduce opportunities for post hoc reinterpretation and make statistical evaluation more transparent.
However, objective scoring has important limits.
A ranking system still depends on the judge’s initial assessment of similarity. Numerical scores cannot remove ambiguity if the underlying descriptions remain vague. Likewise, a transcript that matches several targets moderately well may still receive favourable rankings simply because one alternative appears slightly better than the rest.
Another limitation concerns target selection. If candidate targets are highly distinctive, judging becomes easier. If they share many visual or conceptual features, disagreement naturally increases. Objective scoring cannot compensate for poorly constructed target pools.
For these reasons, mainstream methodology treats blind judging as one component of a broader quality-control system rather than a complete safeguard. Strong experiments combine independent judging with careful target design, strict blinding, preregistered analyses and transparent reporting of both successful and unsuccessful trials.[National Security Archive]nsarchive2.gwu.eduIn this review, they were to cover four general topics: •.Read moreNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Utts and…
Why Over-Matching Matters for Repeatability
The over-matching problem directly affects repeatability because flexible interpretation can make isolated successes appear stronger than they truly are.
If one experiment produces encouraging results primarily because a particular judge interpreted transcripts generously, another laboratory using different judges may fail to reproduce the findings. The apparent effect therefore depends on the interpretation process rather than on a stable underlying phenomenon.
From the perspective of mainstream science, this is precisely why independent replication places so much emphasis on judging procedures. A phenomenon that survives changes in judges, laboratories and scoring systems is more convincing than one whose success depends on subjective interpretation.
Supporters of remote viewing generally agree that rigorous blinding and independent judging improve experimental quality, while sceptics argue that these safeguards must also demonstrate high agreement between judges and resistance to over-matching before positive findings can be considered reliable evidence. The continuing debate is therefore not simply about whether judges are blind, but whether the matching process itself produces results that remain stable across independent evaluators and repeated experiments. National Security Archive+2UC Irvine Bren School[nsarchive2.gwu.edu]nsarchive2.gwu.eduIn this review, they were to cover four general topics: •.Read moreNational Security ArchiveAn Evaluation of Remote Viewing: Research and…March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Utts and…
Amazon book picks
Further Reading
Books and field guides related to When Vague Hits Look Too Good. Use these as the next step if you want deeper reading beyond the article.
Flim-flam!
First published 1980. Subjects: Controversial literature, Occultism, Psychical research, Parapsicología, Ocultismo.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
1.
Source: nsarchive2.gwu.edu
Title: In this review, they were to cover four general topics: •.Read more
Link:https://nsarchive2.gwu.edu/NSAEBB/NSAEBB438/docs/doc_57.pdf
Source snippet
National Security ArchiveAn Evaluation of Remote Viewing: Research and...March 13, 2015 — by MD Mumford · 1995 · Cited by 76 — Utts and...
Published: March 13, 2015
2.
Source: ics.uci.edu
Link:https://www.ics.uci.edu/~jutts/hyman.html
Source snippet
She does not...Read more...
3.
Source: koestler-parapsychology.psy.ed.ac.uk
Title: KPU 1034 Published Results
Link:https://www.koestler-parapsychology.psy.ed.ac.uk/Documents/KPU_1034_Published_Results.pdf
Source snippet
Koestler Parapsychology UnitAssociative remote viewing projects: Assessing rater...by DL KATZ · 2021 · Cited by 10 — Three teams of judg...
Additional References
4.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180005-5.pdf
Source snippet
AN EVALUATION OF THE REMOTE VIEWING PROGRAMUtts and Hyman were asked to prepare independent reports based on their review. In this review...
5.
Source: academia.edu
Title: Remote Viewing a 1974 2022 systematic review and [meta analysis]({{ ‘meta-analysis/’ | relative_url }})
Link:https://www.academia.edu/111635854/Remote_Viewing_a_1974_2022_systematic_review_and_meta_analysis
Source snippet
Remote Viewing: a 1974-2022 systematic review and meta...The analysis indicates that remote viewing exhibits a 19.3% hit rate, significa...
6.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180006-4.pdf
Source snippet
ly, a set of decoys. In most recent laboratory...Read more...
7.
Source: researchgate.net
Title: An Assessment of the Evidence for Psychic Functioning
Link:https://www.researchgate.net/publication/333228024_An_Assessment_of_the_Evidence_for_Psychic_Functioning
Source snippet
Besides the ganzfield, the most prominent psi protocol is "remote viewing." In this, as described by Utts (1996) -in her aforementioned r...
8.
Source: youtube.com
Title: How We Know What Isn’t So — Why Humans See Patterns in Randomness
Link:https://www.youtube.com/watch?v=wDp65U7U5_A
Source snippet
Parapsychology blind judging remote viewing methodology Test your remote viewing abilities §∞ Sophia Angel Clairvoyant & Tarot ∞§...
9.
Source: scribd.com
Title: RVMeta Analysis Tressoldi Katz
Link:https://www.scribd.com/document/859542014/RVMeta-Analysis-Tressoldi-Katz
Source snippet
Remote Viewing: 1974-2022 Review | PDFThis document presents a systematic review and meta-analysis of remote viewing studies conducted fr...
10.
Source: youtube.com
Link:https://www.youtube.com/watch?v=d2r7Bk1NlgU
Source snippet
Clustering Illusion: See the Bigger Picture - Cognitive Biases Series...
11.
Source: youtube.com
Title: Clustering Illusion: See the Bigger Picture
Link:https://www.youtube.com/watch?v=cyNBRwmYy-g
Source snippet
How We Know What Isn’t So — Why Humans See Patterns in Randomness...
12.
Source: youtube.com
Title: Remote Viewing and Statistical Validation
Link:https://www.youtube.com/watch?v=YrwAiU2g5RU
Source snippet
Introduction to Cognitive Bias: Crash Course Scientific Thinking #1...
13.
Source: youtube.com
Title: [Statistics]({{ ‘statistics/’ | relative_url }}) in Parapsychology with Jessica Utts
Link:https://www.youtube.com/watch?v=WmYGtKB9EEA
Source snippet
Remote Viewing and Statistical Validation...
Topic Tree