Within Hits
Can Blind Judges Separate Hits From Hindsight?
Blind judging and decoys test whether a transcript fits the real target better than plausible alternatives.
On this page
- Why self scoring is not enough
- How decoy targets test flexible matches
- Where judging still leaves room for ambiguity
Page outline Jump by section
Introduction
Blind judging and decoy targets were introduced into remote-viewing research to answer a simple but crucial question: does a session genuinely match its intended target better than it matches other plausible alternatives? Without such safeguards, a transcript can appear impressively accurate only because the reader already knows the correct answer and unconsciously interprets vague descriptions to fit it. Blind judging attempts to remove that hindsight by ensuring that the person evaluating a transcript does not know which target is correct, while decoy targets provide realistic alternatives that the transcript must outperform. Together, these procedures are among the principal methodological safeguards developed to address the pattern-matching problem in free-response remote-viewing experiments. Even so, they reduce rather than eliminate subjectivity, and debates continue over how much protection they provide against flexible interpretation.[CIA+2National Security Archive]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMA judge then examines the viewer's report and determines if this report matches the target…
Why self-scoring is not enough
The most obvious weakness in evaluating a remote-viewing session is allowing either the viewer or someone who already knows the target to judge its accuracy. Once the true target is revealed, almost any sufficiently broad description can be reinterpreted in its light. Words such as “curved”, “bright”, “industrial”, “open”, or “movement” may seem compelling after seeing a photograph but could often apply to numerous unrelated locations or objects.
Blind judging was developed specifically to separate interpretation from knowledge of the correct answer. In a typical protocol:
- A viewer produces a written transcript and sketches without receiving feedback.
- An independent judge receives that transcript together with several possible targets.
- The judge does not know which target is correct.
- The judge ranks or scores the available targets according to how well they match the transcript.
If the genuine target consistently receives the highest ranking across many independent trials, researchers argue that the results become more difficult to explain purely by hindsight or selective interpretation. This approach shifts the question from “Can someone see similarities?” to “Can someone identify the correct target while unaware of which one it is?”[CIA+2CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMA judge then examines the viewer's report and determines if this report matches the target…
Importantly, blind judging does not require the judge to believe in remote viewing. The safeguard is procedural rather than psychological: keeping evaluators ignorant of the correct answer reduces opportunities for conscious or unconscious bias.
How decoy targets test flexible matches
Blind judging becomes much stronger when the correct target is presented alongside carefully selected decoy targets.
Rather than asking whether a transcript resembles one photograph, researchers ask whether it resembles the intended photograph more than several realistic alternatives. A common design presents one genuine target together with four decoys, requiring the judge to rank all five from best to worst fit. Under chance alone, the correct target would be expected to rank first only about one-fifth of the time in a five-target set. Repeated performance substantially above that expectation is treated as evidence that deserves statistical analysis.[National Security Archive]nsarchive2.gwu.edudoc 57National Security ArchiveAn Evaluation of Remote Viewing: Research and…by MD Mumford · 1995 · Cited by 76 — The response was later bli…
The value of decoys lies in exposing flexible matching. Imagine a transcript describing:
- a tall structure,
- metal,
- water nearby,
- open space.
Viewed alone beside a suspension bridge, those elements might seem remarkably accurate. However, if the same transcript also fits a harbour crane, offshore platform, lighthouse or riverside industrial plant almost equally well, its evidential value becomes much weaker. Decoy targets force judges to confront those competing interpretations instead of focusing only on the successful comparison.
Researchers therefore try to choose decoys that are plausible rather than obviously different. If the alternatives are too easy to reject, blind judging loses much of its value because the correct answer stands out for reasons unrelated to the transcript.
Where judging still leaves room for ambiguity
Although blind judging addresses one major source of bias, it does not remove every subjective element.
Choosing the scoring method
Different laboratories have used different judging procedures. Some ask judges simply to choose the best match. Others require complete rank ordering. Some assign numerical similarity scores across several dimensions, while others compare conceptual features extracted from transcripts. Different methods can produce different statistical outcomes, making comparisons across studies difficult. CIA-sponsored protocol documents describe efforts to formalise rank-order procedures precisely because informal matching proved too subjective.[CIA]cia.govA REMOTE VIEWING EVALUATION PROTOCOL…The first step toward systematizing the rank order judging procedure was to preprocess the raw…
The quality of the decoys
Decoy selection is itself a methodological decision.
If decoys differ dramatically from the real target—for example, comparing a lighthouse with four indoor office scenes—the judging task may become artificially easy. Conversely, if every decoy shares many visual characteristics with the true target, distinguishing them becomes much harder. Designing balanced target pools therefore requires careful consideration to avoid unintentionally favouring either positive or negative outcomes.
Persistent interpretive freedom
Even without knowing the correct answer, judges still interpret language and sketches.
Two independent judges may legitimately disagree over whether phrases such as “energy”, “movement”, “vertical”, or “contained space” better describe one target than another. Sketches present similar problems because rough geometric forms can often resemble multiple objects. Blind judging reduces confirmation bias but cannot eliminate the inherent ambiguity of free-response descriptions.
What evaluations found
By the time of the U.S. government’s review of the Stargate programme, blind judging with decoy targets had become standard practice in many laboratory experiments. The American Institutes for Research noted that remote-viewing studies commonly compared transcripts against the intended target and several decoys using blind rank-order procedures, reflecting an awareness that free-response material required formal evaluation rather than informal impression.[National Security Archive]nsarchive2.gwu.edudoc 57National Security ArchiveAn Evaluation of Remote Viewing: Research and…by MD Mumford · 1995 · Cited by 76 — The response was later bli…
The review nevertheless concluded that methodological improvements did not settle broader questions about operational usefulness. While some laboratory findings appeared statistically interesting, reviewers argued that the available evidence did not clearly demonstrate a reliable intelligence capability. Better judging procedures strengthened experimental design but did not by themselves establish the reality or practical value of remote viewing.[CIA]cia.govAN EVALUATION OF THE REMOTE VIEWING PROGRAMA judge then examines the viewer's report and determines if this report matches the target…
More recent reviews of the literature similarly treat independent judging as an important design feature rather than proof of the underlying phenomenon. Contemporary researchers continue to recommend independent judges, preregistered analysis plans, transparent scoring rules and open data as complementary safeguards intended to reduce researcher flexibility and improve reproducibility.[ResearchGate]researchgate.net374881423 Remote Viewing A 1974 2022 Systematic Review and Meta AnalysisResearchGate(PDF) Remote Viewing: A 1974-2022 Systematic Review…26 Oct 2023 — This is the first meta-analysis of all studies related t…
Why blind judging remains an important safeguard
Blind judges and decoy targets represent one of the most significant methodological responses to the pattern-matching problem in remote-viewing research. They acknowledge that free-response transcripts are inherently open to interpretation and attempt to make that interpretation measurable rather than anecdotal.
Their main strength is forcing a transcript to compete against plausible alternatives before anyone knows which target is correct. That substantially reduces the influence of hindsight and selective matching. However, the safeguard is not absolute. Judging criteria, target selection, transcript ambiguity and statistical choices all continue to influence results.
For that reason, blind judging is best understood not as a guarantee of objective accuracy but as a necessary control that raises the evidential standard. A claimed “hit” carries considerably more weight when it emerges from successful blind comparison against carefully constructed decoy targets than when it depends solely on a reader recognising similarities after the correct target has already been revealed.
Amazon book picks
Further Reading
Books and field guides related to Can Blind Judges Separate Hits From Hindsight?. Use these as the next step if you want deeper reading beyond the article.
Statistics Done Wrong
First published 2015. Subjects: Missing observations (Statistics), Methodology, Statistics.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
1.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00791R000200180005-5.pdf
Source snippet
AN EVALUATION OF THE REMOTE VIEWING PROGRAMA judge then examines the viewer's report and determines if this report matches the target...
2.
Source: cia.gov
Link:https://www.cia.gov/readingroom/document/cia-rdp96-00788r001800080001-5
Source snippet
A REMOTE VIEWING EVALUATION PROTOCOL...The first step toward systematizing the rank order judging procedure was to preprocess the raw...
3.
Source: cia.gov
Link:https://www.cia.gov/readingroom/docs/CIA-RDP96-00788R002000240029-4.pdf
Source snippet
th a given remote viewer, the results are subjected to...Read more...
4.
Source: researchgate.net
Title: 374881423 Remote Viewing A 1974 2022 Systematic Review and [Meta Analysis]({{ ‘meta-analysis/’ | relative_url }})
Link:https://www.researchgate.net/publication/374881423_Remote_Viewing_A_1974-2022_Systematic_Review_and_Meta-Analysis
Source snippet
ResearchGate(PDF) Remote Viewing: A 1974-2022 Systematic Review...26 Oct 2023 — This is the first meta-analysis of all studies related t...
5.
Source: nsarchive2.gwu.edu
Title: doc 57
Link:https://nsarchive2.gwu.edu/NSAEBB/NSAEBB438/docs/doc_57.pdf
Source snippet
National Security ArchiveAn Evaluation of Remote Viewing: Research and...by MD Mumford · 1995 · Cited by 76 — The response was later bli...
6.
Source: academia.edu
Title: Remote Viewing A 1974 2022 Systematic Review and Meta Analysis
Link:https://www.academia.edu/108445581/Remote_Viewing_A_1974_2022_Systematic_Review_and_Meta_Analysis
Source snippet
Remote Viewing: A 1974- 2022 Systematic Review and...The meta-analysis reveals that remote viewing protocols produced an average effect...
Additional References
7.
Source: pmc.ncbi.nlm.nih.gov
Title: PMCFollow‐up on the U.S
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10275521/
Source snippet
Central Intelligence Agency's (CIA...by Á Escolà‐Gascón · 2023 · Cited by 10 — Programs addressed remote viewing (RV), that is, determin...
8.
Source: scribd.com
Title: CIA RDP96 00791R000200180006 4 1 pdf
Link:https://www.scribd.com/document/395167398/CIA-RDP96-00791R000200180006-4-1-pdf
Source snippet
Cia RDP96 00791R000200180006 4 PDF... EVALUATION OF REMOTE VIEWING: RESEARCH AND APPLICATIONS MICHAEL D.... Remote Viewing Protocol Over...
9.
Source: youtube.com
Title: Confirmation Bias and The Scientific Method I: How to Avoid Confirmation Bias
Link:https://www.youtube.com/watch?v=tOZLL7CsHCU
Source snippet
Placebo Effect, Control Groups, and the Double Blind Experiment (3.2)...
10.
Source: youtube.com
Title: Remote Viewing Explained: How the Mind Sees Without Eyes
Link:https://www.youtube.com/watch?v=whpZzUwrUNU
Source snippet
CONTROLLED REMOTE VIEWING - Lori Williams #32...
11.
Source: youtube.com
Title: The Double-Blind Study Explained
Link:https://www.youtube.com/watch?v=NdqcOZADAg8
Source snippet
Confirmation Bias and The Scientific Method I: How to Avoid Confirmation Bias...
12.
Source: youtube.com
Title: Placebo Effect, Control Groups, and the Double Blind Experiment (3.2)
Link:https://www.youtube.com/watch?v=GMqrOdCx4Yg
13.
Source: youtube.com
Title: CONTROLLED REMOTE VIEWING
Link:https://www.youtube.com/watch?v=wWqldsr_zIg
Source snippet
The Double-Blind Study Explained...
Topic Tree