Within Fair Test
The Hidden Risk of Early Feedback
Trial-by-trial feedback can echo into later responses, so fair tests must lock files and control judging order before revealing targets.
On this page
- How feedback can contaminate later sessions
- Why judging order matters in block designs
- When immediate feedback needs its own test plan
Page outline Jump by section
Introduction
In a fair remote viewing test, when feedback is given can matter almost as much as how the session is conducted. If viewers learn the target immediately after each trial, that knowledge can influence later sessions, consciously or unconsciously. Likewise, if judges see transcripts in their original order or know which sessions have already received feedback, they may infer relationships that have nothing to do with the viewing itself. For this reason, well-controlled experimental designs separate data collection, judging and feedback into distinct stages, particularly when multiple trials are grouped into a session block. This approach is intended to reduce ordinary sources of carryover and expectation effects before any claims about remote viewing performance are assessed. Research on remote viewing protocols, together with broader findings from experimental psychology on feedback timing and learning, supports treating feedback as a controlled experimental variable rather than an administrative detail.[PMC]pmc.ncbi.nlm.nih.govFeedback and Stimulus-Offset Timing Effects in Perceptual…by DA Worthy · 2013 · Cited by 59 — We examined how feedback delay and st…
How feedback can contaminate later sessions
The central concern is not that feedback is inherently harmful, but that it changes the participant’s information state. Once a viewer knows the correct target for one trial, that knowledge may alter later descriptions through memory, expectation, strategy changes or attempts to correct perceived mistakes. If several sessions are collected from the same participant over a short period, these influences can become intertwined with whatever process the experiment is intended to measure.
In remote viewing research this issue is especially important because responses are typically open-ended. Sketches, descriptive phrases and sensory impressions often contain broad imagery that can overlap across targets. After feedback, a participant may begin favouring features that appeared successful previously or avoid themes that seemed to produce poor matches. Such adaptation makes it difficult to determine whether later performance reflects the original phenomenon under investigation or ordinary learning from previous outcomes. General experimental psychology consistently shows that feedback timing changes subsequent behaviour and learning, demonstrating that feedback is an active intervention rather than a neutral endpoint.[PMC]pmc.ncbi.nlm.nih.govFeedback and Stimulus-Offset Timing Effects in Perceptual…by DA Worthy · 2013 · Cited by 59 — We examined how feedback delay and st…
Carryover can also occur without deliberate effort. Participants frequently remember unusual targets, emotionally striking images or distinctive judging comments. Those memories may later appear in fresh transcripts as genuine impressions or simple recall, making independent trials less independent than they appear.
For this reason, many experimental protocols recommend collecting an entire predefined block of sessions before revealing any targets. Delaying feedback preserves the independence of the responses within that block and makes statistical interpretation more straightforward.
Why judging order matters in block designs
Separating judging from feedback is only part of the solution. The order in which transcripts are judged can also introduce unwanted information.
When judges receive transcripts in chronological order, they may notice changes in style, confidence or recurring imagery and unconsciously use those patterns to infer target identities. Earlier debates over remote viewing methodology highlighted the danger of extraneous cues embedded in transcripts and experimental materials, including clues arising from session order and handling procedures rather than the descriptive content itself.
A stronger design removes these opportunities by:
- assigning anonymous trial identifiers instead of dates or session numbers;
- randomising transcript order before judging;
- randomising the order of candidate targets presented to judges;
- ensuring judges do not know which sessions belong to the same participant or experimental block; and
- completing all judging before any participant receives target feedback.
These safeguards reduce the risk that judges unknowingly reconstruct information from chronology rather than from the transcript itself. The underlying principle mirrors good practice in many behavioural sciences, where random presentation order and blinded assessment are standard methods for limiting observer bias and improving inter-rater reliability.[ScienceDirect]sciencedirect.comScienceDirect Interrater ReliabilityInterrater Reliability - an overviewInterrater reliability is enhanced by training data collectors, providing them with a gu…
When immediate feedback needs its own test plan
Immediate feedback is not automatically inappropriate. Some researchers may wish to investigate whether rapid feedback influences future remote viewing performance, learning or motivation. However, that becomes a different experimental question.
Rather than mixing immediate-feedback trials with delayed-feedback trials inside the same dataset, a clearer design specifies feedback timing in advance as an independent experimental condition. Participants can then be randomly assigned, for example, to:
- Immediate-feedback blocks, where the target is revealed after every trial.
- Delayed-feedback blocks, where all judging is completed before any targets are disclosed.
- No-feedback conditions, where participants receive feedback only after the entire experiment or not at all until data collection has finished.
Treating feedback timing as a planned variable allows researchers to compare conditions directly instead of introducing an uncontrolled source of variation. Experimental research outside parapsychology has repeatedly shown that delayed and immediate feedback can produce different learning patterns, reinforcing the need to specify feedback schedules before data collection begins.[PMC]pmc.ncbi.nlm.nih.govFeedback and Stimulus-Offset Timing Effects in Perceptual…by DA Worthy · 2013 · Cited by 59 — We examined how feedback delay and st…
Practical safeguards for session blocks
For a remote viewing experiment intended to minimise carryover between trials, several implementation choices are particularly valuable.
Lock session records before feedback. Once a transcript or drawing is completed, it should be timestamped, archived and protected from later editing before any target information is released.
Judge first, reveal later. Independent judging should be completed using anonymised materials before participants learn any target identities.
Keep complete blocks together. If a study defines a block of ten or twenty sessions, feedback should normally wait until every session in that block has been collected and scored.
Separate operational roles. Personnel responsible for releasing feedback should not also perform judging or transcript preparation, reducing opportunities for accidental information leakage.
Document the schedule in advance. The experimental protocol should specify exactly when feedback will occur so that timing decisions cannot be altered after examining interim results.
These measures do not assume that feedback necessarily produces bias in every experiment. Instead, they prevent ordinary psychological processes from becoming plausible alternative explanations for apparent success.
Why this matters for interpreting results
Claims about remote viewing often depend on small statistical differences accumulated across many trials. In that setting, even modest carryover between sessions can complicate interpretation because observations are no longer fully independent.
By delaying feedback until after judging is complete and by removing chronological cues from transcript evaluation, researchers reduce opportunities for learning effects, expectancy and observer inference to influence the results. These procedures cannot establish whether remote viewing exists, but they strengthen the fairness of the test by ensuring that any apparent performance is less likely to arise from ordinary feedback-driven contamination or judging artefacts.
Amazon book picks
Further Reading
Books and field guides related to The Hidden Risk of Early Feedback. Use these as the next step if you want deeper reading beyond the article.
Statistics Done Wrong
First published 2015. Subjects: Missing observations (Statistics), Methodology, Statistics.
The elusive quarry
First published 1989. Subjects: Book reviews, Controversial literature, Parapsychology, Psychical research.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
1.
Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC3560315/
Source snippet
Feedback and Stimulus-Offset Timing Effects in Perceptual...by DA Worthy · 2013 · Cited by 59 — We examined how feedback delay and st...
2.
Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC11182045/
Source snippet
by K Nunn · 2024 · Cited by 3 — Delaying feedback may increase MTL involvement and, for older adults, improve category learning. Age-r...
3.
Source: sciencedirect.com
Title: ScienceDirect Interrater Reliability
Link:https://www.sciencedirect.com/topics/nursing-and-health-professions/interrater-reliability
Source snippet
Interrater Reliability - an overviewInterrater reliability is enhanced by training data collectors, providing them with a gu...
Additional References
4.
Source: researchgate.net
Link:https://www.researchgate.net/figure/Showing-the-nine-phases-of-a-typical-ARV-protocol_tbl1_344470071
Source snippet
Showing the nine phases of a typical ARV protocolThis double-blind study utilised dreaming instead of remote viewing as a precognitive to...
5.
Source: craftingjustice.files.wordpress.com
Link:https://craftingjustice.files.wordpress.com/2017/04/david-boud-rosemary-keogh-david-walker-reflection_-turning-experience-into-learning-routledge-1985-pp-1-165.pdf
Source snippet
wordpress.comREFLECTION: TURNING EXPERIENCE INTO LEARNINGEach person immediately after a turn as client mapped out the series of states p...
6.
Source: research.hva.nl
Title: focused versus unfocused corrective feedback from part iv feedbac
Link:https://research.hva.nl/en/publications/focused-versus-unfocused-corrective-feedback-from-part-iv-feedbac/
Source snippet
versus Unfocused Corrective Feedback: from Part IVby C van Beuningen · 2021 · Cited by 18 — This chapter zooms in on two feedback types t...
7.
Source: kb.roboticseducation.org
Title: 4972627664919 Guide to Judging Remote Judging
Link:https://kb.roboticseducation.org/hc/en-us/articles/4972627664919-Guide-to-Judging-Remote-Judging
Source snippet
to Judging: Remote Judging20 Aug 2025 — Remote judging occurs when either the Engineering Notebook Judging process, Initial Team Intervie...
8.
Source: youtube.com
Title: Neuroscientist on Remote Viewing, [Stargate]({{ ‘stargate/’ | relative_url }}) and Telepathy
Link:https://www.youtube.com/watch?v=GIGwduXbM50
Source snippet
This video on within-subjects experimental design explains the mechanics of carryover and order effects in repeated trial blocks, detaili...
9.
Source: scribd.com
Title: Katz Associative Remote Viewing
Link:https://www.scribd.com/document/1005021917/KatzAssociativeRemoteViewing
Source snippet
PDF | Parapsychology17 Nov 1986 — This book explores Associative Remote Viewing (ARV) as a method for p...
10.
Source: youtube.com
Title: Animation: Carryover Effect and How to Remove It
Link:https://www.youtube.com/watch?v=phRWcJSFmjQ
Source snippet
Feeling The Future: Is ESP Real According to Science?...
11.
Source: youtube.com
Title: Within-Subjects Design: Order and Carryover Effects
Link:https://www.youtube.com/watch?v=pSwLzUdtQfg
Source snippet
Animation: Carryover Effect and How to Remove It...
12.
Source: youtube.com
Title: How to Test the Paranormal
Link:https://www.youtube.com/watch?v=cMfqRzJIVvI
Source snippet
Neuroscientist on Remote Viewing, Stargate and Telepathy...
13.
Source: youtube.com
Title: Feeling The Future: Is ESP Real According to Science?
Link:https://www.youtube.com/watch?v=qM1xv4a6Ls8
Source snippet
How to Test the Paranormal...
Topic Tree