Ask What the Pilot Would Teach
A pilot is a bounded attempt to learn about a proposed approach before a wider commitment. It is useful when a consequential uncertainty can be examined at a smaller scale, the activity can be run responsibly and the result could change a decision. It is not automatically the right next step whenever evidence is incomplete.
Separate three questions: could the arrangement operate; would intended users be able to use it; and would it produce the desired longer-term result? A short operational trial might inform the first two without establishing the third. Do not promise an impact finding from a feasibility exercise.
Compare a Pilot With Other Next Steps
An existing record check may resolve a mistaken assumption without starting a service. A qualified technical review may be needed before any experiment is permissible. An urgent known need may require an established response within the responsible team's mandate rather than a trial that withholds essential help. Conversely, a new arrangement with uncertain staffing or access may benefit from a reversible trial once the necessary approvals and safeguards are in place.
The UNDP project-formulation procedure, steps 3.1-3.3, illustrates stakeholder participation, full costing and risk assessment in its own institutional process. It does not mandate an NGO pilot or supply the thresholds below. The decision method and service case here are original planning guidance.
A Complete Fictional Pilot Decision
Harbor Advice is considering an afternoon session at one of its two desks. Existing visit logs cover current users; they cannot establish demand among people who cannot attend. Some consultation participants identify opening hours as a barrier, while others identify transport and privacy. A venue is available, but staff cover remains unconfirmed.
Learning question: can the team provide an accessible afternoon session without reducing essential morning cover, and what practical barriers arise for intended users? Out of scope: district-wide demand, causal outcome effects, permanent expansion and individual eligibility decisions.
Prerequisites: the director authorizes the scope and resources; the operations manager confirms staff cover and venue access; responsible staff settle safe communication, consent for any feedback, minimal records and referral arrangements. Requested staffing is not a fulfilled prerequisite. If it remains unconfirmed, do not launch the proposed trial.
Bounded design: agree a small number of sessions and a decision meeting around the operational calendar, with a cost ceiling, responsible lead and a way to return to the prior arrangement. Those parameters must fit local need and duties; there is no universal four-week trial or sample size. Invite perspectives through accessible permitted routes, including those missing from current service records, without treating self-selected feedback as representative.
| Decision at Review | Evidence and Reasoning in the Fictional Case |
|---|---|
| Continue within the existing boundary | Staff cover was maintained, the service operated as intended and the remaining uncertainty can be examined responsibly; this is permission for the agreed next step only |
| Change the arrangement | Afternoon access is feasible but transport or privacy barriers remain; redesign the access response with affected users instead of assuming a longer trial will solve them |
| Stop or pause the proposed arrangement | Essential morning cover is lost, an unmanageable protection concern arises or resources are unavailable; use the responsible escalation and continuity arrangements |
| Do not infer wider effectiveness | More visits or positive comments alone do not establish who was excluded, what caused a change or whether long-term outcomes improved |
Record the Evidence That Could Change Your Mind
Before starting, specify the meaningful observations, how they will be recorded safely, the responsible reviewer and the decisions they can inform. Avoid a success rule based solely on a convenient attendance number. A well-attended session may still exclude the people the change intended to reach, while low initial use may reflect an inaccessible invitation rather than lack of need.
At review, compare intended and actual delivery, costs and burdens, access, contrary feedback and unresolved conditions. Preserve what was learned even if the answer is to stop. A pilot report is advice to the authorized owner, not automatic approval for scale. Use the assumption guide to record what changed and the response-options guide to reconsider alternatives.