Publicly funded organisations work to improve the value they deliver over time by testing promising new ideas in small-scale, low-cost pilots. If a pilot demonstrates a superior new approach, then it can be implemented at scale, often with resources transferred from less effective alternatives. If not, the pilot should be abandoned. New ideas, however, can gather momentum and backing in testing for even unsuccessful pilots to remain in flight indefinitely.
Pilots make innovation feasible for publicly funded organisations by testing the effectiveness of new policy ideas at relatively lower cost, and lower risk, than full-scale implementation. Real-world trials, often limited to one geographical area or subset of the target demographic, also offer useful insights to guide wider roll-out. Many pilots can be trialled at one time, accelerating collection of a broad evidence base and informing decisions about ongoing initiatives.
The people involved in a pilot, both participants and providers, tend to perceive the positive features of an intervention, and often advocate for the pilot to continue or expand. These advocates, however, seldom have visibility of the larger policy landscape and the relative or opportunity cost of the initiative.
Most pilots deliver some benefits and individual beneficiaries tend to judge the value of the intervention on their own terms, rather than by comparison with alternatives that deliver more benefits more efficiently. Passionate advocates can distort decision making by creating pressure for a pilot to remain in flight, even if it is outperformed by the status quo or other pilots.
Pilots that continue even when they have been demonstrated not to deliver superior value consume resources that could be better deployed elsewhere. The cost of retaining ineffective or inefficient initiatives reduces an organisation’s capacity to implement stronger alternatives, or experiment with other promising ideas. Pilots that go on for too long impose immediate costs and reduce future public value.
Pilots need to be transparent from the start about how their performance will be evaluated, including the threshold to be considered successful and how opportunity cost will be valued. A clear timetable should be set for evaluation and decisions about either discontinuing a pilot or transitioning to wider rollout.
Being transparent about assessment criteria, timing, and funding also helps to set clear expectations for beneficiaries and other stakeholders, making it harder for special interests to overrule or influence decisions that should maximise overall value for money.
Showcasing competition can further reinforce that only the most successful pilots will continue, perhaps with funding from less successful pilots being explicitly allocated to fund wider rollout of the most successful.
Managed well, pilots can reduce the inherent risks of trialling innovative approaches to a policy problem. Clear evaluation criteria and comparisons between pilots and the status quo help to identify the best value for money public investments.
Stakeholders with advance notice of transparent and fair criteria are less likely to hijack the process, and less able to distort later decision making through advocacy in their own interests. Less successful pilots can then be abandoned, as they should be, to focus on delivery of more effective and efficient alternatives.
When only initiatives with superior outcomes and value for money attract funding and support to expand or continue, then effectiveness and efficiency improve over time, lifting public value to a higher plane.