Who’s calling? Unexpected insights from a telephone script intervention

What did we do?

To better understand how to recruit vulnerable audiences, BehaviourWorks Australia (BWA) ran a randomised control trial with SV and two energy retailers.

The aim was to test if a behaviourally-informed phone script could increase interest and sign up among energy hardship customers (one of the eligible householder groups).

‍

Intervention design

The intervention involved an SV employee reading one of two scripts to the 185 hardship customers in the final sample. Both scripts were developed by BWA and incorporated information about the program and the potential benefits.

Additionally, one of the scripts, the intervention script, included the behavioural techniques of scarcity, loss aversion, norms and making future consequences more salient.

‍

What did we find?

Among households who were read the intervention script, 40% signed up to the program and 34% of households who were read the control script signed up. Although this improvement was not statistically significant, it suggests that the additional behavioural techniques may encourage some households to benefit from the program.

‍

Insights

As this was BWA’s first telephone recruitment study with SV, the research team learned some other valuable lessons when it comes to recruiting participants over the phone and setting up trials like this:

• While BWA does a lot of work incorporating behavioural techniques into written communications, designing a phone script is quite different in that, a script needs to read a lot more conversationally and only contain short sentences.
• It can be really difficult to reach participants, especially during office hours when they might be working. This can reduce the sample size of a trial considerably
• Despite the team’s best efforts to anticipate possible circumstances and irregularities during phone calls, it is impossible to anticipate them all.
• If a project requires thorough data collection (like this one), researchers/project designers should allocate sufficient time for queries from the employee making the phone call, particularly at the start of the project.
• The employee undertaking the recruitment obviously needs to possess a good ability to conduct phone calls and stick to a script. In a project like this, they also need good data recording skills. This is obvious in hindsight but wasn’t the first criteria the research team thought of during recruitment. While, in this project, SV recruited a very skilled employee to make the phone calls, this skill should be an explicit requirement for employment.

It should be noted that, with 185 householders, the team had a fairly small sample and this may be one reason why they didn’t find significant differences between the two scripts.

In fact, the sample size may have been too small.

With all the above learnings, the research team feels prepared to run similar trials on a larger scale in the future.