Can we have a quiet word about behavioural science?

While we agree with some of the concerns expressed by the author, as a collective of researchers who, at times, use tools from behavioural economics (along with many other areas), we felt compelled to respond. Especially to the criticisms that cast a broad shadow on the use of behavioural tools to improve the lives of individuals or society at large.

A toolkit for opening the black box

To start with, some arguments we agree with. It is true that there’s a black box approach in a lot of the application of behavioural economics.  The suite of tools that can be found in behavioural economics tend to work a little bit on large audiences.  Typically, these tools are in written form and are sent out en masse and they’ve been used to increase, by a small percentage, the number of people doing things like saving for retirement, getting vaccinated, using less energy and water, donating organs, and applying for jobs they otherwise would not have applied for.  These tools, and the experiments which test them, are often unconcerned with the mechanism for change, but rather focus on whether and to what extent change happened.

This means, as the author correctly states, that if they don’t work we are limited in our ability to know why. This can then lead to assumptions about the mechanisms for change and, in some cases, an inability to replicate results. To address this, BehaviourWorks Australia recommends a Deep Dive, particularly for important or significant behaviours, to understand drivers and barriers to change through interviews with people in the target audience.  Doing so may well have prevented some of the issues identified by Commissioner Holmes in her report on the failings of the robodebt scheme.

Let’s get ethical

Another point made in the article that we agree with, but perhaps not to the same degree, is that the use of behavioural economics in the case of robodebt was unethical. At BehaviourWorks Australia we have been involved in projects encouraging people to make payments (such as fines and vehicle registration), some using nudges, but this work has always been done in a context where it was clear that money was owed or payment due. The article appears to conflate the ethics of asking people to pay a debt that wasn’t owed (which is unethical) with the use of behavioural economics (which isn’t inherently unethical). Frameworks such as the FORGOOD framework are an easy way to assess whether a behavioural intervention is ethical. Furthermore, Richard Thaler (an author of ‘Nudge’), states that nudges should only be used when they are transparent and not misleading, they are easy to opt out of and they improve the welfare of those being nudged. 

One could argue that the second point raised by Richard Thaler was breached by designing a scheme making it hard for people to seek help (robodebt letters had no phone numbers for people to speak to someone). However, the most egregious breaches were not the fault of behavioural economics. Should those in charge have questioned the ethics of nudging vulnerable people to pay a large sum of money? YES. The blame lies at the feet of those that designed and approved an automated system to enable the identification and contact of welfare recipients in the first place (which was, as they were advised at the time, both unethical and illegal).

BETA testing

What we find concerning about the article is that it may cast a shadow on the broader application of behavioural science in policy. Contrary to what the author seems to be arguing, there certainly are places for nudge-type tools and as the author points out, there are dozens of examples on the BETA site where simple, one-off behavioural nudges have resulted in better outcomes for the community. Perhaps the best argument for this is how much easier it is to fill out a tax return now than it was 20 years ago - the product of one of Australia’s oldest nudge units in the ATO. Across the numerous nudge units globally, there are thousands of other cases which have helped without harming and shouldn’t be ignored.

Of course, nudges, and more broadly behavioural economics tools, aren’t always the right ones to use. As we’ve suggested, where behaviour is significant or important, then talking to target audiences, forming ideas about how change can happen, and systematically testing and evaluating these ideas (including for unintended consequences) in early trials is a better approach. These processes are outlined in the BehaviourWorks Method, which can more appropriately be considered a suite of tools that sit in the behavioural sciences (as opposed to its more focused cousin; behavioural economics), and which advocates for consultation with target audiences, testing and evaluations that consider unintended consequences.

Before we go

One final point we’d like to make is on concerns expressed about the formation of a new evaluation unit. The author suggests that this unit’s purpose is to find out what works, which we would broadly agree with. But this is a good thing. So much government (and other) expenditure is spent on programs without ever knowing if they're really effective, or what unintended consequences may result. 

A well-established evaluation unit would seek to do the necessary work. That includes establishing ‘lead’ and ‘lag’ indicators of success (including the unexpected), testing and trialling initiatives and providing rapid feedback to allow program adjustments. Most importantly, it would consider the counterfactual - “what would have happened if we didn’t run the program or alternative forms of it?”, which will allow program delivery agencies the ability to learn before it’s too late. 

If such an evaluative unit existed before the robodebt scheme, and were engaged by those involved, we’re quite sure they would have run some early trials and noticed issues with unintended outcomes pretty quickly!

‍

Written by Liam Smith, Director BehaviourWorks Australia

‍