Lyn Carson – Research Director, The newDemocracy Foundation
What is the question?
In 2016, newDemocracy had oversight of a massive public deliberation in South Australia on the extremely contentious issue of nuclear waste storage. It was our biggest project to date. Any journey into the unknown is a rich site for learning, and this project was no exception. Given newDemocracy’s commitment to an action-learning approach, what did we learn from this bold experiment?
Why did we do it?
This project was preceded by a Royal Commission in South Australia. Initial advice to newDemocracy noted two problems for the government: first, that no one will actually read a Royal Commission report, and comment will be left to active interests with polarised positions; and second, that citizens do not trust government agencies as neutral and independent arbiters of community feedback, and assume that their feedback will be ignored. Therefore, a genuine exercise in public deliberation was designed.
The South Australia Nuclear Fuel Cycle Jury was not a typical jury of 20-40 people (See, Process Design). It was an ambitious public deliberation under the imprimatur of the Premier of South Australia, Jay Weatherill. The brief from his government was to design an engagement where the community’s input was visible and central to the final decision by government. Governments too often decide, announce, and defend. This was to be qualitatively different, a rare opportunity for newDemocracy.
What was the process?
The project involved thousands of South Australian citizens, and its size alone tested the limits of the jury model. newDemocracy thinks that our democracy needs radical innovation, and it’s on hard topics where breakthroughs will be made. We could see the benefit of tackling a large-format deliberation because of the potential learning that we could later use — knowing it would also benefit others.
In summary, the deliberative design began by involving 52 randomly-selected citizens in an agenda setting jury. It then reached out to thousands at public events, smaller workshops, and online, culminating in more than 300 randomly-selected citizens (including some of the original 52) forming a final jury for a face-to-face deliberation in the state capital, Adelaide. This research note is mainly focused on the final face-to-face sessions that involved 328 randomly-selected citizens over three weekends, but also documents learnings from the original 52-person jury, and the wider engagement process.
To visualise this large-scale event — in particular, the final jury—imagine hundreds of people seated in the same vast space, other times moving toward allocated rooms, occasionally with television cameras focused on them, chatting over meals, exploring the library of resources, enthusiastically discussing issues along the corridors — the mood was upbeat.
What worked well?
Recruitment
The recruitment for the first jury was highly effective, and for the larger jury was still remarkably free from skews and biases. Participants were randomly recruited to match a demographic profile, not for their attitude to nuclear waste, yet there emerged a comparable proportion of strongly-held views as one would expect to see in the wider community (~5%).
Agenda-setting jury
The initial agenda-setting jury of 52 citizens yielded tremendous satisfaction based on the feedback that newDemocracy received, both from participants and government. They found that jury to be productive and enjoyable. The 328 citizens who participated in the final session were not as satisfied. There are some obvious reasons for this discrepancy — the first jury was smaller, but it also had a non-polarising remit: to examine the Royal Commission report and identify areas of concern that the whole of South Australia would wish to discuss. The first jury attended conscientiously to that task. In contrast, the final jury had a tougher remit.
Final jury
For the most part, the deliberation in the larger jury worked well. The fourth day of the final jury was operationally strong. Working in groups of seven for an hour, jurors shared individual reflections on what they had learned, and then had an enforced mix to new tables (defeating any echo chamber behaviour) and repeated the exercise to share what their last group had been influenced by. This led into 90 minutes of small group generation of best argument for, against or an ancillary statement, which were then clustered by jurors and used as themes for a 90-minute writing task (again in small groups, writing one page on a single point to reflect the intent of the whole jury). The process worked very well, and generated a high-quality draft by the end of that day.
To reiterate, on this very-productive fourth day of the 328 member jury, the process was how newDemocracy would want a large format process to work — two rotations where participants hear and reflect on the views of others, then free writing at small tables. The quality of the draft report then reflected a very successful day and a model that newDemocracy plans to repeat.
At the end of the final session, 328 people from all over the state were sitting together having had an informed discussion about the long-term storage of high-level nuclear waste. This is not a conversation that would otherwise have occurred.
Integrating quantitative research
newDemocracy was pleasantly surprised to see how quantitative research could be integrated with a public deliberation. The market researchers, Colmar Brunton, who had undertaken phone-based and focus group research were in attendance and their databases were opened to any juror who was interested. Jurors were interested in the thoughts of other citizens. Market researchers were on hand to help or resolve specific queries, and they demonstrated an ability to connect feedback from the wider population with the activities of the jury.
Insightful final report
While the final report was not of the usual form or quality (due to significant timing errors on the final two days), it still managed to show genuine insight beyond that which commentators, experts or politicians showed. There was content that was linked to evidence that had not emerged from any other format — not the $7.2m Royal Commission, nor the Parliamentary Inquiry, or the 40,000 online comments — and ultimately could have informed the government’s course of action. Participants showed clear and critical thinking, delivering that to the government. This was aided by the critical thinking exercises that newDemocracy has developed, in the company of a number of skilled facilitators (See, Importance of Facilitation and Critical Thinking).
Credibility of a large jury
A frequent media criticism of juries of 20-50 people is “how can a small group possibly be representative?” With more than 300 people, the final jury was a visible swathe of the entire population from all around the state. Having the larger jury made a noticeable difference in media coverage (See, InDaily Coverage).
Not a rubber stamp
newDemocracy demonstrated that a citizens’ jury is not simply a rubber stamp for government. Should a government choose to undertake something similar for other difficult conversations or contentious policy matters, that government would be able to point to the nuclear jury as clear evidence that the outcomes of a newDemocracy process are beyond the control of government, and that the jury’s recommendations won’t necessarily accord with a government’s current beliefs.
What do we wish had worked better?
There is always room for improvement. newDemocracy’s big goal is that everyone leaves feeling a sense of ownership over the whole report, and we did not achieve that. Here are just a few of the areas for improvement, for the final jury as well as the wider engagement.
Jurors expression of protest
An issue for newDemocracy is the perhaps-inevitable recruitment of activists when an issue is as contentious as nuclear waste storage. This led to the nightmare of the ‘red dots’— dots designed to signify NO — which began to appear on name tags. A faction emerged within the jury in the later stages, and this faction wanted to make a statement of intent. This is entirely counter to the principles of deliberation and respect for others’ views (See, Deliberation). It said to other jurors “I’m not going to in any way change my mind no matter what alternative arguments are presented”. As the process unfolded a notable number of jurors began to feel intimidated by this collective (newDemocracy noted approximately 30 comments). Juries work because random groups typically do not factionalise into blocs. Doing so is akin to sitting on a criminal jury with a “GUILTY” pin in one’s lapel.
Accusations against the organiser
Another area of concern is the extent to which an organiser can intervene to correct misinformation, given that we offer critical thinking exercises and are relying on participants to exercise critical thinking at all times. Here’s an example of a dilemma that we faced. and it goes to the issue of an organiser’s integrity: to be hands-off or involved.
Participants reported aspersions directed at newDemocracy and its founder, Luca Belgiorno- Nettis, whose family foundation funds what newDemocracy does. The accusation was that he had energy interests through Transfield Holdings. He did not. For the record, Transfield Holdings was Australia’s biggest investor in Concentrated Solar Thermal Power Technology (through Novatec Solar, a company based in Germany) but exited the investment in 2014 (See, Further Information below). A large solar operator, had this investment been extant, might have had a bias toward the ‘No’ argument, following the line of advocacy that money spent exploring the nuclear opportunity could instead be spent in renewables. How this was turned into “manufacturing consent” (for ‘Yes’) was illogical, but newDemocracy was loathe to intervene and did not do so. We remain unsure if that was the right decision.
Timing
Time is of the essence in a large-scale project. newDemocracy does not believe that we got the timing right—a very common problem in public deliberations (and also for parliamentarians, who often find themselves debating until the early hours). In particular, this project had several problems with timing.
The 328-person jury was run over six days, on three weekends, on both Saturdays and Sundays. Running on both days of a weekend is not ideal. It was essential given the narrow window of available time, coupled with the cost pressures of having 20% of participants travel from regional South Australia. As a result, there are downsides, such as lower response rates to invitations (this affects the diversity in the room), less time for reflection between meetings (which can make deliberations shallower), and jurors’ perception of being rushed (undermining their willingness to trust the process). It also makes it more difficult to correct any flaws in process design or on-the-spot decisions about the agenda. newDemocracy now considers that both-day weekends should be an absolute last resort and should not be used more than once during the life of a project.
When sessions run late, there is a domino effect that can damage the overall deliberative design. This happened on the penultimate day, and it cut into the important time devoted to writing a draft in preparation for the last sessions. This made life difficult for facilitators and participants alike.
The issue of time was also evident in the wider context. Following the release of the Royal Commission report, the government agency was given six months to deliver a state-wide engagement. Ideally, a project of this size would demand a time frame of two years. The truncated time frame the project was given was driven by the realities of political timing. The South Australian Government was faced with the urgency of a jobs and economy crisis. Taking two years to discuss a project that would be a massive economic driver (were the community to support it proceeding) would be poorly publicly received as failing to prioritise the need to get people into jobs in the short-term. In this context, there are downsides to all decisions.
Some issues could benefit from project timelines which deliver a recommendation to the parliament after the next election. However, governments will only commission deliberative processes if they are both good politics and good policy. The act of delivering a benefit after the election erodes the benefit as the desired political positioning is to be able to say “we dealt with a difficult issue by having citizens as partners in problem solving” — the issue won’t have been dealt with, so a key campaign selling point is removed.
Relationship between newDemocracy and Advisory Board
The normal operating relationship would involve a government agency retaining newDemocracy and valuing the independent final oversight of the process. The Advisory Board members were tremendously experienced with criminal juries, medical and market research — but critically not with deliberative democratic models. In a feat of timing, the first meeting for newDemocracy with Advisory Board members was on the morning of the first citizens’ jury. This meant that the Advisory Board lacked context of participating in planning discussions with representatives of the Premier’s Office as they occurred before the Board’s formation which would have resulted in newDemocracy’s prior experience being valued. Ultimately, newDemocracy did not have practical ‘final call’ oversight during this project. There were too many layers of responsibility, all wanting to act responsibly.
Information for the jury
newDemocracy’s original advice on the overarching Engagement Strategy emphasised a need for three things in the wider community engagement phase being rolled out to all South Australians: (1) high quality strategic communications advice; (2) avoidance of one- sided presentations and the importance of not “selling” a particular outcome; (3) asking questions (through collateral such as Kitchen Table Discussion kits) rather than flat statements which people could rightly reject if they conflict with their innate beliefs. Two key principles of deliberation are to offer an open question (rather than selling an answer) and to encourage review of multiple, diverse sources. Traditional public sector culture made adhering to these two principles rare, as evidenced by much of the public collateral distributed by the agency. This contributed to a low level of general public trust for the incoming 328 jurors, who felt that the government was pushing something onto them. It also led to increased frustration among the facilitators when they were shown examples of collateral as “proof” of a pre-determined outcome. The collateral made this line of argument plausible, undermining the process and the Premier’s reassurance, when he attended, as inconsistent with the available hard evidence.
Citizens expected to hear from government “What are you planning? Show me. What will it mean?” One would expect to see the big picture and have various components explained. However, the initial baseline information presented on the first day fo the larger jury instead sought to explain the Royal Commission process rather than give citizens an understanding of what would ultimately be built — a stadium-sized concrete bunker several hundred metres underground, the types of technologies being used, and the radiation at ground level. It is understandable that speakers focused on the Commission’s process, in the context of activist attacks on the integrity of the process. However, this emphasis placed considerable time and information pressure on subsequent activities that were dependent on this baseline information. A deliberative design will generally prioritise broadening the range of sources, and do so by asking the citizens what questions they have as a result of the initial presentation. Instead, these questions became based around a lower than expected level of information.
What do the results mean for the practice of deliberative democracy?
Challenges of large face-to-face groups
There needs to be much more attention paid to the most efficacious method of working with large face-to-face groups on complex topics. newDemocracy considers that a process like this one works best with six full days, with each spread approximately three weeks apart to allow time for reading and reflection and multiple rounds of involvement by expert witnesses. The original designer of citizens’ juries, Ned Crosby, has been firm about a five- day process involving consecutive days (Crosby & Nethercut, 2005: 114). Our experience suggests that modifications improve deliberative designs and are consistent with other Australian adaptations (Carson & Hartz-Karp 2005).
Dividing large face-to-face groups into separate smaller groups, then integrating them
The operation of the 52-person first jury was predictable and cohesive. As group size increases, newDemocracy is aware that incentives to read and work with others decline. Large-scale projects such as Deliberative Polls are more survey-oriented and do not require the depth of reading that is essential for a citizens’ jury (Fishkin, 2011). newDemocracy has previously operated juries of 100 and 150+ successfully. With 20:20 hindsight, we consider that we “should” have run this as seven 40-50 person local juries for three or four days, then brought those seven juries together for a final two or three days (ideally separated — not over a weekend). newDemocracy never offered government this advice — this is simply what we think after the project.
newDemocracy believes that the benefit of smaller, separate juries would have been to create greater group dialogue and cohesion. It would have allowed for diversity in selection of expert witnesses, and deeper discussion at the 50-person level (which was a valid criticism of the dynamics of the larger group). After this, jurors would have brought summaries of evidence they relied upon to the larger group. Perspectives could be shared using the large- format exercise that worked so well for the final jury.
Using a separate agenda jury
newDemocracy has convened simultaneous juries before (with staff members and citizens in separate juries). However, rarely do we see an agenda-setting jury convened. More often, a steering committee will be assembled to set the agenda. This could be a very fruitful area of future practice and evaluation.
Integrating quantitative research
Referring to the successful integration of quantitative research (mentioned above), this is an integration worthy of repetition. Since citizens do not trust governments as independent arbiters of mass feedback, this is an effective format to practically deliver citizens’ input. Having the researchers on hand, and making all data available, definitely strengthens trust in the collection and reporting of public opinion.
Working with government staff
The Premier, Jay Weatherill, is an advocate for deliberative democracy (Thompson, 2016) and has been influenced, in particular, by Daniel Yankelovitch’s work on dialogue and coming to public judgement (Yankelovitch, 1991, 1999). This does not mean that everyone in the government agrees with the need for public deliberations. Government employees who are charged with delivering projects such as these will vary from enthusiastic to obstructive. We can keep wishing this was not so, but the issue is not going to go away, and it has an impact on all government-supported projects, as it did with this one. An elected representative’s passion is not necessarily congruent with all staff or vice versa. newDemocracy spent considerable time briefing government staff but something more is needed. This is a very fruitful area of research and to some extent has been investigated by a research project that newDemocracy funded (See, Riedy & Kent 2017). We know what can happen—but we need to understand how best to overcome this impediment to good practice.
What remains unresolved?
Indigenous participation
For the first jury, the random selection process resulted in three indigenous participants, but for the final jury we believe there were only four. In any random sampling exercise this number will ebb and flow on each project. It continues to be a difficult aspect, as questions about Indigeneity are not answered honestly and we have no means of checking the accuracy of anyone’s answers. While newDemocracy uses other methods to have Indigenous conversations and input in order to supplement this weakness, we would like to find better ways of ensuring Indigenous involvement.
Activists in juries
The Nuclear Jury pre-dated newDemocracy’s work with personal (or cognitive) biases, and it was a catalyst for our thinking in that area (See, Critical Thinking). In a public deliberation, a brake must be applied to judgment — not to suspend judgment, but to delay it — long enough to take in contrary views. If one arrives with a pre-determined view, all sorts of biases will arise (for example, confirmation bias), and this will impede learning new information. Worse, time will be dedicated to persuading others to this pre-existing view and this pre-occupation with attracting others to a particular viewpoint defeats our ability to interrogate all knowledge (See, Hartz-Karp & Carson, 2013).
Accusations of bias on the part of organisers
Another problem can arise: accusations against the organisers as ‘manufacturers of consent’, even though the facilitators are merely trying to ensure that all views are canvassed. It might also be that such a large group, convened over many weeks, is not ideal. The online environment, for example, can exacerbate this predilection.
How to enable good deliberation online
Online environments can become toxic when strong opinions are presented as the only acceptable or credible opinions. Those with contrary views are howled down and leave the site. When vitriol forces online contributors to vacate the site, it can leave an agglomeration of like-minded people with few opportunities for learning — again, anathema to deliberation. Moderating the site, asking contributors to curb their behaviour, can inflame those with strong views and, once more, can be seen as organisers ‘manufacturing consent’. newDemocracy is interested in exploring other, more deliberative, online environments.
A word about acknowledging challenges and learning
These reflections and more are part of newDemocracy’s commitment to reflect on practice in the company of others who share a commitment to good practice, and this includes the many randomly-selected citizens who have participated in public deliberations. Participants give newDemocracy so much of their time. This research note is one way to offer insight into the challenges we encountered in a particular project as well as an admission of sometimes- flawed, but well-meaning, decisions that were made along the way.
Further information
For details of newDemocracy’s funding disclosure, see: https://newdemocracy.com.au/who-we-are/governance/funding
For Transfield Holdings Investments, see:
http://www.transfield.com.au/investments/current-investments-portfolio
References
Carson, L. & Hartz-Karp, J (2005) “Adapting and Combining Deliberative Designs: Juries, Polls, and Forums”, in Gastil, J & Levine, P (eds) (2005) The Deliberative Democracy Handbook: Strategies for Effective Civic Engagement in the Twenty-First Century, Jossey-Bass, pp.120-138
Crosby, N. & Nethercut, D (2005) “Creating a Trustworthy Voice of the People”, in Gastil, J & Levine, P (eds) (2005) The Deliberative Democracy Handbook: Strategies for Effective Civic Engagement in the Twenty-First Century, Jossey-Bass, pp.111-119
Fishkin, J.S. (2011) When the People Speak. Deliberative Democracy and Public Consultation, Oxford University Press
Hartz-Karp, J. & Carson, L. (2013) ‘Putting citizens in charge: Comparing the Australian Citizens’ Parliament and the Australia 2020 Summit’, in Carson, L., Gastil, J., Hartz- Karp, J., & Lubensky, R. (Eds) The Australian Citizens’ Parliament and the future of deliberative democracy. University Park, PA: Pennsylvania State University Press, pp. 21-34
Riedy, C & Kent, J (2017) “Systemic Impacts of Mini-Publics”, Research report prepared for
The newDemocracy Foundation, https://www.newdemocracy.com.au/research/research-papers Accessed 11 July 2017
Thompson, N (2016) ‘1.3 A Politician’s Perspective with Premier Jay Weatherill”, Real Democracy Now! A Podcast, published Oct.17, (http://realdemocracynow.com.au/1- 3/)
Yankelovitch, D. (1991) Coming to Public Judgment: Making Democracy Work in a Complex World, Syracuse: Syracuse University Press.
Yankelovitch, D. (1999) Magic of Dialogue, Allen & Unwin.
Lyn Carson 
This paper draws lessons from newDemocracy’s experiences the South Australia Nuclear Fuel Cycle Engagement Project.
* newDemocracy is an independent, non-partisan research and development organisation. We aim to discover, develop, demonstrate, and promote complementary alternatives which will restore trust in public decision making. These R&D notes are discoveries and reflections that we are documenting in order to share what we learn and stimulate further research and development.
