Earlier this year, the government launched its ‘AI Opportunities Action Plan’, which heavily emphasised the potential of AI to ‘unlock data assets in the public and private sector’. The curation of a ‘National Data Library’ aims to foster research, innovation and development within the UK by making public and private sector data more accessible.
An area where there is a wealth of underutilised public data is public opinion data. Public opinion data is essential for shaping policies and engaging with citizens. However, its reliance on open-ended responses makes it both time-consuming and expensive to analyse, often leaving valuable insights unused.
To combat this issue, the University of Bath has announced its latest AI tool: DECOTA (Deep Computational Text Analyser). Where thematic analysis traditionally requires human expertise to manually identify patterns and themes in free-text data, the tool employs a large language model which can analyse responses 380 times faster than manual methods. The cost of human-coded analysis can be as high as $1,500 (£1,300), but DECOTA costs as little as $0.82 (£0.62) per 1,000 participants.
As well as being faster and more cost effective than manual thematic analysis, it delivers results with 92% agreement to human accuracy. One of DECOTA’s key features is its ability to detect both broader themes and finer sub-themes. For example, being able to determine which demographic groups are more likely to bring up certain themes.
DECOTA was initially developed to analyse public feedback on climate policies, however, its applications extend beyond climate initiatives, offering transformative potential across various sectors. By processing vast amounts of free-text data within minutes, DECOTA will enable organisations to identify actionable insights that would otherwise take months of manual analysis.
Professor Lorraine Whitmarsh, who helped develop DECOTA, says: “DECOTA offers a huge leap forward in the analysis of open-ended questionnaire data. Applying machine learning to analyse large volumes of text will save time and money for researchers and policymakers wanting to understand public attitudes, allowing for a stronger role of public engagement in policy design.”
Whilst the tool provides an exciting opportunity to unlock important public data, it is important to acknowledge that human oversight remains crucial to ensure the data analysis captures cultural contexts, emotional undertones, and broader societal implications that AI might miss. Rather than replacing the need for human analysts, those behind the tool hope it will enable greater research.
Recognising these issues, Dr Player emphasises that DECOTA is intended to enhance human research efforts rather than replace them.
“Importantly, DECOTA is not designed to replace human thematic analysis, but rather complement it,” explains Dr Player. “We want it to unlock the huge volumes of data going unanalysed, allowing more voices to be heard in policy and decision-making settings, and freeing up valuable researcher time for deeper, more interpretative work.”
The launch of DECOTA marks a significant step forward in unlocking public data assets, aligning with the government’s AI Opportunities Action Plan. This innovation will undoubtedly contribute to greater research, more inclusive policymaking, and a deeper integration of public perspectives into decision-making processes.