Detecting The Corruption Of Online Questionnaires By Artificial Intelligence

Our recent paper on detecting the corruption of online questionnaires by artificial intelligence was recently published in the Frontiers In Robotics and AI Journal. We created a short explainer video about our project:

Here is the abstract of the paper:

Online questionnaires that use crowd-sourcing platforms to recruit participants have become commonplace, due to their ease of use and low costs. Artificial Intelligence (AI) based Large Language Models (LLM) have made it easy for bad actors to automatically fill in online forms, including generating meaningful text for open-ended tasks. These technological advances threaten the data quality for studies that use online questionnaires. This study tested if text generated by an AI for the purpose of an online study can be detected by both humans and automatic AI detection systems. While humans were able to correctly identify authorship of text above chance level (76 percent accuracy), their performance was still below what would be required to ensure satisfactory data quality. Researchers currently have to rely on the disinterest of bad actors to successfully use open-ended responses as a useful tool for ensuring data quality. Automatic AI detection systems are currently completely unusable. If AIs become too prevalent in submitting responses then the costs associated with detecting fraudulent submissions will outweigh the benefits of online questionnaires. Individual attention checks will no longer be a sufficient tool to ensure good data quality. This problem can only be systematically addressed by crowd-sourcing platforms. They cannot rely on automatic AI detection systems and it is unclear how they can ensure data quality for their paying clients.

The Dorian Gray Refutation

The limitations of theories

I finally published an article with a title that could be from The Big Bang Theory: The Dorian Gray Refutation. It is an open peer commentary for the article “Social robots as depictions of social agents” in the Behavioral and Brain Sciences journal. Here is the abstract:

Theories are an integral part of the scientific endeavour. The target article proposes interesting ideas for a theory on human–robot interaction but lacks specificity that would enable us to properly test this theory. No empirical data are yet available to determine its predictive power.

New Zealanders respond to strict regulation of artificial intelligence

What a national survey reveals about AI in New Zealand

We just published the article “Personality and demographic correlates of support for regulating artificial intelligence” in the AI and Ethics journal. We analysed data from the New Zealand Attitudes and Values Study survey that collected around 48 thousand responses from New Zealanders. The survey asked them if they support the strict regulation limiting the development and use of Artificial Intelligence. We analysed if the response to this question is related to the respondent’s personality and social economic indicators.

Data revealed that support for strict regulation of AI is positively related with agreeableness, neuroticism, and honesty–humility. However, it is negatively related to openness to experiences. Being female, older, non-European, religious, being single, a parent, living rural, being born in NZ, and living in a more economically deprived region were all related to support for strict regulation of AI. However, how secure one felt in their current job and one’s level of education were not significantly related to their support for the strict regulation of AI.

This study will inform policy makers and technology companies on how New Zealanders feel about AI. You can read the full article for free from this address:

Predicting the robot future

We are still waiting for Able Mable

Predicting is hard, in particular when it is about the future. In 1966 the BBC predicted that it would take 1 million pound and ten years to create the Able Mable. She is capable of letting his lordship in, do all the household chores, and walk the dog.

56 years later our robots look different, but we still promise that their deployment is just around the corner. Just a few more years and we will have our robotic paradise. Some things change, other do not. I wonder if there was person inside the Able Mable robot to move the arms.