The paper “The Effect of Education in Prompt Engineering: Evidence from Journalists” by Amirsiavosh Bashardoust, Yuanjun Feng, Dominique Geissler, Stefan Feuerriegel, Yash Raj Shrestha has been accepted at AAAI ICWSM 2026!
This study introduces a field experiment with journalists shows that prompt engineering training improves users’ perceived expertise but has mixed effects on accuracy and perceived usefulness of LLMs. It analyzes the effect of training on three dimensions: (1) the user experience of journalists when interacting with LLMs, (2) the accuracy of the texts (assessed by a domain expert), and (3) the reader perception, such as clarity, engagement, and other text quality dimensions (assessed by non-expert readers). Our results show: (1) Our training improved the perceived expertise of journalists but also decreased the perceived helpfulness of LLM use. (2) The effect on accuracy varied by the difficulty of the task. (3) There is a mixed impact of training on reader perception across different text quality dimensions.
