In recent years, many research fields started questioning how the research was conducted—and reported—as many studies could not replicate (i.e., the ‘replicability crisis’). This open science movement has resulted in substantial efforts to evaluate and improve the credibility of published research. The importance of this movement is also being acknowledged in the field of Human-Computer Interaction (HCI). To increase research efficiency and enable robust scientific findings, researchers must adopt transparent and open research practices, allowing for the replication and reproducibility of their studies and allowing other researchers to extend and build further research from their studies. Furthermore, HCI studies are typically concerned with a broad spectrum of human experiences and activities. Thus ethical research considerations are highly relevant to the community, and the HCI community is becoming increasingly concerned with practices of transparency and ethics in empirical research.
In the PET Lab, we closely commit to Good Research Practices (GRPs) on three different levels: (1) Our own research practices, (2) From Pedagogical Aspect, (3) As a Research Agenda.
(1) As HCI researchers in the PET Lab, we strive to commit to the following practices:
- Transparent Research. We Disseminate our research methodology, including experimental design, materials, data collection, analysis procedure, and procedure of artifact design (e.g., interface design).
- Open Research. We publish our primary and supplementary materials openly and free of charge.
- Research Ethics. We follow the ethical guidelines to ensure scientific integrity, and collegiality, protecting the rights of research participants and enhancing research validity through socially responsible collaboration between science and society.
(2) As a pedagogical effort, Prof. Cherubini (the head of the PET Lab) teaches a 3 ECTS course dedicated to GRP to train Ph.D. students in UNIL and other institutions in Switzerland. This course covers research ethics, scientific integrity, replicability, and reproducibility and is provided every two semesters.
(3) As for our first research agenda, our lab members sought to measure to what extent the HCI community commits to GRPs. This was mainly motivated by many initiatives aiming at improving research ethics, openness, and transparency of empirical studies. In the last few years, we observed several community-led events, research transparency surveys, and opinion pieces aiming at raising awareness of the issue.
Therefore, we investigated whether these initiatives had the desired effect on the community. We focused on the ACM CHI conference as a prestigious and flagship HCI venue. We analyzed a sample of papers published at CHI 2017 and CHI 2022, evaluating them against 45 criteria. We found an improvement overall. But, the findings showed that our community has much room to improve. We highlight these areas that might benefit from further attention from the HCI community. We also developed a proof-of-concept screening system to explore the potential of assisting the verification process. For more details, please check out the paper and watch our presentation at ACM CHI 2023 (23–28 April 2023).
Project Leaders: Kavous Salehzadeh Niksirat, Lahari Goswami, Pooja Rao, James Tyler, Alessandro Silacci, Sadiq Aliyu, and Mauro Cherubini.
UNIL Collaborators: Annika Aebli.
External Collaborators: Chat Wacharamanotham (Swansea University).
Publication
Salehzadeh Niksirat, K., Goswami, L., S. B. Rao, P., Tyler, J., Silacci, A., Aliyu, S., Aebli, A., Wacharamanotham, C., & Cherubini, M. (2023) Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23), April 23–28, 2023, Hamburg, Germany. ACM, New York, NY, USA, 23 pages. Temporary Link: https://serval.unil.ch/en/notice/serval:BIB_364ED171A347 , DOI: 10.1145/3544548.3580848
Videos
