Voluntary Consent
You are being asked to volunteer for a research study. It is up to you whether you choose to participate or not. There will be no penalty or loss of benefits to which you are otherwise entitled if you choose not to participate or discontinue participation.
Purpose, Procedure, and Duration
We are researchers from Colorado State University (CSU), Department of Computer Science, inviting you to participate in a code review study. We want to learn how developers comprehend and evaluate source code that was written by a person versus code generated by an AI tool.
If you agree to participate, you will be asked to review three coding questions, one at each of three difficulty levels (easy, medium, and hard). Each question is presented with two solutions to the same problem. For each solution you will rate qualities such as readability and understandability, judge whether it is high quality, and briefly explain why, and you will answer a comprehension check for each question. After seeing both solutions you will choose which one you would approve in a real code review and record a short spoken explanation of your reasoning. One of the solutions in each pair was written by a person and one was generated by an AI tool; which is which will not be revealed during your review.
Your review will take about 20–30 minutes to complete. About 100 people are expected to take part.
Eligibility
You must meet the following requirements to participate in this research study:
- You are 18 years of age or older and have programming experience through coursework, personal projects, open-source contribution, or professional software development.
Risks
Your participation in this study does not involve a potential risk of physical or emotional harm beyond what you would encounter in daily life.
A portion of the review asks you to record a short spoken explanation of your reasoning. This response is spoken only and cannot be typed. Your speech is sent to Deepgram, a third-party transcription service, which converts it directly into text. Under an agreement with Deepgram, your audio is not used or retained by Deepgram for its own purposes. Your audio recording is not retained or stored by the research team, and only the resulting text transcription is kept for analysis. Every effort will be made to safeguard your data.
Benefits
You may not benefit personally from being in this study, but your answers could help improve understanding of how developers comprehend and evaluate source code during review.
Compensation
You will not be paid for participating in this study. You may optionally enter a drawing for a $100 gift card. Five gift cards will be awarded among all entrants, giving you an approximate 1 in 20 chance of winning. To enter, you will be given a separate survey on the final page, after you have finished the review, where you can provide an email address. This separate survey is stored apart from your study responses, so your email address is not connected to your answers. Your email is used only to notify and deliver the prize to winners, and is permanently deleted after the drawing. You may complete the review without entering the drawing.
Entry into the drawing is conditioned on completing the review with valid answers for all 3 question reviews.
Confidentiality and Future Use
Your responses are kept confidential. Your responses are labelled only with a randomly generated ID, not your name. If you choose to enter the gift card drawing, your email address is collected through a separate survey and stored apart from your study responses, with no link between the two. Your name, internet (IP) address, and other identifiers are not collected. Your responses are stored on CSU-approved systems accessible only to the research team.
After the study is complete, we may publish the anonymous responses in a public research repository so that other researchers can verify and build on our findings. The published data will not include your email address, your name, or anything else that identifies you.
Complaints or Concerns
For questions, concerns, or complaints about the study, you may contact Fabio De Abreu Santos, Computer Science Scholar, fabio.deabreusantos@colostate.edu, (970) 491-8791.
If you have any questions about your rights as a volunteer in this research, contact the CSU IRB at CSU_IRB@colostate.edu.
Thank you for taking the time to consider our study. You do not have to participate in our study, but we hope you will.
Please select an option below to indicate you read this information and you wish to take the survey:
For the best experience recording your spoken answers, please use Google Chrome. Other browsers haven't been tested for this study, and voice recording may not work reliably.