Skip to main contentResource added 
To help instill an appreciation for the practical barriers to producing reproducible informatics research and to prepare students to address these barriers in their own work, we taught a project-based seminar for Ph.D. and MS students in biomedical and health informatics. Students chose published works of interest and attempted to reproduce the presented results using the indicated data and methods. Students created an affinity diagram at the end of the course, summarizing barriers encountered and possible solutions across projects.
Even though project selection criteria included high reproducibility potential (e.g., simple/well-described methods, code and data availability), only one out of nine projects was reproducible without significant barriers. Notably, students encountered difficulties that were unexpected in light of only a theoretical understanding of reproducibility issues. As a result, students gained an appreciation for what is required to produce reproducible scientific reports in practice.
Our experience provides evidence for the claim that students of biomedical informatics and other computational disciplines stand to gain critical experiences from attempting to reproduce published work. Thus, hands-on reproducibility projects extend and complement the teaching of the ideological foundation of reproducibility and the technical skills required to produce shareable artifacts.
Teaching Issues of Research Reproducibility Through First-Hand Experience
Full description
Video Presentation
Authors:
- Hannah Burkhardt, BIME/BHI, UW Seattle
- Jake Portanova, BIME/BHI, UW Seattle
Abstract:
A critical topic for the integrity of science, the reproducibility of published scientific work has gained recognition in computational disciplines. However, most classes, seminars, and workshops on this topic focus on the importance of reproducibility from a philosophical perspective or on teaching technical skills needed to create and share the artifacts that support reproducibility. In practice, however, published work is often not reproducible for reasons unrelated to the author’s understanding of the importance of reproducibility or technical abilities.To help instill an appreciation for the practical barriers to producing reproducible informatics research and to prepare students to address these barriers in their own work, we taught a project-based seminar for Ph.D. and MS students in biomedical and health informatics. Students chose published works of interest and attempted to reproduce the presented results using the indicated data and methods. Students created an affinity diagram at the end of the course, summarizing barriers encountered and possible solutions across projects.
Even though project selection criteria included high reproducibility potential (e.g., simple/well-described methods, code and data availability), only one out of nine projects was reproducible without significant barriers. Notably, students encountered difficulties that were unexpected in light of only a theoretical understanding of reproducibility issues. As a result, students gained an appreciation for what is required to produce reproducible scientific reports in practice.
Our experience provides evidence for the claim that students of biomedical informatics and other computational disciplines stand to gain critical experiences from attempting to reproduce published work. Thus, hands-on reproducibility projects extend and complement the teaching of the ideological foundation of reproducibility and the technical skills required to produce shareable artifacts.
Poster PDF
View a PDF version of the poster in Google Drive to enlarge the image or download a copy.
Comments
The presenter for this poster will be available to respond to comments during Poster Session 2 on April 20, 3:45-4:30 p.m.Comments
Log in to view and add comments.
Annotations
No one has annotated a text with this resource yet.
- typeImage
- created on
- file formatjpg
- file size2 MB
- publisherUniversity of Washington
- rights