Conference Survey FAQ


  1. Who are you?

    We are Christian Collberg and Todd Proebsting from the Department of Computer Science at the University of Arizona.

  2. Why are you doing this?

    In order to repeat, reproduce, or extend published computer science research, we need, at the very least, access to the code and data (the research artifacts) that were used in collecting the results of the original paper. Experience has shown that gaining access to research artifacts can be difficult, particularly when a few years have passed since publication. A related problem is being able to contact the authors of a paper to enquire about some issue, given that email addresses change over time. We want to encourage sharing in the computer science research community. Please read our CACM paper and technical report which describe our previous experience with sharing of research artifacts.

  3. What are the goals of this work?

    1. To study the current practices of top conferences with respect to how they encourage or require artifact sharing;
    2. To gauge the future of artifact sharing by investigating the attitudes to conference chairs towards artifact sharing.

  4. Who is funding you?

    We have received a grant from LJAF.

  5. Do you have IRB approval?

    Yes, we have University of Arizona Institutional Review Board (IRB) approval.

  6. What, exactly, do you mean by a research artifact?

    A research artifact is any code or data produced in the research that leads to the publication of a research paper which did not make it into the published paper itself, but which may be useful when assessing or extending the work.

  7. How does this study work? What is the protocol?

    We collect a list of program chairs from computer science conferences. We also read the call for papers and websites for these conferences to extract what we believe to be the policies (if any) with respect to artifact sharing and reviewing. We send a first email to the chairs, asking them to verify that this information is correct and to categorize their policies via an online survey. We send a second email based on the information gathered in the survey, asking the chairs to project the course of artifact sharing in their fields of expertise into the future. The responses will be anonymized. We expect to write up the results for publication.