Startseite > Fachgruppen > Secure Software Engineering > Lehre > Vorlesungsarchiv > SS 2021 > Seminar Replicability of Software Engineering Artifacts SS 2021

Seminar Replicability of Software Engineering Artifacts SS 2021

Course number and language

This is course L.079.08016

The teaching language will be English. Informal communication in German is permitted.

Registration and topic assignments

If you would like to take part in a seminar in Summer Semester 2021, you must register for the course "All Seminars Summer Term 2021". The course captures the registration for ALL seminars.
Further information can be found on the Website or
See also:
Zuordnungsverfahren für Projektgruppen/Seminare
Matching method for project groups/seminars


After passing this assignment procedure, you will be able to participate in this seminar.

Panda page

We provide a Panda page for this course.

The panda system will be used for:

  • Providing material, e.g., slides or templates
  • Handing in Assignments
  • Communication

You will be added to the panda course automatically after getting assigned to this seminar.


In this seminar, students will learn basic skills on how to conduct scientific research, which will be useful in their studies and further academic career. They will learn how to perform a replicability study, document their work, write a report, and present their findings.

In particular, this seminar will follow the ACM classification on availability and replicability of artifacts published at major software engineering conferences such as ICSE, ESEC/FSE, ASE, MSR and ISSTA. Artifact evaluation in the software engineering community was intoduced in 2011 at ESEC/FSE and since then has been added to many conferences in order to increase awareness for the importance of artifacts created by researchers. However, recent study has shown that the criterias and processes are still heterogenious and further standardization is needed (check the study on this video).

Students will perform a replicability study on a given set of already published artifacts. Their main task is to verify if the artifacts are still available, read the publications, follow the documentation of the artifacts to replicate the results, and report the findings.

The seminar consists of 3 phases.

  1. Reading lectures - four online sessions (one per week) in which students will read and discuss relevant publications on artifacts evaluation to gain knowledge on the topic and relevant skills to perform the replicability.
  2. Replicability - independent work in which each student is given a set of publications with artifacts. Students will document their independent work in a lab report.
  3. Report and presentation - students will write a final report based on their experimental work documented in the lab report from phase 2. Additionally, they will prepare a 10 minutes presentation.

There will be several intermediate duties, all of which are prerequisites for passing this course! A detailed plan will be published on this website and within Panda.


Mandatory sessions

The seminar will be comprised of mandatory sessions. In the remainder of the semester, students will work on their research topic independently, and refer to their mentors when necessary.

This course will be held as an online course!

Attendance is mandatory for all students registered to the seminar.

The sessions are on the the following days (these are preliminary; confirmed dates by the beginning of summer semester):

  • Meeting 1 (April, 13th at 13:00): Organizational matters
  • Meeting 2 (April, 20th at 13:00): Reading lecture 1
  • Meeting 3 (April, 27th at 13:00): Reading lecture 2
  • Meeting 4 (May, 4th at 13:00): Reading lecture 3
  • Meeting 5 (May, 11th at 13:00): Reading lecture 4
  • Meeting 6 (July, 29th at 9:00): Online block seminar

During phase two there will be one optional Q&A session for students.

Learning outcomes:

  • Develop and improve your writing and presentation skills.
  • Learn how research artifacts are created and evaluated.
  • Develop your communication skills and critical thinking. Learn how to give a constructive feedback to your peers.


  • Be registered in the Master program at UPB.
  • Be registered to the course "All Seminars Summer Term 2021" on PAUL and getting assigned to this course after the application procedure for seminars.