User Study: En­han­cing Se­cure Cod­ing Prac­tices

Are you an experienced developer passionate about secure software development? We need your expertise! 

We are researchers from the Secure Systems Engineering lab at Paderborn University, working on advancing secure software development practices. We have developed SecAI, an innovative tool designed to help developers write and understand secure code. SecAI leverages static analysis and LLM to generate, analyze, and explain secure coding practices.

We are conducting a user study to evaluate SecAI. If you have extensive experience in security-focused development, we invite you to participate in our study and contribute to shaping this novel approach to secure coding. Participation is voluntary, and you can withdraw at any time. 

 

Information about the SecAI study

In this study several technoloigies will be used, including Large Language Models (LLMs) and Static Application Security Testing (SAST).

LLMs are part of recent Machine Learning approachs and have shown significant promise to generate code in recent developments, but they lack in terms of explainability for their decision making. LLMs receive input prompts, process the received data, and generate output. Further data processing (e.g., for training the LLM) depends on the provider of the LLM.

SAST tools have been effective in detecting code issues but are limited by false positives due to its nature and the complexities to resolve them.

Duration: The study will take approximately 60 minutes. 

Tasks: Participants will perform Java-specific coding tasks related to security with and without using SecAI, followed by a questionnaire and an interview to discuss their experience and assess the tool’s impact.

 

Data Collection and Processing

  • The study will be held in an oline setting (online meeting) where you get access to virtual machines to work on the tasks.
    • Meeting recordings (screen and audio) can be made for data extraction, e.g., how tasks were solved, creating transcripts of discussion.
    • When working with SecAI, logfiles wiil be stored.
  • After completing the tasks, participants will complete a questionnaire.
  • Your data will be anonymized, we may use short anonymized quotes at the most.

The user study design was approved by the Ethics committee of Paderborn University.

 

Data Handling:

  • No names or other identifying personal information will be collected. Recorded data will be anonymized and screen/audio recordings will be deleted after anonymization.
  • All data will be anonymized and anonymized data (e.g., anonymized transcripts of the recording are stored, but the recording is deleted after anonymization, latest at the end of 2025) stored at Paderborn University. 
  • Participant codes (pseudonyms) (e.g., P01, P02) will be used in reporting results. 
  • Study findings may be published, but identifying information will not be included.
  • Participants will sign a suitable consent form before the user study is conducted.

Please read the privacy policy for this study which was approved by the Data protection office of Paderborn University.

 

How to Participate

Please register for the study here: Registration (https://umfragen.uni-paderborn.de/index.php/261669?lang=en)

business-card image

Michael Schlichtig

Secure Software Engineering / Heinz Nixdorf Institut

Write email +49 5251 60-6580