LLM-Augmented Java Program Analysis via Sootup Integration
Thesis type: Master Thesis
Language: English
Description:
Large language models (LLMs) have demonstrated impressive capability for local, syntactic reasoning about code, but sometimes fail at tasks requiring non-local, whole-program reasoning — resolving dynamic dispatch, tracing inter-procedural data flow, and answering reachability questions. These are precisely the questions that whole-program static analysis frameworks such as SootUp can answer with formal precision.
This thesis investigates whether equipping an LLM with on-demand access to SootUp-backed static analysis — exposed via a structured tool-calling interface — leads to measurably better outcomes on downstream Java program analysis tasks such as bug detection and call-graph question answering.
The study uses a controlled three-condition experimental design: (A) LLM with source code only, (B) LLM with pre-injected static analysis results, and (C) LLM with on-demand tool calling to SootUp. All conditions are evaluated against established benchmarks with known ground truth, providing a rigorous empirical answer to the question of whether and when static analysis augmentation adds value to LLM-based code reasoning.
Requirements:
Interest in software security and static program analysis [required]
Solid Java or Python development skills [required]
Willingness to work with large existing codebases and toolchains [required]
Prior knowledge of SootUp, Soot, or call-graph analysis [optional]
Familiarity with MCP (Model Context Protocol) [optional]
Previous attendance of DECA1 and/or DECA2 [optional]
Tasks:
Survey existing MCP servers and tool-calling interfaces for program analysis; review the literature on LLM augmentation with external tools for code reasoning.
Design and implement a SootUp tool-calling API exposing call-graph construction , virtual call resolution, reachability queries, and inter-procedural data-flow summaries.
Wrap the tool-calling API as an MCP server.
Run evaluation on proper large real-world benchmark across all conditions and LLM variants.
Learning Outcomes:
Hands-on experience designing and implementing LLM architecture with a static analysis tool-calling interface on top of SootUp.
Hands on experience designing and deploying MCP server.
Deep understanding of the strengths and limitations of LLMs for non-local Java program reasoning.
Experience designing and conducting rigorous empirical software engineering evaluations.
Familiarity with industry-standard benchmarks for LLM evaluation
