Apply several skills to build an observable security pipeline for agentic AI systems and investigate how runtime observability can support the detection and analysis of AI-specific security threats:

• AI & Large Language Models
• Agentic AI
• Software Engineering
• Cybersecurity
• Observability & Monitoring

You find yourself? Then this project group is for you!

Project_Proposal.pdf

Abstract

Agentic AI systems increasingly combine large language models with external tools, databases, APIs, memory, and other services. This makes it difficult to understand what an AI agent is doing at runtime and to detect security problems that may emerge during interaction.

Your task is to develop an observable agentic AI pipeline that collects and correlates runtime information such as traces, events, logs, and metrics. The project will investigate whether this observability information can be used to detect and analyze security threats such as prompt injection, hallucination, and sensitive information exposure.

The project will use existing observability and security technologies rather than implementing an observability platform from scratch. In particular, students will investigate and integrate technologies

Motivation

Large Language Model (LLM)-based applications are becoming increasingly complex. Modern AI agents do not simply receive an input and generate an output. An agent may interpret a user request, invoke an LLM several times, retrieve information from a database, access external tools, communicate through protocols such as MCP, maintain memory, and finally generate an answer or perform an action.

This complexity introduces new security challenges. For example, a malicious user input may cause prompt injection, an LLM may produce a hallucinated or unreliable response, or sensitive information may unintentionally appear in an input, intermediate processing step, tool call, or generated output.

Traditional application logging is often insufficient to understand these problems because the relevant information is distributed across multiple components and interactions.

AI observability provides a way to record and correlate the execution of an AI application. For example, a single user request can be represented as a trace containing the user interaction, LLM calls, retrieval operations, tool invocations, security checks, and final response. Metrics and events can additionally provide aggregated information about the system's behavior. The objective of the project is to develop an observable agentic AI pipeline that captures and correlates runtime traces, events, logs, and metrics using technologies such as OpenTelemetry and Langfuse.

The project will investigate how these observability signals can support the detection and analysis of prompt injection, hallucination, and sensitive information exposure. Students will also explore lightweight policy-based security controls (e.g., OPA) and evaluate the effectiveness of the approach using controlled scenarios.