Senior/Staff AI Engineer, Quality & Evals (m/f/x)

About this role

Responsibilities

• We are looking for an engineer with strong backend, data, and AI systems experience to build the evaluation and observability foundation for production-grade LLM agents used in complex audit workflows.
• This role sits at the intersection of backend engineering, data infrastructure, and AI quality. You will build the evaluation systems that power our multimodal retrieval agents and continuously improve critical quality metrics across current and future pipelines.
• You’ll work at the edge of applied AI and information retrieval, building multimodal agentic pipelines and solving hard context and agent-harness engineering problems.
• This is not a traditional analytics, BI, or dashboarding role. You should expect to write production code, design data architecture, work inside backend systems, and directly improve the quality, cost, reliability, and performance of LLM-based agents.
• What you’ll do
• You will help build and operate the technical systems around our AI agents, with a focus on data infrastructure, evaluation, observability,

About Cortea AI

We’re Cortea, a Berlin startup transforming audits with AI. Manual, document-heavy audits waste expert time while demand keeps rising. Our AI-powered software and specialized AI agents remove the repetitive work so auditors can focus on judgment. Backed by top-tier VCs with >15m EUR funding, with a working product and paying customers, we’re rapidly scaling. We value first-principles thinking, speed, trust, and kindness. We build side by side in our Berlin office.