AI Builder — Working Student

EggAI Labs · Working Student · In Person (Tübingen, Germany)

Designing lifecycle evaluation for nondeterministic domain-specific AI systems.

EggAI Labs

We're opening the EggAI Labs in Tübingen. The Labs purpose is to test new AI capabilities, identify what works, how they'd improve delivery, and then build assets to support teams and clients. EggAI's mission cannot be achieved through one-off client projects alone and the Labs is the way to adapt and scale.

As part of the first cohort, you will join a small peer group and work directly with EggAI's CAIO, a Tübingen alumnus. You will also collaborate with our engineering, product, and project leads, who will bring you insights from client problems, help you understand their context and challenge your solutions for them. Your fellow AI Builders will be deliberately chosen to span different passions and strenghts. You will have room to explore and be expected to turn that exploration into something useful. This is not a client-facing role.

The problem: EvalOps

How do we prove an AI system acts as expected under various conditions?

We will explore EvalOps: the methods and tools used to check whether domain-specific AI systems meet their behavioral requirements. Among the key challenges for operating AI systems are nondeterminism, ground-truth curation, evaluation metric-design, latency and costs. EvalOps need to overcome all of them in a disciplined way.

These questions will guide the initial work:

Regression suites: How can domain experts turn traces, code, and prompts into useful evals in a way that scales?

<strong