The opportunity
Unity’s Ads Experimentation Platform team is looking for a Staff Data Scientist to lead the evolution of how we validate and optimize our global advertising ecosystem. At Unity, our ads reach billions of devices, and the ability to rapidly iterate on machine learning models or ads delivery pipelines is critical to our growth.
In this role, you will be the technical authority for our experimentation and evaluation roadmap. You will bridge the gap between advanced statistical methodology and large-scale engineering, building the tools that allow ML teams to iterate faster and with higher confidence. You will join a team focused on high-velocity innovation, moving beyond standard A/B testing into the next generation of causal inference and high-sensitivity evaluation methodologies.
What you'll be doing
Advance A/B Testing Efficiency: Lead the implementation of variance reduction (CUPED) and sequential testing to increase platform sensitivity and speed. You will develop strategies to mitigate interference and network effects, ensuring results reflect true causal impact in a complex auction environment.
Scale Ranking Evaluations: Architect and deploy interleaving frameworks to rapidly assess ranking model performance in real-time. This enables the team to identify winning candidates orders-of-magnitude faster than standard bucket testing.
Reduce Budget Cannibalization Effects: Design methodologies to account for finite advertiser budgets and prevent experimental groups from cannibalizing each other's spend. You will develop pacing-aware evaluation techniques to maintain marketplace integrity and ensure unbiased results.
Define Long-Term Value (LTV) Proxy Metrics: Research and validate surrogate metrics that correlate highly with long-term user retention and value. You will provide ML teams with short-term signals that accurately predict long-term impact, enabling faster optimization for long-term growth.
Drive Automated ML Experimentation: Build the statistical foundations for automated pipelines that autonomously test and select optimal features and hyperparameters. This reduces manual engineering overhead and accelerates the deployment of high-performing models at scale.
Cross-Functional Technical Leadership: Serve as the lead subject matter expert on experimentation for ML, Product, and Engineering teams. You will ensure statistical rigor is integrated throughout the product lifecycle, from initial model training to live production auctions.