Data Engineering Lead (Data Acquisition)

, United Kingdom

Job Family Group:

Trading Jobs

Worker Type:

Regular

Posting Start Date:

June 5, 2026

Business Unit:

Trading and Supply

Experience Level:

Experienced Professionals

Job Description:

What’s the role

As a Data Engineering Lead (Data Acquisition), you will lead Shell Energy Trading & Supply’s data acquisition function: own the ingestion strategy and build pipelines on Azure; act as the bridge from end-user needs to engineered solutions; serve as the domain’s go-to Data Manager for mastership, lineage, and quality; and lead a team of data engineers to deliver reliable, compliant, high-value data to Trading, Risk, Finance, and Operations.

What you’ll be doing

Own the ingestion strategy & build high-quality pipelines

Define target-state patterns for batch/streaming ingestion; standardize connectors, schemas, contracts and controls.

Lead design and build on Azure data platforms (ADLS, ADF, Databricks, Synapse, Event Hubs/Kafka), with CI/CD, IaC and automated testing.

Industrialize onboarding of sources (ETRM/Endur, Trayport, market data vendors, SAP, SCADA/IoT) with SLAs, observability and runbooks.

Bridge end‑user requests to engineered solutions

Translate trading desk, risk, operations and finance requirements into a prioritized acquisition backlog; shape epics/stories with clear acceptance criteria.

Partner with Product Owners, Business Interface and Solution/Data Architects to align designs, models and mastership decisions for the domain.

Go‑to Data Manager responsibilities for the domain

Steward data definitions, ownership, lineage, metadata, quality KPIs and access policies; chair data change forums.

Apply Shell’s technical information & data lifecycle: register, receive, set up/load, update/maintain, publish/archive or dispose; uphold audit trail and records management.

Lead and develop a team of Data Engineers

Build a high-performing, inclusive team; coach in coding standards, design reviews, performance tuning and production operations.

Shape capacity plans, skills roadmap and vendor contributions; foster a culture of safety, compliance and continuous improvement.

Operate reliably & safely

Ensure resilience, recoverability and cyber hygiene; meet controls for data privacy, confidentiality and regulatory obligations in trading (e.g., REMIT/EMIR, privacy).

Drive incident, change & problem management; measure and report service health and user satisfaction.

What you bring

Extensive experience in data engineering, with proven years leading teams/products in complex, real‑time data environments.

Expertise in Azure data services (ADLS, ADF, Databricks, Synapse, Event Hub/Kafka), Python, SQL, Spark/PySpark.

Strong knowledge of streaming and batch patterns, schema management, CDC, API integration.

Experience with data modelling, metadata and lineage tooling; performance & cost optimization.

Proven leadership in engineering teams; Agile/DevOps ways of working; CI/CD; test automation.

Preferred: exposure to energy trading systems (ETRM/Endur), market data (ICE, Platts, Bloomberg), SAP.

Degree in Computer Science/Engineering or equivalent experience; Azure Data Engineer or Databricks certifications a plus.