Data Engineering Lead (Data Acquisition)
Shell
Greater London, England, United Kingdom
Full-Time
Competitive
OfficeManager
Job Family Group:
Worker Type:
Posting Start Date:
Business Unit:
Experience Level:
Job Description:
What’s the role
As a Data Engineering Lead (Data Acquisition), you will lead Shell Energy Trading & Supply’s data acquisition function: own the ingestion strategy and build pipelines on Azure; act as the bridge from end-user needs to engineered solutions; serve as the domain’s go-to Data Manager for mastership, lineage, and quality; and lead a team of data engineers to deliver reliable, compliant, high-value data to Trading, Risk, Finance, and Operations.
What you’ll be doing
Own the ingestion strategy & build high-quality pipelines
Define target-state patterns for batch/streaming ingestion; standardize connectors, schemas, contracts and controls.
Lead design and build on Azure data platforms (ADLS, ADF, Databricks, Synapse, Event Hubs/Kafka), with CI/CD, IaC and automated testing.
Industrialize onboarding of sources (ETRM/Endur, Trayport, market data vendors, SAP, SCADA/IoT) with SLAs, observability and runbooks.
Bridge end‑user requests to engineered solutions
Translate trading desk, risk, operations and finance requirements into a prioritized acquisition backlog; shape epics/stories with clear acceptance criteria.
Partner with Product Owners, Business Interface and Solution/Data Architects to align designs, models and mastership decisions for the domain.
Go‑to Data Manager responsibilities for the domain
Steward data definitions, ownership, lineage, metadata, quality KPIs and access policies; chair data change forums.
Apply Shell’s technical information & data lifecycle: register, receive, set up/load, update/maintain, publish/archive or dispose; uphold audit trail and records management.
Lead and develop a team of Data Engineers
Build a high-performing, inclusive team; coach in coding standards, design reviews, performance tuning and production operations.
Shape capacity plans, skills roadmap and vendor contributions; foster a culture of safety, compliance and continuous improvement.
Operate reliably & safely
Ensure resilience, recoverability and cyber hygiene; meet controls for data privacy, confidentiality and regulatory obligations in trading (e.g., REMIT/EMIR, privacy).
Drive incident, change & problem management; measure and report service health and user satisfaction.
What you bring
Extensive experience in data engineering, with proven years leading teams/products in complex, real‑time data environments.
Expertise in Azure data services (ADLS, ADF, Databricks, Synapse, Event Hub/Kafka), Python, SQL, Spark/PySpark.
Strong knowledge of streaming and batch patterns, schema management, CDC, API integration.
Experience with data modelling, metadata and lineage tooling; performance & cost optimization.
Proven leadership in engineering teams; Agile/DevOps ways of working; CI/CD; test automation.
Preferred: exposure to energy trading systems (ETRM/Endur), market data (ICE, Platts, Bloomberg), SAP.
Degree in Computer Science/Engineering or equivalent experience; Azure Data Engineer or Databricks certifications a plus.
What we offer
You bring your skills and experience to Shell and in return you work with talented, committed people on one of the most important challenges facing our planet. You’ll have the opportunity to develop the skills you need to grow in an environment where we value honesty, integrity, and respect for one another. You’ll be able to balance your priorities as you become the best version of yourself.
Progress as a person as we work on the energy transition together.
Continuously grow the transferable skills you need to get ahead.
Work at the forefront of technology, trends, and practices.
Collaborate with experienced colleagues with unique expertise.
Achieve your balance in a values-led culture that encourages you to be the best version of yourself.
Benefit from a range of flexible working options.
Perform at your best with a competitive starting salary and annual performance related salary increase – our pay and benefits packages are considered to be among the best in the world.
Take advantage of paid parental leave, including for non-birthing parents.
Join an organisation working to become one of the most diverse and inclusive in the world. We strongly encourage applicants of all genders, ages, ethnicities, cultures, abilities, sexual orientation, and life experiences to apply.
Grow as you progress through diverse career opportunities in national and international teams.
Gain access to a wide range of training and development programmes.
We are committed to attracting a broader and more diverse pool of candidates. If this position doesn’t feel like the perfect fit for your qualifications right now, we’d still love to hear from you.
Shell is working to advance an inclusive, psychologically safe and accessible environment where people with disabilities can excel. If you require any accommodations or accessibility adjustments (e.g. assistive technology, communication support, or other) during the application or interview process, please let us know when prompted in your application. We strive to ensure that our process and workplace is accessible to everyone and are dedicated to making reasonable adjustments to support your needs.
Shell in The United Kingdom
Shell UK remains one of the North Sea’s biggest producers, supplying around 10% of the UK’s total oil and gas needs. But perhaps the most recognisable face of Shell in the UK is our network of over 1,000 Shell-branded service stations.
In the years ahead, as the UK looks to strengthen energy security and deliver its 2050 net-zero goal, Shell UK aims to play a crucial role. We aim to be a major investor in the UK energy system by helping our customers decarbonise with a focus on transport and industry.
-
DISCLAIMER: