Role
This role sits in the commodities data team at Balyasny - at the intersection of the front office and central data services. The team is responsible for commodities specific data ingestion, modelling and distribution across a range of technologies and data frequencies. Data that is managed in the team is used directly by commodities trading teams; as such the role has a direct impact on trading and requires working with traders and researchers to ensure optimum performance of data systems at all times.
Experience
• 5+ years in data engineering including ownership of production data systems.
• Has built low latency or near real-time pipelines.
• Has run trade-critical systems on a formal on-call rota, including fixing live incidents under pressure.
• Has supported front-office or technical users directly (traders, PMs, quants or similar), resolving issues quickly and explaining them clearly.
• Has worked on long-running platform migration on containerised, cloud-native infrastructure.
• Has integrated and normalised a wide range of non-standard external data sources.
• Commodities or energy market exposure (power, gas, oil, shipping, agriculture) is a strong plus but not essential.
Skills
• Python (expert): Production services, custom data transformation, bespoke data pipelines
• SQL (expert): Complex modelling and performance-aware DML and DDL across Snowflake/Clickhouse
• Kubernetes and Docker
• Kafka
• Strong understanding of Git, code review, CI, TDD
• Apache Airflow - desirable, but not essential
Responsibilities
• Design, build and own bespoke low latency pipelines for our most demanding data sources.
• On-call rota, owning detection, triage and resolution of production data incidents.
• Be a subject matter expert for PMs and quants on data issues - fixing production issues quickly and communicating technical points clearly and reliably.
• Hold engineering standards across the team: review, testing, release.