Senior Engineer, Commodities Technology
Role
This role sits in the commodities data team at Balyasny - at the intersection of the front office and central data services. The team is responsible for commodities specific data ingestion, modelling and distribution across a range of technologies and data frequencies. Data that is managed in the team is used directly by commodities trading teams; as such the role has a direct impact on trading and requires working with traders and researchers to ensure optimum performance of data systems at all times.
Experience
- 5+ years in data engineering including ownership of production data systems.
- Has built low latency or near real-time pipelines.
- Has run trade-critical systems on a formal on-call rota, including fixing live incidents under pressure.
- Has supported front-office or technical users directly (traders, PMs, quants or similar), resolving issues quickly and explaining them clearly.
- Has worked on long-running platform migration on containerised, cloud-native infrastructure.
- Has integrated and normalised a wide range of non-standard external data sources.
- Commodities or energy market exposure (power, gas, oil, shipping, agriculture) is a strong plus but not essential.
Skills
- Python (expert): Production services, custom data transformation, bespoke data pipelines
- SQL (expert): Complex modelling and performance-aware DML and DDL across Snowflake/Clickhouse
- Kubernetes and Docker
- Kafka
- Strong understanding of Git, code review, CI, TDD
- Apache Airflow - desirable, but not essential
Responsibilities
- Design, build and own bespoke low latency pipelines for our most demanding data sources.
- On-call rota, owning detection, triage and resolution of production data incidents.
- Be a subject matter expert for PMs and quants on data issues - fixing production issues quickly and communicating technical points clearly and reliably.
- Hold engineering standards across the team: review, testing, release.