Data Engineer - Commodities

Data Engineer - CommoditiesMillennium is a top tier global hedge fund with a strong commitment to leveraging innovations in technology and data science to solve complex problems for the business. We are assembling a strong Commodity Technology team to build our next generation in-house commodity technology platform and associated ecosystem of tools, applications and systems.Commodity Technology provides a dynamic and fast-paced environment with excellent growth opportunities and projects involving cutting edge technologies.ResponsibilitiesDevelop data and analytics platform using standard tools for batch and stream processing, container management, observability, and persistenceWork closely with traders, quants, and other technologists globally to understand system development requirementsDevelop software solutions in an agile fashion using modern software development practices e.g. comprehensive testing, version control practices, CI/CD, etc.Test and deploy software solutions in an automated fashion using CI/CD practicesEnsure DevOps style management and operation of DEV/UAT/PROD deployment environmentsMandatory Requirements5+ years’ of professional experience with Python software developmentExperience with Airflow, event streaming (e.g., Kafka) and processing (e.g., Flink), big data table formats like Iceberg or Delta Lake, and KubernetesWorking knowledge of traditional and cloud SQL databasesPreferred RequirementsFamiliarity with quantitative finance and electronic trading concepts.Experience using AWS technologies such as S3, Athena, SQS, Batch, LambdaKnowledge of the various Python dependency management and packaging tools (e.g., Poetry) and data quality, observability, and testing tools (e.g., Great Expectations, Soda)

#J-18808-Ljbffr