Lead Bigdata Engineer

Description 8 years experience of implementing data intensive solutions using agile methodologies Proficient in programming languages commonly used in data engineering such as Scala Spark Python and Java Experience of building data processing pipelines for use in production hands off batch systems including either or preferably both traditional ETL pipelines and/or analytics pipelines Strong coding experience in the likes of Scala Java or Python Enthusiasm to learn and develop emerging technologies and techniques Must have strong SQL and RDBMS Data model skills Demonstrating deep technical knowledge and expertise in software development including programming languages frameworks and best practices Providing guidance and mentorship to junior team members Actively contribute to the implementation of critical features and complex technical solutions Write clean efficient and maintainable code that meets the highest standards of quality Write defensive fault tolerant and efficient code for data processing Provide guidance on scalable robust and efficient solutions that align with business requirements and industry best practices. Use emerging and open-source technologies such as Spark Hadoop and Scala Quantexa Certified Data Engineer Collaborate on scalability issues involving access to massive amounts of data and information Take on ad hoc tasks as required Quantexa Certified Data Engineer Familiarity with open-source data engineering tools and frameworks eg Spark Kafka Trino is a valuable asset Exposure to a range of table and file formats including Hive Parquet and JSON Experience in cloud native technologies and patterns AWS Google Cloud Experience of driving and or influencing the data strategy of your team or organization Banking domain Domain knowledge