Data Engineer

Data Tech team is looking for a Data Engineer to join a diverse team dedicated to providing best in class data services to our customers, stakeholders and partners. As a part of our organization, you will work with our Client Engineers, Data Scientist and various BUs to define solutions for operationalizing data-driven decision making in a cost effective and scalable manner.

Qualifications:

• Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience
• Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises
• AWS Solutions Architect or AWS Big Data Certification preferred
• Programming experience with Java, Python/ Scala, Shell scripting
• Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
• Solid experience implementing solutions on AWS based data lakes
• Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB
• Experience in AWS data lake, data warehouse and business analytics
• Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS
• Knowledge of ETL/ ELT
• Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired
• Experience working on Agile projects

Requirements

• 3 - 5 years of experience as Data Engineer
• Experience developing business applications using NoSQL/ SQL databases.
• Experience working with Object stores (S3) and JSON is must have
• Should have good experience with AWS Services - API Gateway, Lambda, Step Functions, SQS, DynamoDB, S3, ElasticSearch.
• Serverless application development using AWS Lambda.

Responsibilities:

• Designing, building and maintaining efficient, reusable and reliable architecture and code
• Ensure the best possible performance and quality of high scale web applications and services
• Participate in the architecture and system design discussions
• Independently perform hands on development and unit testing of the applications
• Collaborate with the development team and build individual components into complex enterprise web systems
• Work in a team environment with product, frontend design, production operation, QE/ QA and cross functional teams to deliver a project throughout the whole software development cycle
• Responsible to identify and resolve any performance issues
• Keep up to date with new technology development and implementation
• Participate in code review to make sure standards and best practices are met

Plus

• Experience with business intelligence tools such as Tableau, Power BI or equivalent
• Aware of Machine learning algorithms (supervised/unsupervised)
• Nice to have AWS developer certificate