Diverse Lynx
Big Data Engineer
Diverse Lynx, Denver, Colorado, United States, 80285
Position:
Big Data EngineerLocation: Denver, CO/Charlotte, NCDuration: Contract
4-8 years
of Exp.Job Description:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP),•Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Big Data EngineerLocation: Denver, CO/Charlotte, NCDuration: Contract
4-8 years
of Exp.Job Description:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP),•Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.