Diverse Lynx
Data Engineer - Bigdata
Diverse Lynx, Raleigh, North Carolina, United States, 27601
Role name:
Engineer
Role Description:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), •Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Competencies:
Digital : BigData and Hadoop Ecosystems, Microsoft SQL Server 2019, Digital : Jenkins, Digital : PySpark
Experience (Years):
4-6
Essential Skills:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), •Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Desirable Skills:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), •Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Country:
United States
Branch | City | Location:
Raleigh, NC~TCS - Denver, COCHARLOTTE~DelawareCharlotte, NC~Delaware, I
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Engineer
Role Description:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), •Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Competencies:
Digital : BigData and Hadoop Ecosystems, Microsoft SQL Server 2019, Digital : Jenkins, Digital : PySpark
Experience (Years):
4-6
Essential Skills:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), •Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Desirable Skills:
Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), •Primary Skills:o Big Data: Spark, Scala, Pyspark, HDFSo Microsoft Stack: MS-SQL with strong knowledge in RDBMS conceptso Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)o Scripting Languages: Batch Script, Shell Script, Python•Additional Skills:o Agile, Scrum, Jira, Git, SVN, Liquibase•Work on Pyspark Framework to ingest data from different sources system to Hadoop and SQL Server regions.•Sound knowledge in Python and Good to have enough knowledge in Scala.•Strong experience in database migration to MS SQL Server/Hadoop•Extensive experience in database query tuning, performance tuning, and troubleshooting application issues on OLTP/OLAP systems.•RDBMS Architecture, T-SQL query and Query Optimization knowledge and work experience•Provide support to team members and helping them to understand the projects and requirements and guiding them to create the optimized solution of it.•team player and proven track record of working in various team sizes performing cross-functional roles.•Setup CICD pipeline for database changes using GitHub, Jenkin & Liquibase)•Good to have experience data migration from Hadoop to CDP
Country:
United States
Branch | City | Location:
Raleigh, NC~TCS - Denver, COCHARLOTTE~DelawareCharlotte, NC~Delaware, I
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.