Collabera
Senior Data Engineer
Collabera, Houston, Texas, United States, 77246
Description
HomeSearch JobsJob Description
Senior Data Engineer
Remote: Houston, Texas, US
Salary: $150000.00 Per Annum
Job Code: 354768
End Date: 2024-11-07Days Left: 23 days, 3 hours left
Apply
Location: 100% Remote (EST/CST)
About the role:
We are looking for a skilled Data Engineer to join our engineering team, focusing on leveraging Azure SQL, CosmosDB & DataBricks to meet our growing data demands. As a Sr. data engineer in the Apex organization, you will be part of the Scrum team, developing data capabilities powering customer-facing applications used by thousands of residents. You will play a critical role in supporting the data needs of the engineering team as you develop and optimize the backend for customer-facing real estate applications
What You Will Do100% hands-on development - Azure SQL, CosmosDB, DataBricks: develop and unit test database code, including but not limited to T-SQL, stored procedures, functions and views.Own/Maintain the DataBricks data ingestion & output for our MS Customer Insights CDP platformCreate and maintain database structures.Participate in the design of databases, using first, second or third normalized form as needed to support business requirements.Create, deploy, and maintain ADF pipelines, adhering to client's standards and documented best practices.Monitor database performance, identify bottlenecks and implement improvements to ensure scalable and reliable database operations in a production environment.Perform analysis of complex data and document findings.Prepare data for prescriptive and predictive modeling.Combine raw data from different external sources and build and support complex ingestionsCollaborate with application developers/data analysts who will be consuming the data.Play a direct role in the maintenance, technical support, documentation, and administration of databases.Ensure standards are followed by participating in code reviewsYou are a thought leader who will bring up new ideas or methods and deliver awesome codeTechnical Qualifications
6+ years relevant and progressive data engineering experienceDeep Technical knowledge and experience in MS Azure architecture, including Azure PaaSdatabases, Synapse, ADF pipelines and Synapse stored procedures3+ Years of Experience with CosmosDB2+ Years of Experience with DataBricksMinimum of 3 years of relevant experience working with Azure Data Factory.Hands-on skills working with data pipelines using SQL and No-SQL databasesMinimum of 1 year of relevant experience working with Azure Data Lakes Gen 2.Experience with Power Platform / Power BIExperience in engineering practices such as code refactoring, design patterns, CI/CD, and highly scalable data applicationsExperience developing batch ETL pipelines; real-time pipelines are a plusKnowledge of advanced data engineering concepts such as dimensional modeling, ETL, data governance, data warehousing, structured and unstructured dataUnderstanding / Experience with Python; additional knowledge of parallel processing frameworks likeMapReduce, Spark, Scala is a plusExperience / Familiarity with D365 Customer Insights platform / Dataverse is a plusOverall Qualifications
Knowledge of Agile software development processExcellent problem-solving skills and experienceStrong communication and collaboration skills"Self-starter" attitude and the ability to make decisions with minimal guidance from othersInnovative and passionate about your work and the work of your teammatesAbility to comprehend and analyze operational systems and ask appropriate questions to determine how to improve, migrate or modify the solution to meet business needsJob Requirement
Azure DatabricksAzure Data FactoryCosmos DBAzure SQLConsumer Facing ApplicationReach Out to a RecruiterRecruiterEmailPhoneChinmay Modakchinmay.modak@collabera.com
Apply Now
HomeSearch JobsJob Description
Senior Data Engineer
Remote: Houston, Texas, US
Salary: $150000.00 Per Annum
Job Code: 354768
End Date: 2024-11-07Days Left: 23 days, 3 hours left
Apply
Location: 100% Remote (EST/CST)
About the role:
We are looking for a skilled Data Engineer to join our engineering team, focusing on leveraging Azure SQL, CosmosDB & DataBricks to meet our growing data demands. As a Sr. data engineer in the Apex organization, you will be part of the Scrum team, developing data capabilities powering customer-facing applications used by thousands of residents. You will play a critical role in supporting the data needs of the engineering team as you develop and optimize the backend for customer-facing real estate applications
What You Will Do100% hands-on development - Azure SQL, CosmosDB, DataBricks: develop and unit test database code, including but not limited to T-SQL, stored procedures, functions and views.Own/Maintain the DataBricks data ingestion & output for our MS Customer Insights CDP platformCreate and maintain database structures.Participate in the design of databases, using first, second or third normalized form as needed to support business requirements.Create, deploy, and maintain ADF pipelines, adhering to client's standards and documented best practices.Monitor database performance, identify bottlenecks and implement improvements to ensure scalable and reliable database operations in a production environment.Perform analysis of complex data and document findings.Prepare data for prescriptive and predictive modeling.Combine raw data from different external sources and build and support complex ingestionsCollaborate with application developers/data analysts who will be consuming the data.Play a direct role in the maintenance, technical support, documentation, and administration of databases.Ensure standards are followed by participating in code reviewsYou are a thought leader who will bring up new ideas or methods and deliver awesome codeTechnical Qualifications
6+ years relevant and progressive data engineering experienceDeep Technical knowledge and experience in MS Azure architecture, including Azure PaaSdatabases, Synapse, ADF pipelines and Synapse stored procedures3+ Years of Experience with CosmosDB2+ Years of Experience with DataBricksMinimum of 3 years of relevant experience working with Azure Data Factory.Hands-on skills working with data pipelines using SQL and No-SQL databasesMinimum of 1 year of relevant experience working with Azure Data Lakes Gen 2.Experience with Power Platform / Power BIExperience in engineering practices such as code refactoring, design patterns, CI/CD, and highly scalable data applicationsExperience developing batch ETL pipelines; real-time pipelines are a plusKnowledge of advanced data engineering concepts such as dimensional modeling, ETL, data governance, data warehousing, structured and unstructured dataUnderstanding / Experience with Python; additional knowledge of parallel processing frameworks likeMapReduce, Spark, Scala is a plusExperience / Familiarity with D365 Customer Insights platform / Dataverse is a plusOverall Qualifications
Knowledge of Agile software development processExcellent problem-solving skills and experienceStrong communication and collaboration skills"Self-starter" attitude and the ability to make decisions with minimal guidance from othersInnovative and passionate about your work and the work of your teammatesAbility to comprehend and analyze operational systems and ask appropriate questions to determine how to improve, migrate or modify the solution to meet business needsJob Requirement
Azure DatabricksAzure Data FactoryCosmos DBAzure SQLConsumer Facing ApplicationReach Out to a RecruiterRecruiterEmailPhoneChinmay Modakchinmay.modak@collabera.com
Apply Now