Airetel Staffing, Inc.
Senior Data Engineer
Airetel Staffing, Inc., Kansas City, Missouri, United States, 64101
One of Airetel Staffing's valued clients is looking for a Senior Data Engineer located in Kansas City, MO.
Job DescriptionThe Data Engineer will help build and maintain the cloud DeltaLake platform leveraging Databricks. Candidates will be expected to contribute to all stages of the data lifecycle including data ingestion, data modeling, data profiling, data quality, data transformation, data movement, and data curation.
Responsibilities
Design, implement (deploy) and support on-premise and cloud-based data infrastructure (systems, flow) that are resilient to disruptions and failuresEnhance and support corporate SQL/NoSSQL database, DWH assets and streaming data solutionsEnsure high uptime for all data services and consider enhanced solutions through scheduled or event-driven designBring multi cloud/cross-platform agnostic technologies and practices into the system to enhance reliability and support rapid scaling of the business's data needsScale up our data infrastructure to meet cross-functional, multi industry business needsDevelop, leverage and maintain end-to-end data pipelines in productionProvide subject matter expertise and hands on delivery of data acquisition, curation and consumption pipelines on Azure, Databricks, AWS, ConfluentResponsible for maintaining current and emerging state of the art compute and cloud based solutions and technologies.Build effective relationships with internal stakeholdersFamiliarity with the technology stack available in the industry for metadata management: Data Governance, Data Quality, MDM, Lineage, Data Catalog etc.Hands-on experience implementing analytics solutions leveraging Python, Spark SQL, Databricks Lakehouse Architecture, orchestration tools Kubernetes, Dockerll other duties as assigned
Requirements
:
Bachelor's degree in Computer Science, Information Technology, Management Information Systems (MIS), Data Science or related field. 6+ years of experience may be substituted for degree requirement. (required)Minimum 8 years of experience in software engineering (required)Experience with leading large and complex data projects(required)Experience with large-scale data warehousing architecture and data modeling, (preferred)Worked with Cloud-based architecture such as Azure, AWS or Google Cloud, (required)Experience working with big data technologies e.g. Snowflake, Redshift, Synapse, Postgres, Airflow, Kafka, Spark, DBT, (preferred)Experience implementing pub/sub and streaming use cases, (preferred)Experience leading design reviews, (preferred)Experience influencing a team's technical and business strategy by making insightful contributions to team priorities and approaches, (preferred)Working knowledge of relational databases, (preferred)Expert in SQL, Python, Java and high-level languages such as Scala, C#, or C (required)Demonstrate the ability to analyze large data sets to identify gaps and inconsistencies in ETL pipeline and provide solutions for pipeline reliability and data quality, (preferred)Experience in designing and implementing an infrastructure as code / CICD development environment, (required)Proven ability to build, manage and foster a team-oriented environment (required)Excellent communication (written and oral) and interpersonal skills (required)Excellent organizational, multi-tasking, and time-management skills (required)
Airetel Staffing, Inc. is nationally recognized as Best of Staffing - Diamond Award Winner. For more details or to view other opportunities, visit us at
www.airetel.com
or call 214-353-0561.
Job DescriptionThe Data Engineer will help build and maintain the cloud DeltaLake platform leveraging Databricks. Candidates will be expected to contribute to all stages of the data lifecycle including data ingestion, data modeling, data profiling, data quality, data transformation, data movement, and data curation.
Responsibilities
Design, implement (deploy) and support on-premise and cloud-based data infrastructure (systems, flow) that are resilient to disruptions and failuresEnhance and support corporate SQL/NoSSQL database, DWH assets and streaming data solutionsEnsure high uptime for all data services and consider enhanced solutions through scheduled or event-driven designBring multi cloud/cross-platform agnostic technologies and practices into the system to enhance reliability and support rapid scaling of the business's data needsScale up our data infrastructure to meet cross-functional, multi industry business needsDevelop, leverage and maintain end-to-end data pipelines in productionProvide subject matter expertise and hands on delivery of data acquisition, curation and consumption pipelines on Azure, Databricks, AWS, ConfluentResponsible for maintaining current and emerging state of the art compute and cloud based solutions and technologies.Build effective relationships with internal stakeholdersFamiliarity with the technology stack available in the industry for metadata management: Data Governance, Data Quality, MDM, Lineage, Data Catalog etc.Hands-on experience implementing analytics solutions leveraging Python, Spark SQL, Databricks Lakehouse Architecture, orchestration tools Kubernetes, Dockerll other duties as assigned
Requirements
:
Bachelor's degree in Computer Science, Information Technology, Management Information Systems (MIS), Data Science or related field. 6+ years of experience may be substituted for degree requirement. (required)Minimum 8 years of experience in software engineering (required)Experience with leading large and complex data projects(required)Experience with large-scale data warehousing architecture and data modeling, (preferred)Worked with Cloud-based architecture such as Azure, AWS or Google Cloud, (required)Experience working with big data technologies e.g. Snowflake, Redshift, Synapse, Postgres, Airflow, Kafka, Spark, DBT, (preferred)Experience implementing pub/sub and streaming use cases, (preferred)Experience leading design reviews, (preferred)Experience influencing a team's technical and business strategy by making insightful contributions to team priorities and approaches, (preferred)Working knowledge of relational databases, (preferred)Expert in SQL, Python, Java and high-level languages such as Scala, C#, or C (required)Demonstrate the ability to analyze large data sets to identify gaps and inconsistencies in ETL pipeline and provide solutions for pipeline reliability and data quality, (preferred)Experience in designing and implementing an infrastructure as code / CICD development environment, (required)Proven ability to build, manage and foster a team-oriented environment (required)Excellent communication (written and oral) and interpersonal skills (required)Excellent organizational, multi-tasking, and time-management skills (required)
Airetel Staffing, Inc. is nationally recognized as Best of Staffing - Diamond Award Winner. For more details or to view other opportunities, visit us at
www.airetel.com
or call 214-353-0561.