Diverse Lynx
Data Architect @Corvallis, OR
Diverse Lynx, Corvallis, Oregon, United States, 97333
Data ArchitectLocation:
Corvallis, OR - Onsite RoleLong Term Project
Job Description:
The data engineering role is a team member that will help enhance and maintain the CSS Business Analytics and Instant Ink Business Intelligence system. You will drive work you're doing to completion with hands-on development responsibilities, and partner with the Data Engineering leaders to implement data engineering pipelines to build solution to help provide trusted and reliable data to customers.
Responsibilities:At least 15+ years' experience in data engineering , AI Client and Aws/AzureArchitect, Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data/Lakehouse ecosystem.Experience in AI and Client on AWS/AZUREAnalyzes design and determines coding, programming, and integration activities required based on general objectives.Play the technical lead role representing deliverables from vendor team resources at onsite and offshore locations.Lead the technical co-ordination and Business Knowledge transition activities to offshore team.Reviews and evaluates designs and project activities for compliance with architecture, security and quality guidelines and standards.Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component; identifies defects and creates solutions for issues with code and integration into data system architecture.Collaborates and communicates with project team regarding project progress and issue resolution.Works with the data engineering team for all phases of larger and more-complex development projects and engages with external users on business and technical requirements.Collaborates with peers, engineers, data scientists and project team.Typically interacts with high-level Individual Contributors, Managers and Program Teams on a daily/weekly basis.What you bring:
Bachelor's or Master's degree in Computer Science, Information Systems, Engineering or equivalent.6+ years of relevant experience with detailed knowledge of data technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.3+ years of experience with Cloud based DW such as Redshift, Snowflake etc.3+ years' experience in Big Data Distributed ecosystems (Hadoop, SPARK, Unity Catalog & Delta Lake)3+ years experience in Workflow orchestration tools such as Airflow etc.3+ years' experience in Big Data Distributed systems such as Databricks, AWS EMR, AWS Glue etc.Leverage monitoring tools/frameworks, like Splunk, Grafana, CloudWatch etc.Experience with container management frameworks such as Docker, Kubernetes, ECR etc.3+ year's working with multiple Big Data file formats (Parquet, Delta Lake)Experience working on CI/CD processes such as Jenkins, Codeway etc. and source control tools such as GitHub, etc.Strong experience in coding languages like Python, Scala & JavaKnowledge and Skills:
Fluent in relational based systems and writing complex SQL.Fluent in complex, distributed and massively parallel systems.Strong analytical and problem-solving skills with ability to represent complex algorithms in software.Strong understanding of database technologies and management systems.Strong understanding of data structures and algorithmsDatabase architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.Strong analytical and problem-solving skills.Nice to Have:
Experience with visualization tools such as PowerBI, Tableau etc.Experience with transformation tools such as dbt.Have experience in building realtime streaming data pipelinesExperience in pub/sub streaming technologies like Kafka, Kinesis, Spark Streaming etc.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Corvallis, OR - Onsite RoleLong Term Project
Job Description:
The data engineering role is a team member that will help enhance and maintain the CSS Business Analytics and Instant Ink Business Intelligence system. You will drive work you're doing to completion with hands-on development responsibilities, and partner with the Data Engineering leaders to implement data engineering pipelines to build solution to help provide trusted and reliable data to customers.
Responsibilities:At least 15+ years' experience in data engineering , AI Client and Aws/AzureArchitect, Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data/Lakehouse ecosystem.Experience in AI and Client on AWS/AZUREAnalyzes design and determines coding, programming, and integration activities required based on general objectives.Play the technical lead role representing deliverables from vendor team resources at onsite and offshore locations.Lead the technical co-ordination and Business Knowledge transition activities to offshore team.Reviews and evaluates designs and project activities for compliance with architecture, security and quality guidelines and standards.Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component; identifies defects and creates solutions for issues with code and integration into data system architecture.Collaborates and communicates with project team regarding project progress and issue resolution.Works with the data engineering team for all phases of larger and more-complex development projects and engages with external users on business and technical requirements.Collaborates with peers, engineers, data scientists and project team.Typically interacts with high-level Individual Contributors, Managers and Program Teams on a daily/weekly basis.What you bring:
Bachelor's or Master's degree in Computer Science, Information Systems, Engineering or equivalent.6+ years of relevant experience with detailed knowledge of data technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.3+ years of experience with Cloud based DW such as Redshift, Snowflake etc.3+ years' experience in Big Data Distributed ecosystems (Hadoop, SPARK, Unity Catalog & Delta Lake)3+ years experience in Workflow orchestration tools such as Airflow etc.3+ years' experience in Big Data Distributed systems such as Databricks, AWS EMR, AWS Glue etc.Leverage monitoring tools/frameworks, like Splunk, Grafana, CloudWatch etc.Experience with container management frameworks such as Docker, Kubernetes, ECR etc.3+ year's working with multiple Big Data file formats (Parquet, Delta Lake)Experience working on CI/CD processes such as Jenkins, Codeway etc. and source control tools such as GitHub, etc.Strong experience in coding languages like Python, Scala & JavaKnowledge and Skills:
Fluent in relational based systems and writing complex SQL.Fluent in complex, distributed and massively parallel systems.Strong analytical and problem-solving skills with ability to represent complex algorithms in software.Strong understanding of database technologies and management systems.Strong understanding of data structures and algorithmsDatabase architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.Strong analytical and problem-solving skills.Nice to Have:
Experience with visualization tools such as PowerBI, Tableau etc.Experience with transformation tools such as dbt.Have experience in building realtime streaming data pipelinesExperience in pub/sub streaming technologies like Kafka, Kinesis, Spark Streaming etc.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.