Cloud Engineer

Cloud Engineer Job details Posted 01 October 2024 Location Minneapolis, MN Job type Permanent Reference 939007 Job description Title: Cloud Engineer Location - Minneapolis, MN -Hybrid Duration - 6 Months 7 years of professional experience working with: NodeJS, Python, Java, SQL Azure Databases (Oracle, MSSQL, Postgres, etc) SQS Codebuild Postgresql Mongo Atlas DynamoDB Cloudformation Docker Fargate GCP GBQ Cloud Data Fusion Experience with stream processing (SNS, SQS, Kafka, etc) Ability to communicate across diverse audience of business analysts, architect, business user, and development teams Self-starter, capable of driving towards goals independently Strong organization and interpersonal skills. Ability to deal with and balance multiple priorities. Ability to deal with ambiguity and rapid change. Hands-on role requiring rigorous attention to detail. Have demonstrable understanding and awareness of technology and an ability to learn and acquire new technologies as needed. Knowledge of data lakes and data warehousing Bonus experience: ETL tools (Informatica, SSIS, Talend, etc) R Mulesoft Seven or more years of experience in solving challenging technical problems in two or more of the following areas: Single page web application development Unix/Linux environment management Cloud based continuous delivery and environment management Micro-service design and development Large, distributed system development Software security Processing and analyzing large data sets ETL Data Modeling Data Engineering BS degree in Computer Science, similar technical field of study, or equivalent practical experience Key Tasks and Responsibilities: Practical experience and knowledge of industry practices. Expertise in at least one technology or facet of product development (research, design, programming, testing) Strong written and verbal communication skills Collaborate daily with your Team of Engineers, Product Manager, Product Specialists, and Agile Coach to build and support applications. Work with team and Product Manager to manage project priorities, deadlines, and deliverables Interact directly with business customers to gather and understand requirements and how they translate in technical solutions and implementations Develop applications and services for projects owned by the team using the latest tools and technologies Deploy applications across all environments in collaboration with engineering team and Product Management Perform unit and integration testing where applicable Conduct functional and non-functional testing Troubleshoot and debug data pipeline Evaluate existing implementations and identify opportunities to automate, refactor, update and add new features Develop any technical documentation needed to accurately represent implementation Help to shape the vision for their own team's products and those of related teams, taking into account impacts, dependencies, and opportunities to collaborate with other teams and groups Develop effective relationships & communicate within team, with Product Management, and with customers to increase the overall quality of products owned by the team and across related teams. Identify opportunities to develop new and better solutions. Anticipate and address technical and organizational obstacles affecting own team and related teams. Provide guidance and mentoring for technical design and architecture decisions within the team and related teams. Identify and pursue opportunities to improve own knowledge and skills and those of more junior developers. Must have Skills ETL and Data Pipeline Development - Design, develop, and optimize scalable ETL processes using Python, Apache Spark, and Azure Synapse. - Build and manage Azure Data Factory pipelines to orchestrate complex data workflows. - Use SQL Pools and Spark Pools within Synapse to manage and process large datasets efficiently. - Implement Data Warehousing solutions using Azure Synapse Analytics to provide structured and queryable data layers. - Ensure the data platform supports real-time and batch AI/ML data requirements. Azure Cloud Development & CI/CD Deployment - Build, configure, and manage CI/CD pipelines on Azure DevOps for ETL and data processing tasks. - Automate infrastructure provisioning, testing, and deployment using Infrastructure-as-Code (IaC) tools like ARM templates or Terraform. - Optimize Azure Data Lake Storage (ADLS Gen2) to store and manage raw and processed data efficiently, ensuring proper access control and data security. Cross-Functional Collaboration - Collaborate with Data Scientists, Data Engineers, ML Engineers, and Business Analysts to translate business requirements into data solutions. - Work with the DevOps and Security teams to ensure smooth and secure deployment of applications and pipelines. - Act as the technical lead in designing, developing, and implementing data solutions, mentoring junior team members. Data Engineering and API Development - Develop and integrate with external and internal APIs for data ingestion and data exchange. - Build, test, and deploy RESTful APIs for secure data access. - Use Kubernetes for containerizing and deploying data processing applications. - Manage data storage and transformation to support advanced Data Science and AI/ML models. Agile Project Management - Participate in and lead Agile ceremonies, such as sprint planning, daily stand-ups, and retrospectives. - Collaborate with cross-functional teams in iterative development to ensure high-quality and timely feature delivery. - Adapt to changing project priorities and business needs in an Agile environment. Dexian is a leading provider of staffing, IT, and workforce solutions with over 12,000 employees and 70 locations worldwide. As one of the largest IT staffing companies and the 2nd largest minority-owned staffing company in the U.S., Dexian was formed in 2023 through the merger of DISYS and Signature Consultants. Combining the best elements of its core companies, Dexian's platform connects talent, technology, and organizations to produce game-changing results that help everyone achieve their ambitions and goals. Dexian's brands include Dexian DISYS, Dexian Signature Consultants, Dexian Government Solutions, Dexian Talent Development and Dexian IT Solutions. Visit https://dexian.com/ to learn more. Dexian is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.