Dexian
Cloud Engineer
Dexian, Minneapolis, Minnesota, United States, 55400
Title: Cloud Engineer
Location - Minneapolis, MN -Hybrid
Duration - 6+ Months
7+ years of professional experience working with:NodeJS, Python, Java, SQLAzureDatabases (Oracle, MSSQL, Postgres, etc)SQSCodebuildPostgresqlMongo AtlasDynamoDBCloudformationDockerFargateGCPGBQCloud Data FusionExperience with stream processing (SNS, SQS, Kafka, etc)Ability to communicate across diverse audience of business analysts, architect, business user, and development teamsSelf-starter, capable of driving towards goals independentlyStrong organization and interpersonal skills. Ability to deal with and balance multiple priorities. Ability to deal with ambiguity and rapid change.Hands-on role requiring rigorous attention to detail.Have demonstrable understanding and awareness of technology and an ability to learn and acquire new technologies as needed.Knowledge of data lakes and data warehousingBonus experience:ETL tools (Informatica, SSIS, Talend, etc)RMulesoftSeven or more years of experience in solving challenging technical problems in two or more of the following areas:
Single page web application developmentUnix/Linux environment managementCloud based continuous delivery and environment managementMicro-service design and developmentLarge, distributed system developmentSoftware securityProcessing and analyzing large data setsETLData ModelingData EngineeringBS degree in Computer Science, similar technical field of study, or equivalent practical experienceKey Tasks and Responsibilities:
Practical experience and knowledge of industry practices.Expertise in at least one technology or facet of product development (research, design, programming, testing)Strong written and verbal communication skillsCollaborate daily with your Team of Engineers, Product Manager, Product Specialists, and Agile Coach to build and support applications.Work with team and Product Manager to manage project priorities, deadlines, and deliverablesInteract directly with business customers to gather and understand requirements and how they translate in technical solutions and implementationsDevelop applications and services for projects owned by the team using the latest tools and technologiesDeploy applications across all environments in collaboration with engineering team and Product ManagementPerform unit and integration testing where applicableConduct functional and non-functional testingTroubleshoot and debug data pipelineEvaluate existing implementations and identify opportunities to automate, refactor, update and add new featuresDevelop any technical documentation needed to accurately represent implementationHelp to shape the vision for their own team's products and those of related teams, taking into account impacts, dependencies, and opportunities to collaborate with other teams and groupsDevelop effective relationships & communicate within team, with Product Management, and with customers to increase the overall quality of products owned by the team and across related teams.Identify opportunities to develop new and better solutions.Anticipate and address technical and organizational obstacles affecting own team and related teams.Provide guidance and mentoring for technical design and architecture decisions within the team and related teams.Identify and pursue opportunities to improve own knowledge and skills and those of more junior developers.
Must have SkillsETL and Data Pipeline Development- Design, develop, and optimize scalable ETL processes using Python, Apache Spark, and Azure Synapse.
- Build and manage Azure Data Factory pipelines to orchestrate complex data workflows.
- Use SQL Pools and Spark Pools within Synapse to manage and process large datasets efficiently.
- Implement Data Warehousing solutions using Azure Synapse Analytics to provide structured and queryable data layers.
- Ensure the data platform supports real-time and batch AI/ML data requirements.
Azure Cloud Development & CI/CD Deployment- Build, configure, and manage CI/CD pipelines on Azure DevOps for ETL and data processing tasks.
- Automate infrastructure provisioning, testing, and deployment using Infrastructure-as-Code (IaC) tools like ARM templates or Terraform.
- Optimize Azure Data Lake Storage (ADLS Gen2) to store and manage raw and processed data efficiently, ensuring proper access control and data security.
Cross-Functional Collaboration- Collaborate with Data Scientists, Data Engineers, ML Engineers, and Business Analysts to translate business requirements into data solutions.
- Work with the DevOps and Security teams to ensure smooth and secure deployment of applications and pipelines.
- Act as the technical lead in designing, developing, and implementing data solutions, mentoring junior team members.
Data Engineering and API Development- Develop and integrate with external and internal APIs for data ingestion and data exchange.
- Build, test, and deploy RESTful APIs for secure data access.
- Use Kubernetes for containerizing and deploying data processing applications.
- Manage data storage and transformation to support advanced Data Science and AI/ML models.
Agile Project Management
- Participate in and lead Agile ceremonies, such as sprint planning, daily stand-ups, and retrospectives.
- Collaborate with cross-functional teams in iterative development to ensure high-quality and timely feature delivery.
- Adapt to changing project priorities and business needs in an Agile environment.
Dexian is a leading provider of staffing, IT, and workforce solutions with over 12,000 employees and 70 locations worldwide. As one of the largest IT staffing companies and the 2nd largest minority-owned staffing company in the U.S., Dexian was formed in 2023 through the merger of DISYS and Signature Consultants. Combining the best elements of its core companies, Dexian's platform connects talent, technology, and organizations to produce game-changing results that help everyone achieve their ambitions and goals.
Dexian's brands include Dexian DISYS, Dexian Signature Consultants, Dexian Government Solutions, Dexian Talent Development and Dexian IT Solutions. Visit https://dexian.com/ to learn more.
Dexian is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.
Location - Minneapolis, MN -Hybrid
Duration - 6+ Months
7+ years of professional experience working with:NodeJS, Python, Java, SQLAzureDatabases (Oracle, MSSQL, Postgres, etc)SQSCodebuildPostgresqlMongo AtlasDynamoDBCloudformationDockerFargateGCPGBQCloud Data FusionExperience with stream processing (SNS, SQS, Kafka, etc)Ability to communicate across diverse audience of business analysts, architect, business user, and development teamsSelf-starter, capable of driving towards goals independentlyStrong organization and interpersonal skills. Ability to deal with and balance multiple priorities. Ability to deal with ambiguity and rapid change.Hands-on role requiring rigorous attention to detail.Have demonstrable understanding and awareness of technology and an ability to learn and acquire new technologies as needed.Knowledge of data lakes and data warehousingBonus experience:ETL tools (Informatica, SSIS, Talend, etc)RMulesoftSeven or more years of experience in solving challenging technical problems in two or more of the following areas:
Single page web application developmentUnix/Linux environment managementCloud based continuous delivery and environment managementMicro-service design and developmentLarge, distributed system developmentSoftware securityProcessing and analyzing large data setsETLData ModelingData EngineeringBS degree in Computer Science, similar technical field of study, or equivalent practical experienceKey Tasks and Responsibilities:
Practical experience and knowledge of industry practices.Expertise in at least one technology or facet of product development (research, design, programming, testing)Strong written and verbal communication skillsCollaborate daily with your Team of Engineers, Product Manager, Product Specialists, and Agile Coach to build and support applications.Work with team and Product Manager to manage project priorities, deadlines, and deliverablesInteract directly with business customers to gather and understand requirements and how they translate in technical solutions and implementationsDevelop applications and services for projects owned by the team using the latest tools and technologiesDeploy applications across all environments in collaboration with engineering team and Product ManagementPerform unit and integration testing where applicableConduct functional and non-functional testingTroubleshoot and debug data pipelineEvaluate existing implementations and identify opportunities to automate, refactor, update and add new featuresDevelop any technical documentation needed to accurately represent implementationHelp to shape the vision for their own team's products and those of related teams, taking into account impacts, dependencies, and opportunities to collaborate with other teams and groupsDevelop effective relationships & communicate within team, with Product Management, and with customers to increase the overall quality of products owned by the team and across related teams.Identify opportunities to develop new and better solutions.Anticipate and address technical and organizational obstacles affecting own team and related teams.Provide guidance and mentoring for technical design and architecture decisions within the team and related teams.Identify and pursue opportunities to improve own knowledge and skills and those of more junior developers.
Must have SkillsETL and Data Pipeline Development- Design, develop, and optimize scalable ETL processes using Python, Apache Spark, and Azure Synapse.
- Build and manage Azure Data Factory pipelines to orchestrate complex data workflows.
- Use SQL Pools and Spark Pools within Synapse to manage and process large datasets efficiently.
- Implement Data Warehousing solutions using Azure Synapse Analytics to provide structured and queryable data layers.
- Ensure the data platform supports real-time and batch AI/ML data requirements.
Azure Cloud Development & CI/CD Deployment- Build, configure, and manage CI/CD pipelines on Azure DevOps for ETL and data processing tasks.
- Automate infrastructure provisioning, testing, and deployment using Infrastructure-as-Code (IaC) tools like ARM templates or Terraform.
- Optimize Azure Data Lake Storage (ADLS Gen2) to store and manage raw and processed data efficiently, ensuring proper access control and data security.
Cross-Functional Collaboration- Collaborate with Data Scientists, Data Engineers, ML Engineers, and Business Analysts to translate business requirements into data solutions.
- Work with the DevOps and Security teams to ensure smooth and secure deployment of applications and pipelines.
- Act as the technical lead in designing, developing, and implementing data solutions, mentoring junior team members.
Data Engineering and API Development- Develop and integrate with external and internal APIs for data ingestion and data exchange.
- Build, test, and deploy RESTful APIs for secure data access.
- Use Kubernetes for containerizing and deploying data processing applications.
- Manage data storage and transformation to support advanced Data Science and AI/ML models.
Agile Project Management
- Participate in and lead Agile ceremonies, such as sprint planning, daily stand-ups, and retrospectives.
- Collaborate with cross-functional teams in iterative development to ensure high-quality and timely feature delivery.
- Adapt to changing project priorities and business needs in an Agile environment.
Dexian is a leading provider of staffing, IT, and workforce solutions with over 12,000 employees and 70 locations worldwide. As one of the largest IT staffing companies and the 2nd largest minority-owned staffing company in the U.S., Dexian was formed in 2023 through the merger of DISYS and Signature Consultants. Combining the best elements of its core companies, Dexian's platform connects talent, technology, and organizations to produce game-changing results that help everyone achieve their ambitions and goals.
Dexian's brands include Dexian DISYS, Dexian Signature Consultants, Dexian Government Solutions, Dexian Talent Development and Dexian IT Solutions. Visit https://dexian.com/ to learn more.
Dexian is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.