Saxon Global
Data Engineer (PySpark - Data Bricks)
Saxon Global, Carlisle, Pennsylvania, United States, 17013
Position Summary
Plays a critical role in the design, development, and roll-out of the Data & Analytics Platform, along with partnering with team members to develop the data consumption tools for the Data & Analytics Platform. Engages through the entire lifecycle of all the projects within Business Intelligence & Analytics department from data modeling, to data engineering, and finally data consumption. Becomes a subject matter expert on the Data & Analytics Platform. Delivers reporting and analysis to the enterprise.
Principle Duties and Responsibilities
Develop data engineering processes in azure using components like Azure Data Factory, Azure Data Lake analytics, HDInsights, DataBricks etc.Works with business users to design, develop, test, and implement business intelligence solutions in the Data & Analytics Platform.Solves moderate to complex application errors, as well as designs and resolves application problems, following up in a timely manner with all appropriate customers and IT personnel.Documents all phases of work including gathering requirements, joining relationship diagrams, creating database diagrams, report layouts and other program technical specifications using current specified design standards for new or revised solutions.Assists in assessing alternative software solutions for workability and technical feasibility.Relates information from various sources to draw logical conclusions.Assists in identifying the impact of proposed application development/enhancements projects.Conducts data analysis (SQL, Excel, Data Discovery, etc.) on legacy systems and new data sources.Performs data profiling and process analysis to understand key source systems and uses knowledge of application features and functions to assess scope and impact of business needs.Reviews and validates QA test plans and supports QA team during test execution.Participates in designing data transformation and data pipe linesEnsures change control and change management procedures are followed within the program/project as they relate to requirements.Documents reporting requirements through engagement with business process SMEs.Creates source to target data mappings for data pipelines and integration activities.Designs and develops reports, dashboards to support decision-making within the business and IT.Conducts unit testing on ELT and report development.Conducts data lineage and impact analysis as a part of the change management process.Delivers SDLC Deliverables as defined.Contributes to Business Case Development through information gathering and analysis.Basic Qualification
Bachelor's degree in Computer Science, Information Systems, Business Administration, or other related field.Minimum of 10 years IT experience encompassing the following:
Working experience with SDLC and deliverables associated with each phase.Experience in Data Modeling and Advanced SQL techniquesExperience engineering data pipelines using latest technologies and techniquesExperience in Data Visualization Best PracticesExperience working with industry-leading Business Intelligence toolsExperience working with industry-leading database technologiesExperience in Azure Cloud technologiesExperience in Big Data / Hadoop
Technical Competencies
Develop Business Intelligence applications/dashboards/Reports in Microstrategy and Power BI.Develop complex SQL/USQL/pySpark code for Data Engineering pipelines in Azure Data Lake analytics and azure data factory.Develop complex data science algorithms using R programming language and industrialize them.Develop and Implement big data applications using Hadoop components (HDInsight - Microsoft Azure).Create requirement documents, review the functional design documents created by vendor partners as a part of software development life cycle.Work closely with business teams in every stage from gathering requirements to closure of project.Work with the key stakeholders to create requirements for each business solution, ensure the requirements are developed in concert with and agreed upon by the business partner, and that the solution delivers the agreed upon business needs.Skills and Abilities
Outstanding Written and Verbal Communication communications skills; able to articulate complex technical topics for non-technical audiences.Ability to analyze complex situations and develop creative solutions.Ability to multi-task and self-direct.Strong Organization Skills.
Ability to interact with non IT associates and management on projects.Ability to plan and organize work to meet pre-established deadlines.Ability to work with their peers, associates and all levels of management.Ability to explain technical concepts in a non-technical fashion.Strong problem-solving and critical thinking skills; strong deductive and inductive reasoning.Strong Analytical Aptitude.
Ability to work with and influence peers and senior management.Ability to work in a fast-paced and deadline-oriented environment.Self-motivated with critical attention to detail, deadlines and reporting.Preferred Qualification
Experience in a retail environment a plus.Certifications in desired technologies
Required Skills : AzureBackground Check :YesDrug Screen :YesNotes :Selling points for candidate :Project Verification Info :Candidate must be your W2 Employee :YesExclusive to Apex :NoFace to face interview required :NoCandidate must be local :NoCandidate must be authorized to work without sponsorship ::NoInterview times set : :NoType of project :TAPFIN @ Retail Business Services - SOWMaster Job Title :Branch Code :
Plays a critical role in the design, development, and roll-out of the Data & Analytics Platform, along with partnering with team members to develop the data consumption tools for the Data & Analytics Platform. Engages through the entire lifecycle of all the projects within Business Intelligence & Analytics department from data modeling, to data engineering, and finally data consumption. Becomes a subject matter expert on the Data & Analytics Platform. Delivers reporting and analysis to the enterprise.
Principle Duties and Responsibilities
Develop data engineering processes in azure using components like Azure Data Factory, Azure Data Lake analytics, HDInsights, DataBricks etc.Works with business users to design, develop, test, and implement business intelligence solutions in the Data & Analytics Platform.Solves moderate to complex application errors, as well as designs and resolves application problems, following up in a timely manner with all appropriate customers and IT personnel.Documents all phases of work including gathering requirements, joining relationship diagrams, creating database diagrams, report layouts and other program technical specifications using current specified design standards for new or revised solutions.Assists in assessing alternative software solutions for workability and technical feasibility.Relates information from various sources to draw logical conclusions.Assists in identifying the impact of proposed application development/enhancements projects.Conducts data analysis (SQL, Excel, Data Discovery, etc.) on legacy systems and new data sources.Performs data profiling and process analysis to understand key source systems and uses knowledge of application features and functions to assess scope and impact of business needs.Reviews and validates QA test plans and supports QA team during test execution.Participates in designing data transformation and data pipe linesEnsures change control and change management procedures are followed within the program/project as they relate to requirements.Documents reporting requirements through engagement with business process SMEs.Creates source to target data mappings for data pipelines and integration activities.Designs and develops reports, dashboards to support decision-making within the business and IT.Conducts unit testing on ELT and report development.Conducts data lineage and impact analysis as a part of the change management process.Delivers SDLC Deliverables as defined.Contributes to Business Case Development through information gathering and analysis.Basic Qualification
Bachelor's degree in Computer Science, Information Systems, Business Administration, or other related field.Minimum of 10 years IT experience encompassing the following:
Working experience with SDLC and deliverables associated with each phase.Experience in Data Modeling and Advanced SQL techniquesExperience engineering data pipelines using latest technologies and techniquesExperience in Data Visualization Best PracticesExperience working with industry-leading Business Intelligence toolsExperience working with industry-leading database technologiesExperience in Azure Cloud technologiesExperience in Big Data / Hadoop
Technical Competencies
Develop Business Intelligence applications/dashboards/Reports in Microstrategy and Power BI.Develop complex SQL/USQL/pySpark code for Data Engineering pipelines in Azure Data Lake analytics and azure data factory.Develop complex data science algorithms using R programming language and industrialize them.Develop and Implement big data applications using Hadoop components (HDInsight - Microsoft Azure).Create requirement documents, review the functional design documents created by vendor partners as a part of software development life cycle.Work closely with business teams in every stage from gathering requirements to closure of project.Work with the key stakeholders to create requirements for each business solution, ensure the requirements are developed in concert with and agreed upon by the business partner, and that the solution delivers the agreed upon business needs.Skills and Abilities
Outstanding Written and Verbal Communication communications skills; able to articulate complex technical topics for non-technical audiences.Ability to analyze complex situations and develop creative solutions.Ability to multi-task and self-direct.Strong Organization Skills.
Ability to interact with non IT associates and management on projects.Ability to plan and organize work to meet pre-established deadlines.Ability to work with their peers, associates and all levels of management.Ability to explain technical concepts in a non-technical fashion.Strong problem-solving and critical thinking skills; strong deductive and inductive reasoning.Strong Analytical Aptitude.
Ability to work with and influence peers and senior management.Ability to work in a fast-paced and deadline-oriented environment.Self-motivated with critical attention to detail, deadlines and reporting.Preferred Qualification
Experience in a retail environment a plus.Certifications in desired technologies
Required Skills : AzureBackground Check :YesDrug Screen :YesNotes :Selling points for candidate :Project Verification Info :Candidate must be your W2 Employee :YesExclusive to Apex :NoFace to face interview required :NoCandidate must be local :NoCandidate must be authorized to work without sponsorship ::NoInterview times set : :NoType of project :TAPFIN @ Retail Business Services - SOWMaster Job Title :Branch Code :