Apollo ITS
Data Engineer
Apollo ITS, Detroit, Michigan, United States, 48228
Job Title: Data Engineer
Location: Metro Detroit, MI-Hybrid
Duration: 12 Months
Note:
Must be
LOCAL to Metro Detroit
-
HYBRID 3 days in office in Ann Arbor. Interview:
20 mins with manager, 1 hour with manager and 1-2 team members (ETL scenarios and SQL Assessment) Team/Project:
Will be working with the HR team on the UKG software (vendor). They have a BA that interfaces with the vendor but always helpful to have a technical person there as well. This software holds store team member data, log ins, hours, time off etc. All the HR data for store employees. Mirandas team is about 13-14 devs and some BA's Notes from intake we had for previous role:
Main tools
ETL-Talend; DataStage/Informatica might work but lets begin w talend SQL Server DB
Outside of that all other things are a plus
Cloud- Azure is the plan, Kafka (for streaming- its helpful), Databricks/Lakehouse (Esp for mar-tech people) Python/shell scripting would be nice
Don't see a big need for big-data, IoT given domain Comms are big; 2-4 years of Experience is what they want
Additional Job Details The data engineering specialist will primarily focus on development of large volume information ingestion and transformation. This position is responsible for orchestrating data interfaces into (and out of) our Enterprise Data Warehouse using Talend, SQL, Python and other data engineering solutions. GENERAL RESPONSIBILITIES
Design and develop ETL (Talend) / SQL / Python based processes to perform complex data transformation processes. Design, code, and test major data processing features, as well as work jointly with other team members to provide complex software enhancements for the enterprise data storage platforms (RDBMS, Lakehouse, No-SQL platforms) Build Data Integration solutions to handle batch / streaming / IoT data on ETL, Big-Data platforms. Develop and Deliver changes in the Enterprise Data Warehouse according to Data Warehousing best practices Gather requirements and construct documentation to aid in maintenance and code reuse in accordance with team processes and standards Design, code, and test major data processing features, as well as work jointly with other team members to provide complex software enhancements for the enterprise data storage platforms Monitor scheduled jobs and improve reliability of ongoing processing Monitor, measure, and enhance ways to improve system performance Ability to multi-task deliverables and manage them in an efficient manner. Performs other duties as assigned Qualifications Understanding and Hands-on experience with key technologies (SQL, ETL, Data modeling, data processing) Strong SQL skills with Relational DBMS technology - SQL Server 2-4 yrs Hands-on experience with ETL tools (Talend preferred) Good understanding and expertise in relational database concepts and data processing concepts Experience in handling multiple data formats (Delimited file, JSON, XML etc.) Experience with data lakehouses (Databricks) and cloud technologies (Aure) a plus Experience with Customer and digital marketing data, including implementing Customer Data Platform and Identity Resolution solutions and integrating with a ESP (Salesforce), a plus Hands on experience designing and implementing data ingestion techniques for real time processes (IoT, eCommerce) a plus. Development experience in a Big-Data environment a Plus; Spark, Kafka, Message Queues Experience with shell scripting / Python a plus Strong communication skills (oral and written) Good analytical and problem solving skills Knowledge of CRM, MDM, HR Reporting and Business Intelligence a plus Candidate must be thorough and detail-oriented and a team player Able to work on multiple priorities in a deadline-driven environment TOP MUST HAVES:
ETL (Talend) SQL Cloud (azure) and Kafka are helpful
Must be
LOCAL to Metro Detroit
-
HYBRID 3 days in office in Ann Arbor. Interview:
20 mins with manager, 1 hour with manager and 1-2 team members (ETL scenarios and SQL Assessment) Team/Project:
Will be working with the HR team on the UKG software (vendor). They have a BA that interfaces with the vendor but always helpful to have a technical person there as well. This software holds store team member data, log ins, hours, time off etc. All the HR data for store employees. Mirandas team is about 13-14 devs and some BA's Notes from intake we had for previous role:
Main tools
ETL-Talend; DataStage/Informatica might work but lets begin w talend SQL Server DB
Outside of that all other things are a plus
Cloud- Azure is the plan, Kafka (for streaming- its helpful), Databricks/Lakehouse (Esp for mar-tech people) Python/shell scripting would be nice
Don't see a big need for big-data, IoT given domain Comms are big; 2-4 years of Experience is what they want
Additional Job Details The data engineering specialist will primarily focus on development of large volume information ingestion and transformation. This position is responsible for orchestrating data interfaces into (and out of) our Enterprise Data Warehouse using Talend, SQL, Python and other data engineering solutions. GENERAL RESPONSIBILITIES
Design and develop ETL (Talend) / SQL / Python based processes to perform complex data transformation processes. Design, code, and test major data processing features, as well as work jointly with other team members to provide complex software enhancements for the enterprise data storage platforms (RDBMS, Lakehouse, No-SQL platforms) Build Data Integration solutions to handle batch / streaming / IoT data on ETL, Big-Data platforms. Develop and Deliver changes in the Enterprise Data Warehouse according to Data Warehousing best practices Gather requirements and construct documentation to aid in maintenance and code reuse in accordance with team processes and standards Design, code, and test major data processing features, as well as work jointly with other team members to provide complex software enhancements for the enterprise data storage platforms Monitor scheduled jobs and improve reliability of ongoing processing Monitor, measure, and enhance ways to improve system performance Ability to multi-task deliverables and manage them in an efficient manner. Performs other duties as assigned Qualifications Understanding and Hands-on experience with key technologies (SQL, ETL, Data modeling, data processing) Strong SQL skills with Relational DBMS technology - SQL Server 2-4 yrs Hands-on experience with ETL tools (Talend preferred) Good understanding and expertise in relational database concepts and data processing concepts Experience in handling multiple data formats (Delimited file, JSON, XML etc.) Experience with data lakehouses (Databricks) and cloud technologies (Aure) a plus Experience with Customer and digital marketing data, including implementing Customer Data Platform and Identity Resolution solutions and integrating with a ESP (Salesforce), a plus Hands on experience designing and implementing data ingestion techniques for real time processes (IoT, eCommerce) a plus. Development experience in a Big-Data environment a Plus; Spark, Kafka, Message Queues Experience with shell scripting / Python a plus Strong communication skills (oral and written) Good analytical and problem solving skills Knowledge of CRM, MDM, HR Reporting and Business Intelligence a plus Candidate must be thorough and detail-oriented and a team player Able to work on multiple priorities in a deadline-driven environment TOP MUST HAVES:
ETL (Talend) SQL Cloud (azure) and Kafka are helpful