Apolis
Data Engineer
Apolis, Parsippany, New Jersey, us, 07054
Job Title: Data Engineer
Location:
Parsippany, NJ Tax Term (W2, C2C): Responsibilities Experience in Enterprise Data Engineering and Analytics projects including any Cloud Data pipeline development platform and enterprise data warehousing project/ETL. Clear understanding Data warehousing and Data Lake concepts. Must Have Requirement gathering Working with business/product owners to clarify the reporting needs. Working with other stakeholders like application teams etc to clarify the source systems/data. Performing source data analysis, discovery/reverse engineering of existing models. Designing Creating the data model designs/relations. Designing the etl pipeline. Experience with SQL/Stored Procedures . Experience with ETL orchestration. Understanding of metadata driven frameworks for performing ELT. Pros/Cons of using ETL tools VS custom built frameworks in SQL/Python. High level understanding/knowledge of ingestion/collection patterns in to cloud objects storage(streaming, file based ) Good understanding of storage optimizations(partitioning, archiving unused data), knowledge of various file formats CSV/JSON/PARQUET/ORC/AVRO Optimizing the ETL/SQL for performance, performance troubleshooting, DB specific features related to query performance. Intermediate to expert knowledge in python. Exposure to any clouds(AWS/Azure/GCP/OCI) Build Strong experience in working in Star schema models Experience in building and maintaining of Datawarehouse implementations of the data models using ELT. Development of the etl pipeline using redshift SQL/stored procedures. Troubleshooting the ETL loads , failures. Orchestration of pipelines using airflow. Performance optimization of the pipelines. Complete involvement in the SDLC. Nice to Have Data Modelling design (ER/Dimensional Modelling) Conceptual/Logical/Physical. Experience with streaming data ingestions/collections using Kafka etc. Consumption of DWH data in BI platforms like tableau/power BI, data source preparation (live/extracts). Data sharing methodologies. Experience with AWS/OCI. Non-technical
Parsippany, NJ Tax Term (W2, C2C): Responsibilities Experience in Enterprise Data Engineering and Analytics projects including any Cloud Data pipeline development platform and enterprise data warehousing project/ETL. Clear understanding Data warehousing and Data Lake concepts. Must Have Requirement gathering Working with business/product owners to clarify the reporting needs. Working with other stakeholders like application teams etc to clarify the source systems/data. Performing source data analysis, discovery/reverse engineering of existing models. Designing Creating the data model designs/relations. Designing the etl pipeline. Experience with SQL/Stored Procedures . Experience with ETL orchestration. Understanding of metadata driven frameworks for performing ELT. Pros/Cons of using ETL tools VS custom built frameworks in SQL/Python. High level understanding/knowledge of ingestion/collection patterns in to cloud objects storage(streaming, file based ) Good understanding of storage optimizations(partitioning, archiving unused data), knowledge of various file formats CSV/JSON/PARQUET/ORC/AVRO Optimizing the ETL/SQL for performance, performance troubleshooting, DB specific features related to query performance. Intermediate to expert knowledge in python. Exposure to any clouds(AWS/Azure/GCP/OCI) Build Strong experience in working in Star schema models Experience in building and maintaining of Datawarehouse implementations of the data models using ELT. Development of the etl pipeline using redshift SQL/stored procedures. Troubleshooting the ETL loads , failures. Orchestration of pipelines using airflow. Performance optimization of the pipelines. Complete involvement in the SDLC. Nice to Have Data Modelling design (ER/Dimensional Modelling) Conceptual/Logical/Physical. Experience with streaming data ingestions/collections using Kafka etc. Consumption of DWH data in BI platforms like tableau/power BI, data source preparation (live/extracts). Data sharing methodologies. Experience with AWS/OCI. Non-technical