Perennial Resources International
Senior Data and Cloud Engineer
Perennial Resources International, New York, New York, us, 10261
Lead Data EngineerFull-time/PermanentOnsite - NYC
Responsibilities:
Take ownership of the entire data engineering lifecycle, including design, implementation, performance tuning, and monitoringManage and work with the firm's database managed service provider (MSP) to manage and monitor the firm's evolving database requirementsDesign, develop, and maintain scalable data pipelines and ETL processes critical to support business operations and analytics. (Python / SQL)Optimize and tune data systems for high performance and reliabilityCollaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutionsImplement and manage data storage solutions in cloud environments (AWS, Azure, Google Cloud)Ensure data quality and integrity through rigorous testing and validationContinuously evaluate and integrate new technologies and tools to improve data processing capabilitiesManage distributed scheduled tasks on individual servers to a centralized rulesetWork with the development team and database MSP to modernize the backup and recovery strategy for 3-4 SQL servers across different cloud environmentsSupport the addition of new tenants on the firm's production data platformQualifications:
Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field from a top-tier university.4-5 years of relevant professional experience in data engineeringStrong programming skills in SQL and Python.Proficiency in database administrationProven experience migrating on-prem data and associated processes to cloud platforms (AWS, Azure, Google Cloud)Demonstrated ability to design and implement efficient and reliable data pipelines and ETL processesStrong problem-solving skills and ability to troubleshoot complex data issuesExcellent communication skills and ability to work collaboratively in a team environmentA proactive mindset with a willingness to take ownership and drive projects to completion. Strong organizational skills along with a desire to continually be challengedExperience with monitoring, CI/CD, Load Balancing, and Testing tools (e.g. Redgate, Datadog)
Responsibilities:
Take ownership of the entire data engineering lifecycle, including design, implementation, performance tuning, and monitoringManage and work with the firm's database managed service provider (MSP) to manage and monitor the firm's evolving database requirementsDesign, develop, and maintain scalable data pipelines and ETL processes critical to support business operations and analytics. (Python / SQL)Optimize and tune data systems for high performance and reliabilityCollaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutionsImplement and manage data storage solutions in cloud environments (AWS, Azure, Google Cloud)Ensure data quality and integrity through rigorous testing and validationContinuously evaluate and integrate new technologies and tools to improve data processing capabilitiesManage distributed scheduled tasks on individual servers to a centralized rulesetWork with the development team and database MSP to modernize the backup and recovery strategy for 3-4 SQL servers across different cloud environmentsSupport the addition of new tenants on the firm's production data platformQualifications:
Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field from a top-tier university.4-5 years of relevant professional experience in data engineeringStrong programming skills in SQL and Python.Proficiency in database administrationProven experience migrating on-prem data and associated processes to cloud platforms (AWS, Azure, Google Cloud)Demonstrated ability to design and implement efficient and reliable data pipelines and ETL processesStrong problem-solving skills and ability to troubleshoot complex data issuesExcellent communication skills and ability to work collaboratively in a team environmentA proactive mindset with a willingness to take ownership and drive projects to completion. Strong organizational skills along with a desire to continually be challengedExperience with monitoring, CI/CD, Load Balancing, and Testing tools (e.g. Redgate, Datadog)