Perennial Resources International
Senior Data and Cloud Engineer
Perennial Resources International, New York, New York, us, 10261
Lead Data Engineer
Full-time/PermanentOnsite - NYCResponsibilities:
Take ownership of the entire data engineering lifecycle, including design, implementation, performance tuning, and monitoring.Manage and work with the firm's database managed service provider (MSP) to manage and monitor the firm's evolving database requirements.Design, develop, and maintain scalable data pipelines and ETL processes critical to support business operations and analytics. (Python / SQL)Optimize and tune data systems for high performance and reliability.Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.Implement and manage data storage solutions in cloud environments (AWS, Azure, Google Cloud).Ensure data quality and integrity through rigorous testing and validation.Continuously evaluate and integrate new technologies and tools to improve data processing capabilities.Manage distributed scheduled tasks on individual servers to a centralized ruleset.Work with the development team and database MSP to modernize the backup and recovery strategy for 3-4 SQL servers across different cloud environments.Support the addition of new tenants on the firm's production data platform.Qualifications:
Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field from a top-tier university.4-5 years of relevant professional experience in data engineering.Strong programming skills in SQL and Python.Proficiency in database administration.Proven experience migrating on-prem data and associated processes to cloud platforms (AWS, Azure, Google Cloud).Demonstrated ability to design and implement efficient and reliable data pipelines and ETL processes.Strong problem-solving skills and ability to troubleshoot complex data issues.Excellent communication skills and ability to work collaboratively in a team environment.A proactive mindset with a willingness to take ownership and drive projects to completion. Strong organizational skills along with a desire to continually be challenged.Experience with monitoring, CI/CD, Load Balancing, and Testing tools (e.g. Redgate, Datadog).
#J-18808-Ljbffr
Full-time/PermanentOnsite - NYCResponsibilities:
Take ownership of the entire data engineering lifecycle, including design, implementation, performance tuning, and monitoring.Manage and work with the firm's database managed service provider (MSP) to manage and monitor the firm's evolving database requirements.Design, develop, and maintain scalable data pipelines and ETL processes critical to support business operations and analytics. (Python / SQL)Optimize and tune data systems for high performance and reliability.Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.Implement and manage data storage solutions in cloud environments (AWS, Azure, Google Cloud).Ensure data quality and integrity through rigorous testing and validation.Continuously evaluate and integrate new technologies and tools to improve data processing capabilities.Manage distributed scheduled tasks on individual servers to a centralized ruleset.Work with the development team and database MSP to modernize the backup and recovery strategy for 3-4 SQL servers across different cloud environments.Support the addition of new tenants on the firm's production data platform.Qualifications:
Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field from a top-tier university.4-5 years of relevant professional experience in data engineering.Strong programming skills in SQL and Python.Proficiency in database administration.Proven experience migrating on-prem data and associated processes to cloud platforms (AWS, Azure, Google Cloud).Demonstrated ability to design and implement efficient and reliable data pipelines and ETL processes.Strong problem-solving skills and ability to troubleshoot complex data issues.Excellent communication skills and ability to work collaboratively in a team environment.A proactive mindset with a willingness to take ownership and drive projects to completion. Strong organizational skills along with a desire to continually be challenged.Experience with monitoring, CI/CD, Load Balancing, and Testing tools (e.g. Redgate, Datadog).
#J-18808-Ljbffr