Dice
Senior Big Data Engineer
Dice, Charlotte, North Carolina, United States, 28245
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Entech, is seeking the following. Apply via Dice today!We are living and working in the midst of a digital revolution, and the demand for optimized technology practices has never been more critical for businesses. At Entech, it’s our mission to help companies navigate the changing environment and accelerate digital realization through the power of solutions, services, and product platforms. With comprehensive end-to-end offerings, we advise, engineer, and manage, ensuring our clients proactively drive toward success in this ever-evolving digital landscape.Our team is growing, and we’re searching for a Senior Big Data Engineer who is a creative problem-solver with their fingertips on the latest industry practices to help our clients achieve digital realization. You’ll join a group of industry veterans, blending the power of tried-and-true methods with cutting-edge technologies. We prioritize authentic partnerships in a collaborative environment, and we’re excited to welcome a Big Data Engineer who is eager to bring their expertise to our team.Responsibilities
The person in this role will play a crucial role in building scalable and cost-effective data pipelines, data lakes, and analytics systems.Data Ingestion:
Implement data ingestion processes to collect data from various sources, including databases, streaming data, and external APIs.Data Transformation:
Develop ETL (Extract, Transform, Load) processes to transform and cleanse raw data into a structured and usable format for analysis.Data Storage:
Manage and optimize data storage solutions, including Amazon S3, Redshift, and other AWS storage services.Data Processing:
Utilize AWS services like AWS Glue, Amazon EMR, and AWS Lambda to process and analyze large datasets.Data Monitoring and Optimization:
Continuously monitor and optimize data pipelines and infrastructure for performance, cost-efficiency, and scalability.Integration:
Collaborate with data scientists, analysts, and other stakeholders to integrate AWS-based solutions into data analytics and reporting platforms.Documentation:
Maintain thorough documentation of data engineering processes, data flows, and system configurations.Scalability:
Design AWS-based solutions that can scale to accommodate growing data volumes and changing business requirements.Cost Management:
Implement cost-effective solutions by optimizing resource usage and recommending cost-saving measures.Troubleshooting:
Diagnose and resolve AWS-related issues to minimize downtime and disruptions.Qualifications
Must Have:Strong experience with AWS services on big data platforms.Strong Python experience in building Dash applications in AWS Spark.Shell scripting and Linux knowledge.Nice to have:Investment Knowledge preferred.Looking for candidates who have worked in Investment/Finance space.Machine Learning Skills.Qualifications:Educational Background: A bachelor's degree in computer science, information technology, or a related field is typically required.A master's degree is preferred with a GPA of 3.8+Problem-Solving:
Strong analytical and problem-solving skills to address complex data engineering challenges.Communication Skills:
Effective communication and collaboration skills to work with cross-functional teams.Company Benefits
Health, Dental, Vision, 401(k), 25 days of Paid Time Off (PTO).Additionally, Entech is an equal-opportunity employer.This is an On-site Hybrid work schedule based out of our Charlotte, North Carolina Office.No Third-party candidates will be accepted; this is strictly a direct employment opportunity with Entech.
#J-18808-Ljbffr
The person in this role will play a crucial role in building scalable and cost-effective data pipelines, data lakes, and analytics systems.Data Ingestion:
Implement data ingestion processes to collect data from various sources, including databases, streaming data, and external APIs.Data Transformation:
Develop ETL (Extract, Transform, Load) processes to transform and cleanse raw data into a structured and usable format for analysis.Data Storage:
Manage and optimize data storage solutions, including Amazon S3, Redshift, and other AWS storage services.Data Processing:
Utilize AWS services like AWS Glue, Amazon EMR, and AWS Lambda to process and analyze large datasets.Data Monitoring and Optimization:
Continuously monitor and optimize data pipelines and infrastructure for performance, cost-efficiency, and scalability.Integration:
Collaborate with data scientists, analysts, and other stakeholders to integrate AWS-based solutions into data analytics and reporting platforms.Documentation:
Maintain thorough documentation of data engineering processes, data flows, and system configurations.Scalability:
Design AWS-based solutions that can scale to accommodate growing data volumes and changing business requirements.Cost Management:
Implement cost-effective solutions by optimizing resource usage and recommending cost-saving measures.Troubleshooting:
Diagnose and resolve AWS-related issues to minimize downtime and disruptions.Qualifications
Must Have:Strong experience with AWS services on big data platforms.Strong Python experience in building Dash applications in AWS Spark.Shell scripting and Linux knowledge.Nice to have:Investment Knowledge preferred.Looking for candidates who have worked in Investment/Finance space.Machine Learning Skills.Qualifications:Educational Background: A bachelor's degree in computer science, information technology, or a related field is typically required.A master's degree is preferred with a GPA of 3.8+Problem-Solving:
Strong analytical and problem-solving skills to address complex data engineering challenges.Communication Skills:
Effective communication and collaboration skills to work with cross-functional teams.Company Benefits
Health, Dental, Vision, 401(k), 25 days of Paid Time Off (PTO).Additionally, Entech is an equal-opportunity employer.This is an On-site Hybrid work schedule based out of our Charlotte, North Carolina Office.No Third-party candidates will be accepted; this is strictly a direct employment opportunity with Entech.
#J-18808-Ljbffr