JobRialto
Senior Data Engineer
JobRialto, Sunnyvale, California, United States, 94087
Job Summary:
We are looking for a skilled data professional to join our team and work on data engineering and processing tasks. This role will focus on querying and managing large datasets, automating data workflows, and building data models to enhance our analytics capabilities. The ideal candidate will have a strong background in Python programming, SQL, and cloud platforms like GCP, with hands-on experience in ETL/ELT processes.
Key Responsibilities:Data Management & Querying: Proficiency in SQL to manage and query large datasets for insights and reporting.Data Processing & Automation: Utilize Python to process and automate data workflows, ensuring efficient data handling and transformation.Big Data Technologies: Work with Pyspark to process large-scale datasets and enhance data pipeline performance.Cloud Platform Integration: Leverage GCP services like BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage for managing cloud-based data systems.Data Modeling & Architecture: Design and model data structures to ensure efficient data storage and accessibility.ETL/ELT Processes: Implement best practices in ETL/ELT processes to ensure seamless data integration, transformation, and loading into target systems.Problem Solving: Tackle complex data challenges and deliver actionable solutions to improve data processing efficiency and effectiveness.Collaboration: Work closely with business and technical teams to understand data requirements and ensure that technical solutions align with business goals.Communication: Effectively communicate technical information to both technical and non-technical stakeholders.Required Qualifications:
Technical Skills:
SQL: Proficient in querying and managing large datasets.Python: Strong programming skills for data processing and automation tasks.Pyspark: Experience in working with Pyspark to process large data sets efficiently.Cloud Platforms: Experience with GCP services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage).Data Modeling: Expertise in data modeling and designing data architectures for large-scale systems.ETL/ELT: In-depth knowledge of ETL/ELT processes and best practices.Problem-Solving Skills: Proven ability to tackle complex data-related problems and implement effective solutions.Communication: Strong verbal and written communication skills with the ability to explain technical concepts to non-technical audiences.
Preferred Qualifications (if any):
Experience with other cloud platforms like Azure is a plus.
Familiarity with data visualization tools (e.g., Looker, Power BI) is advantageous.
Certifications (if any):
Relevant certifications in Data Engineering, Cloud Platforms, or Big Data technologies are a plus (e.g., Google Cloud Professional Data Engineer or AWS Certified Big Data - Specialty).
Education:
Bachelors Degree
We are looking for a skilled data professional to join our team and work on data engineering and processing tasks. This role will focus on querying and managing large datasets, automating data workflows, and building data models to enhance our analytics capabilities. The ideal candidate will have a strong background in Python programming, SQL, and cloud platforms like GCP, with hands-on experience in ETL/ELT processes.
Key Responsibilities:Data Management & Querying: Proficiency in SQL to manage and query large datasets for insights and reporting.Data Processing & Automation: Utilize Python to process and automate data workflows, ensuring efficient data handling and transformation.Big Data Technologies: Work with Pyspark to process large-scale datasets and enhance data pipeline performance.Cloud Platform Integration: Leverage GCP services like BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage for managing cloud-based data systems.Data Modeling & Architecture: Design and model data structures to ensure efficient data storage and accessibility.ETL/ELT Processes: Implement best practices in ETL/ELT processes to ensure seamless data integration, transformation, and loading into target systems.Problem Solving: Tackle complex data challenges and deliver actionable solutions to improve data processing efficiency and effectiveness.Collaboration: Work closely with business and technical teams to understand data requirements and ensure that technical solutions align with business goals.Communication: Effectively communicate technical information to both technical and non-technical stakeholders.Required Qualifications:
Technical Skills:
SQL: Proficient in querying and managing large datasets.Python: Strong programming skills for data processing and automation tasks.Pyspark: Experience in working with Pyspark to process large data sets efficiently.Cloud Platforms: Experience with GCP services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage).Data Modeling: Expertise in data modeling and designing data architectures for large-scale systems.ETL/ELT: In-depth knowledge of ETL/ELT processes and best practices.Problem-Solving Skills: Proven ability to tackle complex data-related problems and implement effective solutions.Communication: Strong verbal and written communication skills with the ability to explain technical concepts to non-technical audiences.
Preferred Qualifications (if any):
Experience with other cloud platforms like Azure is a plus.
Familiarity with data visualization tools (e.g., Looker, Power BI) is advantageous.
Certifications (if any):
Relevant certifications in Data Engineering, Cloud Platforms, or Big Data technologies are a plus (e.g., Google Cloud Professional Data Engineer or AWS Certified Big Data - Specialty).
Education:
Bachelors Degree