Avispa Technology
Data Engineer
Avispa Technology, San Francisco, California, United States, 94199
Data Engineer 14807
A leading professional networking company is seeking a
Data Engineer.
The successful candidate will support our marketing solutions execution of complex data development and database efficiency, with a deep focus on improving the data quality, accuracy, and timeliness for marketing related products. This role will interact directly with Engineering teams to optimize data operations for product metrics to surface insights at terabyte scale, globally. As our data and business needs increase, we continue to iterate and evolve our processes, best-practice documentation, and deep-dive retrospectives to inform our go/no go decision making. The ideal candidate is comfortable advising a team of data professionals, navigating our tech stack, and recommending solutions to complex data problems. The company offers a great work environment. Data Engineer Pay and Benefits: Hourly pay:
$65-$90/hr
(Pay varies based on the candidate's experience and location) Worksite: Leading professional development and networking company (San Francisco, CA - Hybrid (2 days a week between Tues through Thurs), Open to remote candidates in the United States) W2 Employment,
Group Medical, Dental, Vision, Life, Retirement Savings Program 40 hours/week, 6 Month Assignment Data Engineer Responsibilities: Design, develop, and manage data pipelines and workflows to enable efficient and accurate data processing using Trino SQL/Spark SQL warehoused in HDFS datasets Effectively perform code designs and reviews/approve test cases Implement data quality checks and audits to maintain high data accuracy and integrity Produce elegant and efficient designs, high performance, and scalable code that allows for easy extension to future needs Collaborate with cross-functional teams, especially data engineering, to understand data requirements and implement robust data solutions Work closely with data domain experts to gather data requirements, translate business needs into technical specifications, and communicate data insights effectively for sales representative workflow efficiency Optimize data storage for performance and scalability, ensuring efficient data Extraction, Transformation and Load (ETL) Develop and maintain documentation related to data pipelines, QA, metrics, and data policy as it relates to best practice, compliance and GDPR. Stay up to date with industry best practices and emerging trends in data engineering and analytics, including Generative AI as it impacts our data operations. Data Engineer Qualifications: 2+ years in using SQL and experience optimizing SQL databases for performance (Trino SQL, or Spark). BA/BS in engineering, computer science, or related technical field (such as statistics, or data science) is preferred. Demonstrated experience in managing data pipelines (like HDFS), data repository (like GitHub), workflows (like Apache Airflow), and ETL (best practice coding). Experience working with multiple stakeholders, setting project priorities and delivering on Objectives and Key Results (OKRs). Experience automating script changes in Python. Program Manager experience is preferred. Demonstrated experience in managing data pipelines in HDFS is preferred. Experience running a scrum team and using Jira is preferred. Ability to communicate complex technical concepts to both technical and non-technical individuals. Excellent analytical skills, designing data workflows and analyzing data for anomalies, or setting data quality thresholds via automated solutions is preferred. Familiarity with data governance principles is preferred. Spark SQL is preferred. Data analysis and data engineering.
#J-18808-Ljbffr
Data Engineer.
The successful candidate will support our marketing solutions execution of complex data development and database efficiency, with a deep focus on improving the data quality, accuracy, and timeliness for marketing related products. This role will interact directly with Engineering teams to optimize data operations for product metrics to surface insights at terabyte scale, globally. As our data and business needs increase, we continue to iterate and evolve our processes, best-practice documentation, and deep-dive retrospectives to inform our go/no go decision making. The ideal candidate is comfortable advising a team of data professionals, navigating our tech stack, and recommending solutions to complex data problems. The company offers a great work environment. Data Engineer Pay and Benefits: Hourly pay:
$65-$90/hr
(Pay varies based on the candidate's experience and location) Worksite: Leading professional development and networking company (San Francisco, CA - Hybrid (2 days a week between Tues through Thurs), Open to remote candidates in the United States) W2 Employment,
Group Medical, Dental, Vision, Life, Retirement Savings Program 40 hours/week, 6 Month Assignment Data Engineer Responsibilities: Design, develop, and manage data pipelines and workflows to enable efficient and accurate data processing using Trino SQL/Spark SQL warehoused in HDFS datasets Effectively perform code designs and reviews/approve test cases Implement data quality checks and audits to maintain high data accuracy and integrity Produce elegant and efficient designs, high performance, and scalable code that allows for easy extension to future needs Collaborate with cross-functional teams, especially data engineering, to understand data requirements and implement robust data solutions Work closely with data domain experts to gather data requirements, translate business needs into technical specifications, and communicate data insights effectively for sales representative workflow efficiency Optimize data storage for performance and scalability, ensuring efficient data Extraction, Transformation and Load (ETL) Develop and maintain documentation related to data pipelines, QA, metrics, and data policy as it relates to best practice, compliance and GDPR. Stay up to date with industry best practices and emerging trends in data engineering and analytics, including Generative AI as it impacts our data operations. Data Engineer Qualifications: 2+ years in using SQL and experience optimizing SQL databases for performance (Trino SQL, or Spark). BA/BS in engineering, computer science, or related technical field (such as statistics, or data science) is preferred. Demonstrated experience in managing data pipelines (like HDFS), data repository (like GitHub), workflows (like Apache Airflow), and ETL (best practice coding). Experience working with multiple stakeholders, setting project priorities and delivering on Objectives and Key Results (OKRs). Experience automating script changes in Python. Program Manager experience is preferred. Demonstrated experience in managing data pipelines in HDFS is preferred. Experience running a scrum team and using Jira is preferred. Ability to communicate complex technical concepts to both technical and non-technical individuals. Excellent analytical skills, designing data workflows and analyzing data for anomalies, or setting data quality thresholds via automated solutions is preferred. Familiarity with data governance principles is preferred. Spark SQL is preferred. Data analysis and data engineering.
#J-18808-Ljbffr