Coactive AI
Backend Engineer, Streaming
Coactive AI, San Jose, California, United States, 95199
What we need:
We are looking for a Backend Engineer to develop and maintain scalable streaming data pipelines and workflows. Your role will involve integrating diverse data sources, transforming datasets for machine learning models, and ensuring efficient data processing. Collaborating with backend engineers, ML engineers, and product teams, you'll optimize pipeline performance and uphold data quality standards. You'll also provide expertise in best practices for ingestion frameworks and streaming pipeline development. Monitoring and observability implementation will be key to maintaining high data quality. Your work will shape our engineering practices and contribute to our growing company culture. We seek candidates with extensive experience in data engineering, distributed processing ecosystems like Spark and Kafka, and proficiency in programming languages like Python. Knowledge of database internals, query optimizations, and a strong testing ethos are essential.
What you’ll do:
Design, develop and support resilient, scalable data pipelines and workflows that integrate various sources of structured and unstructured data
Design and develop customizations and data wrangling needed to transform datasets in a way that is best suited for processing by our ML models.
Collaborate closely with backend engineers, ML engineers and product teams to assess pipeline performance and improve results as needed.
Provide guidance on best practices around ingestion frameworks and streaming pipelines development.
Implement monitoring and observability to ensure data quality and efficient data processing.
Shape engineering best practices and
company culture
as we grow
What you’ll do:
A proven track record with 8+ years of professional experience in data engineering, building data pipelines, ETL, streaming data ingestion, distributed processing and databases with large datasets.
Experience in distributed processing ecosystems like Spark, Flink, Kafka, Kubernetes, etc.
Experience designing systems with event driven architecture/streams and message queues.
Proficiency in design of algorithms, data structures, design patterns and deploying scalable microservices.
Deep knowledge of database internals, query planning and optimizations
Experience with one or more general purpose programming languages including but not limited to Python, Go or Java (we use Python)
Experience working with NoSQL databases including but not limited to MongoDB, ElasticSearch, etc and relational databases like Postgres, etc.
Strong testing ethos and knowledge of best practices, including code quality, accessibility, performance, and security
What you can expect from us:
Location: San Jose, California (hybrid, with three days in office)
The estimated annual base salary for this position is between
$180,000-$220,000
.*
At Coactive, cash salary is only one part of our total compensation package. Other benefits for this position include, but are not limited to:
Market leading equity grants
100% medical, dental, & vision coverage for you
Medical, dental, & vision partially covered for your dependents
Unlimited PTO
Social events ranging from book clubs, happy hours, and hiking to board game nights and games of Mario Kart.
Further, you can expect a supportive work environment from us.
We build products, but we develop people.
*Actual pay is dependent on an individual candidate’s professional background, experience, skills and qualifications, as well as market demand and business demands. This pay range is subject to change and may be modified in the future. The salary, other compensation, and benefits information is accurate as of the date of this posting.
#J-18808-Ljbffr
We are looking for a Backend Engineer to develop and maintain scalable streaming data pipelines and workflows. Your role will involve integrating diverse data sources, transforming datasets for machine learning models, and ensuring efficient data processing. Collaborating with backend engineers, ML engineers, and product teams, you'll optimize pipeline performance and uphold data quality standards. You'll also provide expertise in best practices for ingestion frameworks and streaming pipeline development. Monitoring and observability implementation will be key to maintaining high data quality. Your work will shape our engineering practices and contribute to our growing company culture. We seek candidates with extensive experience in data engineering, distributed processing ecosystems like Spark and Kafka, and proficiency in programming languages like Python. Knowledge of database internals, query optimizations, and a strong testing ethos are essential.
What you’ll do:
Design, develop and support resilient, scalable data pipelines and workflows that integrate various sources of structured and unstructured data
Design and develop customizations and data wrangling needed to transform datasets in a way that is best suited for processing by our ML models.
Collaborate closely with backend engineers, ML engineers and product teams to assess pipeline performance and improve results as needed.
Provide guidance on best practices around ingestion frameworks and streaming pipelines development.
Implement monitoring and observability to ensure data quality and efficient data processing.
Shape engineering best practices and
company culture
as we grow
What you’ll do:
A proven track record with 8+ years of professional experience in data engineering, building data pipelines, ETL, streaming data ingestion, distributed processing and databases with large datasets.
Experience in distributed processing ecosystems like Spark, Flink, Kafka, Kubernetes, etc.
Experience designing systems with event driven architecture/streams and message queues.
Proficiency in design of algorithms, data structures, design patterns and deploying scalable microservices.
Deep knowledge of database internals, query planning and optimizations
Experience with one or more general purpose programming languages including but not limited to Python, Go or Java (we use Python)
Experience working with NoSQL databases including but not limited to MongoDB, ElasticSearch, etc and relational databases like Postgres, etc.
Strong testing ethos and knowledge of best practices, including code quality, accessibility, performance, and security
What you can expect from us:
Location: San Jose, California (hybrid, with three days in office)
The estimated annual base salary for this position is between
$180,000-$220,000
.*
At Coactive, cash salary is only one part of our total compensation package. Other benefits for this position include, but are not limited to:
Market leading equity grants
100% medical, dental, & vision coverage for you
Medical, dental, & vision partially covered for your dependents
Unlimited PTO
Social events ranging from book clubs, happy hours, and hiking to board game nights and games of Mario Kart.
Further, you can expect a supportive work environment from us.
We build products, but we develop people.
*Actual pay is dependent on an individual candidate’s professional background, experience, skills and qualifications, as well as market demand and business demands. This pay range is subject to change and may be modified in the future. The salary, other compensation, and benefits information is accurate as of the date of this posting.
#J-18808-Ljbffr