Lead Data Engineer

Cadent powers the evolution of TV brand advertising. We provide marketers, agencies, operators, and media owners with data-driven solutions for buying and selling TV advertising. By connecting brands with opportunities across national inventory sources—cable, broadcast, and digital media—our technology improves efficiencies and boosts the results of linear, addressable, and cross-screen campaigns. Right now we are looking for a self-motivated and talented Lead Data Engineer to join the team responsible for ingesting, processing, and distributing the data used and created by Cadent’s data-driven business systems. The ideal candidate should possess a deep understanding of data engineering principles, excellent leadership skills, and a passion for driving impactful data-driven solutions. About you: • You have a strong aptitude for querying data sets from heterogeneous sources—both structured such as SQL Server, MYSQL and Vertica and unstructured like Excel,CSV using simple and advanced SQL • You have hands-on experience with Databricks (Spark), the AWS ecosystem including (S3, Athena, Glue, Lambda, IAM, EC2) • You have hands-on experience in building large-scale distributed applications, including streaming and Batch pipelines and processing petabytes of data. • You have hands-on experience with the Databricks platform to create ETL pipelines using notebooks, scheduling jobs and create optimized spark cluster configuration for various ETL workloads. • You have experience using Atlassian products including JIRA, Confluence etc • You have conducted code reviews to ensure code quality, adherence to coding standards, and best practices. • You are responsible for working closely with cross-functional teams, including product management and quality assurance, to ensure smooth and timely releases. • You can easily work with a diverse team to achieve a common goal and participate in scrum meetings. • You’re willing to learn new skills and technologies • Any experience with the following a plus but not required: • Data warehousing platforms such as Redshift, Snowflake • Operating systems such as Unix and Linux • Reporting and Analytics tools such as Pentaho, Sisense,SuperSet, Druid etc. • Programmatically interfacing with FTP/SFTP • Microsoft Graph API What you will do: • Participate in scrums, design meetings, and collaborate with quality assurance, business stakeholders, data engineering coworkers, product, software development, and data scientists, as needed for each project • Ability to understand and articulate requirements to/from technical and non-technical audiences working alongside a data science & engineering team • Create design documentation and review it with peers and subject matter experts • Learn the Cadent Data Lake architecture principles and libraries and actively participate in maintaining and improving the core codebase • Build and maintain cost effective and performant Databricks workflows using various data sources through each layer of the Cadent Data Lake; and make data available for querying and reporting • Support the business with ad hoc data analysis to answer data questions and do proof of concepts • Learn the Cadent media and TV business as well as it’s nomenclature, media math, and metrics • Show initiative to identify critical problems and propose effective solutions, while exploring, learning and suggesting the most suitable technology for the problem What you will do: • Participate in scrums, design meetings, and collaborate with quality assurance, business stakeholders, data engineering coworkers, product, software development, and data scientists, as needed for each project • Ability to understand and articulate requirements to/from technical and non-technical audiences working alongside a data science & engineering team • Create design documentation and review it with peers and subject matter experts • Learn the Cadent Data Lake architecture principles and libraries and actively participate in maintaining and improving the core codebase • Build and maintain cost effective and performant Databricks workflows using various data sources through each layer of the Cadent Data Lake; and make data available for querying and reporting • Support the business with ad hoc data analysis to answer data questions and do proof of concepts • Learn the Cadent media and TV business as well as it’s nomenclature, media math, and metrics • Show initiative to identify critical problems and propose effective solutions, while exploring, learning and suggesting the most suitable technology for the problem If the leading edge of media technology is the place you want to be, please contact us today and let’s start the conversation! Cadent is an Equal Opportunity Employer and is committed to supporting all it’s employees when it comes to Inclusion & Diversity. Cadent’s policy is to provide equal opportunity for applicants & employees without regard to race, color, religion, creed, gender, gender identity or expression, sexual identity or orientation, age, national origin or ancestry, citizenship, disability or medical condition (including pregnancy, childbirth, or related medical condition), sexual and reproductive health decisions, genetic information, marital status (including domestic partnerships and civil unions), pregnancy, culture ancestry, familial or caregiver status, military status, veteran status, socioeconomic status, unemployment status, status as a victim of domestic violence or any other basis prohibited by law. and will not discriminate against the basis of disability. This commitment is honored when it comes to decisions on hiring, recruiting, training, promotions, compensations, benefits, transfers and terminations. Cadent is seeking to actively engage with our employees from a wide variety of cultures and to connect with our clients differently. Our workforce has generational diversity that supports greater innovation when we maximize representation of all diversity. Our active employee resource groups promote engagement across all groups of individuals that are represented within the company and externally