Scala Developer IV

Responsibilities: Design and build scalable data pipelines to ingest, transform, and deliver company data to internal and external stakeholders, including multiple reporting teams and external media partners. Develop and maintain data pipelines for real-time and batch processing (hourly). Implement new data exports and update business logic for various data feeds. Contribute to the development of billion-scale streaming data pipeline infrastructure. Ensure data quality and integrity throughout the ingestion and processing lifecycle. Typical Day-to-Day: Participate in brief morning status meeting (15 minutes) and developer sync with the greater team (10-30 minutes). Work on assigned tasks with minimal meetings outside of weekly team syncs. Agile development environment with 2-week sprints and production releases on Tuesday during working hours. Work hours based on Mountain Standard Time (MST). On call responsibilities (Mon-Fri, 8am-7pm MST, about once every six weeks) Skills and Experience: 3 years of experience coding in Scala with a strong foundation in functional programming concepts. Experience with Apache Spark Structured Streaming (in Scala), with a focus on migrating existing projects and building new features using Spark. Experience with Apache Kafka for real-time message queuing, with an understanding of Consumer and Producer offset management and at-least-once & at-most-once semantics. Familiarity with container orchestration concepts, preferably Kubernetes. Basic understanding of AWS cloud technologies S3, Athena, Lambda, IAM, EMR Comfortable using SQL for data exploration. Experience supporting large-scale data feeds. Nice to have: experience building infrastructure with Terraform We are looking for a candidate with a strong foundation in Scala who is transitioning to a data background rather than a PySpark Data Engineer who is new to Scala or streaming concepts. Additional Information: This role offers the opportunity to work on challenging and impactful projects within a collaborative team environment. You will be instrumental in building and maintaining the data infrastructure that supports critical business decisions. Requirements - 3 years' Scala programming experience - Spark streaming experience (using this for batch jobs is a plus) - Kafka Plusses: - Akka Streams data library experience - Core Java development background - SQL database experience - Python - Lambda Beacon Hill is an Equal Opportunity Employer that values the strength diversity brings to the workplace. Individuals with Disabilities and Protected Veterans are encouraged to apply. If you would like to complete our voluntary self-identification form, please click here or copy and paste the following link into an open window in your browser: https://jobs.beaconhillstaffing.com/eeoc/ Completion of this form is voluntary and will not affect your opportunity for employment, or the terms or conditions of your employment. This form will be used for reporting purposes only and will be kept separate from all other records. Company Profile: Beacon Hill Technologies, a premier National Information Technology Staffing Group, provides world class technology talent across all industries utilizing a complete suite of staffing services. Beacon Hill Technologies' dedicated team of recruiting and staffing experts consistently delivers quality IT professionals to solve our customers' technical and business needs. Beacon Hill Technologies covers a broad spectrum of IT positions, including Project Management and Business Analysis, Programming/Development, Database, Infrastructure, Quality Assurance, Production/Support and ERP roles. Learn more about Beacon Hill and our specialty divisions, Beacon Hill Associates, Beacon Hill Financial, Beacon Hill HR, Beacon Hill Legal, Beacon Hill Life Sciences and Beacon Hill Technologies by visiting www.bhsg.com We look forwar to working with you. Beacon Hill. Employing the Future