Lead Data Engineer

Our PurposeWe work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments and businesses realize their greatest potential. Our decency quotient, or DQ, drives our culture and everything we do inside and outside of our company. We cultivate a culture of inclusion for all employees that respects their individual strengths, views, and experiences. We believe that our differences enable us to be a better team – one that makes better decisions, drives innovation and delivers better business results.Title and SummaryLead Data EngineerOverview:Mastercard’s Data Platforms and Engineering Services is seeking a Lead Data Engineer to join our team. As Lead Data Engineer you will have the opportunity to build high performance data pipelines to load into Mastercard Data Warehouse. Our Data Warehouse provides analytical capabilities to number of business users who help different customers provide answer to their business problems through data. You will play a vital role within a rapidly growing organization, while working closely with experienced and driven engineers to solve challenging problems.Mastercard is the global technology company behind the world’s fastest payments processing network. We are a vehicle for commerce, a connection to financial systems for the previously excluded, a technology innovation lab, and the home of Priceless. We ensure every employee can be a part of something bigger and to change lives. We believe as our company grows, so should you. We believe in connecting everyone to endless, priceless possibilities.The Data Platforms and Engineering Services is focused on enabling insights into Mastercard network and help build data-driven products by curating and preparing data in a secure and reliable manner. Moving to a “Unified and Fault-Tolerant Architecture for Data Ingestion and Processing” is critical to achieving this mission.Role: • Partner on the design the next implementation of Mastercard secure, global data and insight architecture, building the data integration and processing capabilities and operationalizing “Unified Data Acquisition and Processing (UDAP) platform”• Identify and proactively resolve performance bottlenecks in data systems.• Collaborate with the customer support group to address performance issues in the field. • Explore automation opportunities and develop tools to automate some of the day-to-day operations tasks.• Maintain dashboards and provide performance metrics to monitor system health.• Conceptualize and implement proactive monitoring where possible to catch issues early• Experiment with tools to streamline development, testing, deployment, and data pipeline management.• Work with cross functional agile teams to drive projects through full development cycle.• Promote best practices in data engineering across teams.• Collaborate with other data engineering teams to improve the data engineering ecosystem and talent within Mastercard.• Creatively solve problems when facing constraints, whether it is the number of developers, quality or quantity of data, compute power, storage capacity or just time.• Maintain awareness of relevant technical and product trends through self-learning/study, training classes and job shadowing.All About You:• Extensive background in data engineering, with a strong background in data pipeline architecture, ETL, and data modeling.• Expertise in data warehousing solutions, database management, and optimization techniques, particularly product or service-based organization.• Previous experience in Site Reliability Engineering or DevOps, along with software engineering or software architecture best practices. • Strong skills in scalability, performance, and stability with expert knowledge in Linux, scripting (Shell and Python preferred) and troubleshooting complex systems.• Operational experience with Big Data stacks (Hadoop ecosystem, Spark is a plus), real-time data streaming frameworks (Kafka, NiFi), and network/server communication troubleshooting.• Proficiency in performance tuning across databases, ETL jobs, and SQL scripts.• Skilled in enterprise metrics/monitoring using tools like Splunk, Druid, and Grafana, with cloud computing experience, especially with Azure or AWS.• Data-driven analytical mindset with a proven problem-solving track record.• Agile methodologies experience. • Proven track record in collaborating with cross-functional teams.• Excellent written and verbal communication skills with the ability to convey complex technical concepts to a diverse audience.• Ability to deal with multiple and competing priorities, structure and manage work streams, and set clear expectations and deadlines.• Bachelor’s degree in Computer Science, Software Engineering, or a related field. Equivalent practical experience will be considered.#LI-NF1Mastercard is an inclusive equal opportunity employer that considers applicants without regard to gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law. In the US or Canada, if you require accommodations or assistance to complete the online application process or during the recruitment process, please contact reasonable_accommodation@mastercard.com and identify the type of accommodation or assistance you are requesting. Do not include any medical or health information in this email. The Reasonable Accommodations team will respond to your email promptly.Corporate Security ResponsibilityAll activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:Abide by Mastercard’s security policies and practices;Ensure the confidentiality and integrity of the information being accessed;Report any suspected information security violation or breach, andComplete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.In line with Mastercard’s total compensation philosophy and assuming that the job will be performed in the US, the successful candidate will be offered a competitive base salary based on location, experience and other qualifications for the role and may be eligible for an annual bonus or commissions depending on the role. Mastercard benefits for full time (and certain part time) employees generally include: insurance (including medical, prescription drug, dental, vision, disability, life insurance), flexible spending account and health savings account, paid leaves (including 16 weeks new parent leave, up to 20 paid days bereavement leave), 10 annual paid sick days, 10 or more annual paid vacation days based on level, 5 personal days, 10 annual paid U.S. observed holidays, 401k with a best-in-class company match, deferred compensation for eligible roles, fitness reimbursement or on-site fitness facilities, eligibility for tuition reimbursement, gender-inclusive benefits and many more.Pay RangesO'Fallon, Missouri: $138,000 - $221,000 USDJob SummaryJob number: R-202538Date posted : 2024-11-06Profession: OtherEmployment type: Full time