Senior Data Engineer

SummaryThe Wikimedia Foundation is looking for a Senior Data Engineer to join our Data Engineering team. As a Data Engineer, you will be responsible for building, maintaining, and expanding the shared data infrastructure that supports insights, research, and new data products, in the Foundation as well as the Wiki Movement. This includes everything from building scalable pipeline infrastructure, and working on the event streaming platform to providing scalable data-serving solutions.We are a fully remote, internationally distributed team. We see each other in person 1-2 times a year during one of our off-sites (the last few have been in places like Berlin, Copenhagen, and New York) or Wikimania, the annual international conference for the Wiki community.Open to candidates located in time zones UTC-8 to UTC+2.You are responsible for:Building scalable data pipeline infrastructure, libraries and processes using Airflow, Spark, Flink, KafkaImplementing data quality monitoring that alerts the team of possible data issuesImplementing a data governance and lineage solution for all Wikimedia dataDesigning the shared data platform that supports use cases for critical aspects of the Wikimedia mission: product analytics, harassment prevention, image classification, bot detection, DDoS attacks flagging and many moreImprove operational excellence of the data platformSkills and Experience:5+ years of relevant industry experienceAdvanced working knowledge of SQL, relational databases, query authoring, ideally in a variety of flavors (in our team alone we deal with MariaDB, HiveQL, CassandraQL, Spark SQL and Presto)Experience with one or more programming languages such as Python, Scala, and JavaExperience building data pipelines using tools such as Airflow, Spark, Gobblin, YarnFamiliarity with stream processing systems using Kafka, Spark streaming and/or FlinkExcellent written and verbal communication skillsStrong interpersonal and collaboration skillsBS or MS degree, preferably in Computer Science, or equivalent work experienceQualities that are important to us:Commitment to the mission of the organization and our valuesCommitment to our guiding principlesCommitment to diversity, equity, and inclusionCross-cultural sensitivity and awarenessCollaborative working experienceAdditionally, we’d love it if you have:Experience with HadoopUnderstanding of related disciplines including Machine Learning, Statistics, Privacy and AlgorithmsExperience working with site reliability engineersAbout the Wikimedia FoundationThe Wikimedia Foundation is the nonprofit organization that operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge freely. We host Wikipedia and the Wikimedia projects, build software experiences for reading, contributing, and sharing Wikimedia content, support the volunteer communities and partners who make Wikimedia possible, and advocate for policies that enable Wikimedia and free knowledge to thrive.The Wikimedia Foundation is a charitable, not-for-profit organization that relies on donations. We receive donations from millions of individuals around the world, with an average donation of about $15. We also receive donations through institutional grants and gifts. The Wikimedia Foundation is a United States 501(c)(3) tax-exempt organization with offices in San Francisco, California, USA.As an equal opportunity employer, the Wikimedia Foundation values having a diverse workforce and continuously strives to maintain an inclusive and equitable workplace. We encourage people with a diverse range of backgrounds to apply. We do not discriminate against any person based upon their race, traits historically associated with race, religion, color, national origin, sex, pregnancy or related medical conditions, parental status, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other legally protected characteristics.The Wikimedia Foundation is a remote-first organization with staff members including contractors based in more than 50 countries. Salaries at the Wikimedia Foundation are set in a way that is competitive, equitable, and consistent with our values and culture. The anticipated annual pay range of this position for applicants based within the United States is US$ 105,309 to US$ 163,646 with multiple individualized factors, including cost of living in the location, being the determinants of the offered pay. For applicants located outside of the US, the pay range will be adjusted to the country of hire. We neither ask for nor take into consideration the salary history of applicants. The compensation for a successful applicant will be based on their skills, experience and location.All applicants can reach out to their recruiter to understand more about the specific pay range for their location during the interview process.If you are a qualified applicant requiring assistance or an accommodation to complete any step of the application process due to a disability, you may contact us at recruiting@wikimedia.org or +1 (415) 839-6885.

#J-18808-Ljbffr