Director, Data Engineering Wikimedia Foundation

The Wikimedia Foundation is seeking an experienced engineering leader to serve as Director of Data Engineering for our Technology department. At the Wikimedia Foundation, we operate the world’s largest collaborative project: a top ten website, reaching a billion people globally every month, while incorporating the values of privacy, transparency and community that are so important to our users.Reporting to the VP of Data Science & Engineering, the Director of Data Engineering is a key member of the Foundation’s data team and an active participant in the strategic decision making framing the work of the technology department, the Wikimedia Foundation and the Wikimedia movement. Working closely with other Technology and Product teams, as well as our community of contributors and readers, you will help shape the next generation of data usage, analysis and access across all Wikimedia projects.This role is responsible for planning and executing on key data engineering initiatives spanning our work in artificial intelligence, machine learning, search, natural language processing and product analytics. As Director, you will lead a team of data engineers, site reliability engineers and software engineers in building and maintaining our entire analytics and events infrastructure, including key systems like Kafka, Cassandra, Druid and our Hadoop/Spark cluster. We build our stacks using fully open source components, we release our data publicly and our puppet repo is public.Here are some examples of projects we have tackled that you may be excited to lead:Releasing public data to the Wikimedia community and the world at large. Our public data offerings are used all over the world by companies and research institutions big and small. A popular example is the Wikipedia Clickstream (a.k.a. Wikipedia’s rabbit hole)Deploying an anomaly detection system to monitor Wikipedia accessibility over the world and detect possible outages (or censorship events)Integrating an open-source GPU software platform like AMD ROCm in Hadoop and in the Tensorflow-related ecosystemBuilding the Foundation’s new event data platform infrastructureMigrate our on-prem Hadoop infrastructure that holds petabytes of data to Hadoop3Evangelize privacy conscious ways to compute metrics. Privacy is key to the work we doWe’d like you to do these things:Partner closely with other teams and departments across the Wikimedia Foundation to define and experiment with machine learning products. These could be brand new feature offerings in Wikipedia or augmentation of existing workflows.Review and advise in code changes and technical decisions made by the teamRepresent team members within the organization and Wikimedia communitySupport and coach your team members in the development of their career pathsRecruit and hire new team membersWork closely with our Research, Architecture, Security, Site Reliability and Platform teams to define our next generation of data architectureEnsure data is available, reliable, consistent, accessible, secure, and available in a timely manner for external and internal stakeholders and in accordance with our privacy policyContribute to our culture by managing, coaching and developing team membersSkills and Experience:Deep experience in leading data science, machine learning, search or data engineering teams that is able to separate the hype in the artificial intelligence space from the reality of delivering production-ready data systems5+ years engineering leadership experienceDemonstrated ability to balance competing interests in a complex technical and social environmentProven success at all stages of the engineering process and product lifecycle, leading to significant, measurable impactPrevious hands-on experience in production big data and machine learning environments at scale.You are no stranger to error budgets, operations at scale and efforts to reduce toil.Experience building and leading diverse, international and remote-first teamsQualities that are important to us:You take a solutions-focused approach to challenging data and technical problemsA passion for people development, team culture and the management of ideasYou have a desire to show the world how data can be done while honoring the user’s right to privacyAdditionally, we’d love it if you have:Experience with modern machine learning, search and natural language processing platformsA track record of open source participationFluency or familiarity with languages in addition to EnglishSpent time having lived or worked outside your country of originExperience as a member of a volunteer communityThe Wikimedia Foundation is the nonprofit organization that hosts and operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge, free of interference.As an equal opportunity employer, the Wikimedia Foundation values having a diverse workforce and continuously strives to maintain an inclusive and equitable workplace. We encourage people with a diverse range of backgrounds to apply. We do not discriminate against any person based upon their race, religion, color, national origin, sex, pregnancy or related medical conditions, parental status, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other legally protected characteristics.If you are a qualified applicant requiring assistance or an accommodation to complete any step of the application process due to a disability, you may contact us at recruiting@wikimedia.org or (415) 839-6885.

#J-18808-Ljbffr