Data Engineer

Description: We’re looking for a highly capable Data Engineer to join our small dev team and help us with our pipeline for importing data and sharing it with the public. Our mission is to make the world’s data understandable and accessible; this means we cover an immense variety of topics, and collect an immense variety of data. This role’s focus is to organise this wealth of data and make it as useful as possible to us and to others.

Contract type: Contractor (full-time, flexible hours)

Location: Remote (US East & EU/African timezones preferred)

Deadline: Hiring on a rolling basis – please apply early, even if you are not available soon

Interview process: We will review applications on a rolling basis and contact candidates for intro calls. Shortlisted candidates will then be contacted for interviews and assessment, involving one or more live coding tests and an in-depth discussion of a past project. We aim to respond to applications within 7 days and to conclude all interviews within 30 days, subject to your availability. You will have the opportunity to ask questions and assess us too as you go.

Compensation: We will consider candidates at different experience levels. Compensation will be discussed early in the selection process and will depend on your profile and experience.

• Develop and evolve the data pipeline that powers our charts & analyses
• Develop a data model that can handle the immense variety of the world’s data
• Help us publish our data in whatever formats and places promote reuse
• Develop, extend and demonstrate open APIs for our data in core analytics languages
• Act as a bridge between the data management team and the dev team
• Support our data managers as expert users of our data pipeline
• Manage and operate small-scale cloud services as needed
• Encourage and nurture open source contributions to our projects

• 3+ years in a data engineering, data science or similar role
• Strong proficiency in the Python scientific computing stack and in SQL
• Prior experience with one or more cloud computing providers and their core offerings
• Extremely comfortable on the command-line and with unix shell
• Prior experience designing ETLs or data flows a bonus

• 🌎 Mission driven: motivated above all by our mission to unlock the world’s data and help the world tackle its big problems
• ⚖️ Economic: considers which changes will bring the team the most benefit when weighed against the time and effort involved
• 🧐 Independent: self-driven and capable of pushing through projects to completion
• ✍️ Communicator: excellent and proactive written communication, important in our full-remote setting
• 🌱 Gardener: routinely makes the small changes, improvements and bugfixes that keep projects at a high level of quality

• Machine learning skills or experience is not especially relevant to our work
• Large-scale data systems experience is not especially relevant to our work

• We currently work in 6-week cycles with 2-week cooldowns
  • You will have the focus time to do your best work
  • You will have flexibility around your exact hours, within a full-time loading
• Our stack
  • The Our World In Data site is a Jamstack project using Typescript, React and Netlify, with WordPress & MySQL as CMS
  • Our data pipeline is a custom ETL written using the Python data science stack
  • Our current data catalog lives in MySQL, our prototype one lives on S3
• All our work is open source

• Migrating our data pipeline for large institutional data into our new compute graph
• Parallelising aspects of our compute graph such as dependency checks, compute steps, or post-run publishing
• Backfilling our new data catalog from MySQL
• Identifying and setting up a notebook environment for the team to share analyses
• Harmonising the metadata kept in our data catalog to the needs of data visualisation

We’re interested in team members from diverse backgrounds, and strive to use fair criteria in hiring. Our team hours are also flexible enough to ensure that those of us with children can manage pick-up, drop-off, sicknesses, and the regular responsibilities that come with everyday life. Come join us!

Email us at jobs@ourworldindata.org with:

• Your CV, resume, or LinkedIn profile.
• A cover letter describing why working at Our World in Data is appealing to you and how you can contribute

Likewise, feel free to send us an email if you have any questions about this role.