Senior Software Engineer - Archiving and Data Services - Internet Archive

Interested in a mission-driven job ensuring open access to information for a global audience? Enjoy building technologies and products critical to thousands of libraries, non-profits, cultural heritage, and educational organizations worldwide? Internet Archive is seeking a Senior Software Engineer for its Archiving & Data Services group. Internet Archive is a non-profit digital library, top 200 website at archive.org, and an archive of over 100 petabytes of digital information running in many self-owned and operated data centers. Internet Archive also provides mission-aligned services to thousands of organizations worldwide, working collaboratively to advance our goal of “Universal Access to All Knowledge.”

We are seeking a Senior Software Engineer to work in our 12 person engineering team and help grow the Archiving & Data Services group’s suite of digital archiving, data, and access services for a global set of partner research, memory, and social good organizations. All engineering roles will work closely with product, program, and operations staff to engage in the full software development lifecycle. Preliminary duties of the role will primarily focus on developing Archive-It, our web archiving service currently used by over 1000 partner organizations to archive and provide public access to web collections totalling hundreds of terabytes of data and billions of files each year. The role may also assist with specific development projects for other department services and systems. We are also seeking someone that can work directly with the department Head of Engineering to assist with cross-department technical management, cross-team engineering projects, and coordinating the work of engineers across our services. Reporting to the department Head of Engineering, the Senior Engineer will help build and improve tools, technologies, and systems that support values-aligned products, at petabyte scale, for a global coalition of institutions providing open access to digital information.

Key Responsibilities:

• Participate in, and sometimes lead, the full software development lifecycle: plan, design, implement, test, deploy, and maintain software solutions

• Collaborate with product, support, and engineering team members to translate requirements and features into technical designs and software solutions

• Develop deep expertise in a range of the technologies in our stack

• Evolve system architectures to support future requirements and reduce operational burden

• Assist with production operation issues

• Help the Head of Engineering with project coordination and strategic and operational plans and implementation

• Help foster a culture of collaboration and learning, reinforce team processes that are working, and identify potential technology or workflow improvements

Qualifications and Skills:

• At least 3-5 years of experience as a software engineer

• Extensive experience using Python to build web and data services is preferred

• Some experience in a project management, management, or other leadership role

• Expertise, or an interest to become an expert, in one or more of the following:

  • Postgres

  • Elasticsearch

  • Cassandra, Scylla, or other distributed KV databases

  • Temporal (temporal.io)

  • Python concurrent programming

• Experience or proficiency in any of the following are a plus:

  • Web crawling or Django experience

  • Broad knowledge of web technologies and protocols

  • Experience building solutions without managed services. We own and operate our own data centers running Linux virtual machines and self-administer the full stack.

  • Rust experience, and an interest in combining Rust with Python

Job Details:

This is a remote-first position working in a distributed team. Candidates will need to have some time overlap with primarily North America (and largely Pacific Time) based colleagues. Compensation and title will be commensurate with experience and the role is open to candidates of varying seniority with a general, but negotiable, salary range of $130,000 to $150,000. References must be made available upon request.

Benefits & Perks:

The Internet Archive is a remote first workplace and provides a comprehensive benefits package including; PTO, paid holidays, and medical benefits. Depending on where you live, we also provide these additional benefits; dental, vision, health savings accounts, flex spending accounts, commuter benefits, short term disability, long term disability and retirement programs.

At the Internet Archive, we believe we do our best work when our employees bring together diverse ideas. Members of all groups under represented in the tech industry and library world are strongly encouraged to apply. We are proud to be an equal opportunity workplace and are committed to equal employment opportunity regardless of race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, genetic information, veteran status, gender identity or expression, sexual orientation, or any other characteristic protected by applicable federal, state or local law. Internet Archive is an Equal Opportunity Employer. Internet Archive complies with the Fair Chance Ordinance. Internet Archive is a 501(c)(3) non-profit library founded in 1996.