Software Engineer - Real Time Data Platform (Senior / Principal) - Blackbird.AI

Blackbird.AI helps organizations discover emergent threats and stay one step ahead of real-world harm through our AI-powered Narrative and Risk Intelligence Platform. Our commitment is to prioritize safety and security, providing the tools to identify potential risks and ensure a safer environment proactively. No matter the job or where it’s located, we’re all connected by a shared vision: To lead and enhance the landscape of risk intelligence. 

As a Senior/Principal Software Engineer reporting to the Director of Engineering, Data Platform, you will not only optimize data pipelines for our advanced real-time streaming cloud-hosted analytics platform but also lead the development of new features and capabilities. Your role will be pivotal in shaping the future of our platform, offering the chance to solve some of the most challenging data problems in the industry.

As a Software Engineer, you’ll have the chance to:

• Design and implement real time distributed data processing systems analyzing public data and detecting emergent threats
• Lead and manage the development and optimization of ETL processes for various data formats from social media, news, and web sources.
• Design and implement robust database systems and develop tools for query and analytic processing, focusing on real-time streaming applications.
• Conduct performance analysis and empirical studies, making strategic decisions on tradeoffs (e.g., cost vs. throughput/latency).
• Develop, manage, and oversee the database architecture for our real-time streaming cloud-hosted analytics platform.
• Spearhead build automation, continuous integration, deployment, and performance optimization efforts, upholding our strict security requirements.
• Drive the project management, leading the development of new features and capabilities.
• Design test suites and implement inline instrumentation to ensure data correctness.

Requirements

• Bachelor's degree in Computer Science or a related field.
• Minimum of 5 years of experience in data engineering and building data platforms.
• Demonstrated success in deploying cloud and SaaS model products, with expertise in building optimized processing pipelines for streaming analytics applications.
• Proficiency in databases and query optimization (PostgresSQL, ElasticSearch, MongoDB, Redis, Druid), including experience with NoSQL and graph databases.
• Experience in horizontally scaling databases.
• Expertise in Kafka and Airflow, with a strong understanding of runtime profiling tools.
• Experience in big data processing systems similar to Apache Spark, Flink, Beam or similar.
• Strong skills in build automation, continuous integration, and deployment (CI/CD) tools (Webpack, Buddy, Jenkins, Docker).
• Expert-level Python coding skills.
• Project management skills and the ability to lead development initiatives.

Desirable Skills:

• Technical background in AI and ML.
• Experience in designing and implementing interactive query-driven main-machine intelligence systems.
• Experience in working with distributed teams.

Visa Sponsorship: No visa sponsorship currently available. Candidates should be able to work without requiring visa sponsorship now or in the future in their preferred work location.

We’ve outlined specific skills, experience, and requirements for this position, but don’t stress if you don’t meet every single one. Our Talent Team is dedicated to discovering exceptional individuals, and they might identify a relevant aspect of your background that suits this role or another opportunity within Blackbird.AI.

This updated job description highlights the seniority and leadership expectations while maintaining the core responsibilities and qualifications of the Software Engineer role. It also emphasizes the opportunity to be part of a team focused on solving complex and significant data challenges.

If you have passion for the role, please still apply.

Benefits

Why you’ll love working here:

• Competitive compensation package, 401(k), and equity - everyone has a stake in our growth!
• Comprehensive health benefits for you and your loved ones, including wellness days and monthly wellness reimbursements - an apple a day doesn’t always keep the doctor away!
• Generous vacation policy, encouraging you to take the time you need - we trust you to strike the right work/life balance!
• A flexible work environment with opportunities to collaborate with your team in person - you can have it all!
• Inclusion and Impact - soar to new heights!
• Bi-annual offsites - have fun with your colleagues!
• Professional development stipend - never stop learning!

Pay Transparency:

For individuals assigned and/or hired to work in New York, Blackbird.AI is required by law to include a reasonable estimate of the compensation range for this role. This compensation range is specific to New York. It takes into account the wide range of factors that are considered in making compensation decisions, including, but not limited to, skill sets, experience and training, licensure and certifications, and other business and organizational needs. At Blackbird.AI, it is not typical for an individual to be hired at or near the top of the range for their role, and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current compensation range for this position is expected to be $150,000-$200,000. This range may vary for positions outside of New York and as it has not been adjusted for the applicable geographic differential associated with the location where the position may be filled. 

Regardless of location, candidates can expect during the first few conversations with Blackbird.AI’s Talent Team and Hiring Managers to share any approved budget.

Apply Today

Equal Opportunity Employer