Principal Data Scientist, Gemini Core Modeling and Evaluations at Google

Minimum qualifications:

• Bachelors degree in Machine Learning, Data Science, Information Management, Computer Science, or other Engineering discipline, or equivalent practical experience.
• 15 years of experience in data science or other quantitative fields such as: statistics, machine learning, software engineering, or ML engineering.
• Experience working on algorithmic products, production machine learning systems, and defining model evaluation metrics.
• Experience in using data insights to drive product excellence.

Preferred qualifications:

• Experience developing high-growth technology products and services.
• Ability to influence without direct authority, working on and with technical and cross-functional or highly matrixed teams who do not report into this role.
• Ability to turn ambiguous problem spaces into clear solutions, and comfortable with new technology and thinking outside the box to develop and implement short- and long-term creative solutions.
• Ability to work well cross-functionally, and manage organizational complexities and competing priorities.
• Excellent project/product management and technical architecture skills, and the ability to deliver projects on time.
• Excellent strategic thinking, data science, and analytical skills.

About the job

As a Principal Data Scientist, you will be a deeply technical and strategic data science leader for Gemini Core Modeling and Evaluations. You (along with a small team of data scientists) will be responsible for defining the model evaluation criteria, influencing the supervised fine tuning data and evaluation prompt sets, identifying valuable signals for reinforcement learning human feedback, and improving human rater efficiencies.

The US base salary range for this full-time position is $262,000-$377,000 + bonus + equity + benefits. Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target salaries for the position across all US locations. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.

Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits. 

Responsibilities

• Develop deep product and user analytics that will help to drive quality and design directions, such as identifying signals that are predictive of user engagement or analyzing Gemini use cases (e.g., coding, fact finding, or creativity and summarization). Develop insights around trends and drive model improvements that help achieve goals.
• Drive forward the science of measuring Gemini, ensuring that our quality, factuality, and safety model evaluation metrics are indicative of the end-user experience, and help guide launch decisions for Gemini.
• Manage experimentation and comparative evaluation, including the science, methodologies, tools, and measurement approaches to accelerate Gemini experimentation and innovation. 
• Guide the models human evaluation process, including analyzing rater performance, influencing rater templates, updating training, and evolving our approach to ensure precise and accurate evaluations.