Lead Annotator - US Tech Solutions, Inc.

Job Description:

This is an On-site Role

Education: Bachelors Experience: 3+ years of overall experience, 1+ years of experience as a team lead

** Ideal Experience: • Financial services experience ideal (not required) • Experience with annotator tools and machine learning ideal (not required) Required Skills: • 3+ years of proven experience in data annotation and LLMs • 1+ year of proven team management/leadership abilities • Familiarity with annotation practices, tools and

Position Description: The mission of the Research team is to make our Client research and its team of analysts available to institutional clients through a Research Portal web site and chat tools including AI-driven chat. To make this vast amount of information more accessible, we have built a variety of tools including a search engine, a machine-learning-driven search intent classifier, an LLM-driven question-answering system, and an ontology/taxonomy, all supported by reference data about companies, industries, and our analysts. Global Research is building a team of SMEs and LLM/NLP experts to support this. The Lead Annotator will be a member of this team. We are looking for a candidate with experience as an annotator for AI and NLP-products, including the annotation of complex datasets to train and refine AI models and to support the ongoing operations and maintenance of such models. The Lead Annotator will work a diverse set of products and projects focused on multiple different uses of large language models – Q&A, summarization, red lining etc. As the Lead Annotator, you will be responsible for developing and overseeing the annotation framework, ensuring high quality labeling and annotations. You will also help design, customize and optimize annotations tools to support the workflows. You will also develop and manage necessary guidance and feedback processes and programs for junior annotators. You will also participate in hands-on annotation work together with the team.

You will be working together with our ontologists and data scientists in understanding our knowledge graphs and NLP/LLM-products and support them with the necessary annotation work and data from annotations. You will help develop the necessary reporting and summarization of the annotation projects for model development and maintenance purposes and will also create the necessary monitoring and reporting to inform management on the progress of the annotation projects and the quality of the AI models. You have a deep understanding of language models and will stay updated on the developments in the AI and language modeling space, in particular as they pertain to the financial services industries.

Responsibilities:

• Develop and refine annotation framework and guidelines. • Oversee the annotation of datasets, ensuring accuracy and consistency. • Implement quality measures and metrics to maintain high standards of data labeling and annotations. • Participate in hands-on annotation work. • Design and customize tools to enhance efficiency and ensure quality annotations. • Train and mentor junior annotators. • Collaborate with team to align annotation strategies with project goals and product missions. • Create and maintain consistent and actionable reporting of annotations. • Monitor and report on the progress of annotation projects.

Experience:

• Proven experience in data annotation and LLMs (at least 3 years) • Proven team management abilities (at least 1 year)

Skills:

• Proven experience in data annotation and LLMs (at least 3 years) • Proven team management abilities (at least 1 year) • Familiarity with annotation practices, tools and • Detail-oriented with a critical eye for accuracy • Excellent communication skills • Ability to work in a fast-paced environment Helpful/Desirable Skills: • Basic understanding of financial research or some experience in the financial services industry is highly preferred, but not mandatory.

Education:

· Bachelor’s degree Required