29058 - QA Analyst - Data Labelling / Category Analyst X2 POST NUMBER: 416437

1. Review of Annotated Data: Review of annotated ground truth data which includes evaluating the accuracy of labels generated through Azure ChatGPT and Google Gemini LLMs. The accuracy of generated labels will be evaluated against provided definitions for each label.
2. Manual Review of Model Outputs for product characteristics: Conduct thorough manual reviews using Excel to analyze ML model outputs using the provided documentation that outlines the criteria for model performance as well as detailed definitions for each product characteristic.
3. Test Case Design and Implementation: Design and implement comprehensive test cases and reports. This includes reviewing user stories and requirements documents to ensure testability and communicating any gaps efficiently.
4. Documentation and Communication: Document edge cases, decisions, and test results meticulously. Communicate quantitative and qualitative findings from test results to the development team, highlighting defects, errors, and areas for improvement.
5. Collaboration and Problem-Solving: Work closely with the development team to understand model dataset requirements and ensure they are met. Proactively raise blockers, ask critical questions, and navigate through challenging examples with a problem-solving mindset.
6. Quality Assurance and Enhancement: Monitor efforts to resolve issues and re-test bug fixes to ensure the final product satisfies the product requirements and meets end-user expectations. Suggest enhancements to improve product efficiency and effectiveness.

• 1-3 years of experience in a QA/manual testing role, preferably with exposure to ML model development or similar technical fields.
• Excellent communication skills, both written and verbal, with the ability to convey complex information clearly and persuasively.
• Strong analytical and critical thinking skills, with a keen eye for detail and a commitment to accuracy.
• Proficiency in reviewing and annotating data, with a basic understanding of ML model development principles.
• Strong collaboration skills, with the ability to work effectively with development teams and stakeholders.
• A proactive approach to problem-solving, willing to challenge test scenarios and suggest improvements.
• A strong foundation in English language proficiency, necessary for reviewing product information against business requirements and communicating effectively within the team.

Vaco values a diverse workplace and strongly encourages women, people of color, LGBTQ+ individuals, people with disabilities, members of ethnic minorities, foreign-born residents, and veterans to apply.

EEO Notice

Vaco is an Equal Opportunity Employer and does not discriminate against any employee or applicant for employment because of race (including but not limited to traits historically associated with race such as hair texture and hair style), color, sex (includes pregnancy or related conditions), religion or creed, national origin, citizenship, age, disability, status as a veteran, union membership, ethnicity, gender, gender identity, gender expression, sexual orientation, marital status, political affiliation, or any other protected characteristics as required by federal, state or local law.

Vaco LLC and its parents, affiliates, and subsidiaries are committed to the full inclusion of all qualified individuals. As part of this commitment, Vaco LLC and its parents, affiliates, and subsidiaries will ensure that persons with disabilities are provided reasonable accommodations. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please contact HR@vaco.com .

Vaco also wants all applicants to know their rights that workplace discrimination is illegal.

By submitting to this position, you agree that you will be giving Vaco the exclusive right to present your as a candidate for the foregoing employment opportunity. You further agree that you have represented information about yourself accurately and have not affirmatively misrepresented your qualifications. You also agree to maintain as confidential, to the fullest extent permitted by law, any information you learn from Vaco about the position and you will limit disclosure of information about the position only to the extent necessary to perform any obligations in furtherance of your application. In exchange, Vaco agrees to exercise reasonable efforts to represent you through all solicitation, job screening and resume dispersal.

Privacy Notice

Vaco LLC and its parents, affiliates, and subsidiaries (“we,” “our,” or “Vaco”) respects your privacy and are committed to providing transparent notice of our policies.

• California residents may access Vaco’s HR Notice at Collection for California Applicants and Employees here.
• Virginia residents may access our state specific policies here.
• Residents of all other states may access our policies here.
• Canadian residents may access our policies in English here and in French here.
• Residents of countries governed by GDPR may access our policies here.

Pay Transparency Notice

Determining compensation for this role (and others) at Vaco depends upon a wide array of factors including but not limited to:

• the individual’s skill sets, experience and training;
• licensure and certification requirements;
• office location and other geographic considerations;
• other business and organizational needs.

With that said, as required by local law, Vaco believes that the following salary range referenced above reasonably estimates the base compensation for an individual hired into this position in geographies that require salary range disclosure. The individual may also be eligible for discretionary bonuses.