LLM Red Team Specialist - AI Job at Mercor, New York, NY

  • Mercor
  • New York, NY

Job Description

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: LLM Red Team Specialist — Failure Modes & Edge Cases
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Evaluate frontier AI models on coding, ML , and analysis tasks to identify vulnerabilities and failure modes.
  • Design complex tasks that challenge models and are fair for grading.
  • Document findings with clear evidence and reproducible steps.
  • Collaborate with task authors to close loopholes and improve grading.
  • Share insights with researchers to enhance benchmark quality.
  • Work independently and asynchronously to meet deadlines and improve AI model performance .

Qualifications

Must-Have

  • MSc or PhD in a STEM field or equivalent experience.
  • 1+ years in research, research-engineering, security, or AI evaluation.
  • Experience identifying vulnerabilities in LLMs or ML systems .
  • Proficiency in Python and Git .
  • Familiarity with LLM capabilities and evaluation techniques.
  • Ability to work 35 hours/week .

Preferred

  • Experience in AI training , model evaluation, or benchmark/task authoring.

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Job Tags

Contract work, Summer work, Remote work

Similar Jobs

Virtual Vocations Inc

Senior Actuary Job at Virtual Vocations Inc

 ...Performing and supporting Property & Casualty reserve analyses, the full-time Senior Actuary will ensure the accuracy and integrity of loss reserves while collaborating with cross-functional teams in a remote work environment.Key responsibilities:Conduct quarterly Loss... 

Premier at The Springs

Treatment/Wound Care Nurse RN or LPN Job at Premier at The Springs

 ...Description Premier at The Springs is actively hiring for an LPN or RN treatment nurse! Full time employees in this position...  ...Staff regarding pressure ulcer risk issues, interventions and wound care needs. 7. Prepares and administers treatments as per... 

El Pollo Loco

Executive Assistant Job at El Pollo Loco

 ...our Support Center, we believe our best work is done Together, when departments collaborate...  ...is an essential function of the role. Remote work is complementary and does not...  ...significance to the company.Ability to work weekends and extended work schedules as needed.Our... 

Bass Pro Shops

Hunting Team Lead Job at Bass Pro Shops

 ...by the Corporate Office. Insures a pleasant and productive shopping experience for all customers. Assists the GSM / DM and Human...  ...Savings ~ Paid holidays ~ Paid vacation ~ Paid sick time ~ Bass Pro Cares Fund ~ And more! Bass Pro Shops is an equal... 

Just Food Inc

Bartendar Job at Just Food Inc

 ... Prepare alcohol or non-alcohol beverages for bar and restaurant patrons Interact with customers, take orders and serve...  ...team. Here at RASA, San Francisco Chronicle's acclaimed "Best Indian Restaurant in the Bay Area," we pride ourselves in excellent...