Role Title: Data Science \& Analytic Methodology SME
Role Type: Vertrag
Standort: Fernbedienung
Required Skills:
LLMs
Pipeline integration
Data analytics
Scaled-analysis judgment
Data science
In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required - your domain knowledge is what matters.
Scope of Work:
-
Map scaled-analysis and pipeline-integration scenarios to an established taxonomy across key evaluation areas.
-
Author detailed analytic cases that reflect authentic deployment patterns, including automated triage, enrichment, and analysis-at-scale, as well as representative failure modes in real-world settings.
-
Evaluate and grade responses based on criteria such as source handling, reasoning, understanding of limitations, and practical utility.
-
Identify and flag instances of model over-refusal that could hinder legitimate analytic workflows or disrupt pipeline continuity.
-
Document challenging edge cases, emerging methodology concerns, and provide actionable recommendations for continuous improvement.
-
Contribute clear, insightful written and verbal feedback that supports iterative, high-quality project outcomes.
Preferred Qualifications:
-
Demonstrated experience integrating LLMs---such as government GPT variants---into complex data or analytic pipelines.
-
Proven capability in conducting analysis at scale, with an understanding of automation failure modes and relevant evaluation metrics.
-
Track record of teaching, mentoring, or technical leadership in data science or related analytic disciplines.
-
Expertise in authoring clear, concise technical documentation and conveying complex concepts through strong written English.
-
Exposure to the design of evaluation frameworks or benchmark systems.
-
Familiarity with defense, intelligence, or government analytic domains is highly valued.
-
Ability to operate effectively in distributed, international project environments with minimal clearance constraints.
Ursprünglich veröffentlicht auf Himalayas
