Gen AI Engineer Job at Lorven Technologies, Mountain View, CA

QW1GOUJwR2Rna2pvVzhYK01FMHlLVTRGeHc9PQ==
  • Lorven Technologies
  • Mountain View, CA

Job Description

Gen AI Engineer


Position: Contract


Location: Mountain View (hybrid) - Remote also fine.


Duration: 12+ months

Job description:


Core Responsibilities


1. Evaluation & Metrics (Primary Focus)
• Define and measure:


o Accuracy (primary metric)


o Precision, recall, confusion metrics


o Hallucination rate


o Faithfulness / grounding
• Build ground truth datasets
• Evaluate retrieval:


o Precision@k


o Recall@k
• Perform:


o Prompt A/B testing


o Bias reduction


2. Evaluation Tooling
• RAG / RAGAS
• Databricks / MLflow
• Arize, TruLens (production monitoring)
• DeepEval (CI/CD integration)
• LangSmith (enterprise)
• LLM-as-judge frameworks


3. GenAI Workflow Orchestration
• Control flow and logging
• Memory management
• Observability pipelines
• Document comparison workflows


4. Agentic AI Evaluation
• Measure:


o Tool correctness


o Agent success rate
• Use:


o LLM-as-judge for agent evaluation


5. Observability
• Monitor:


o Latency


o Token/cost usage


o Response quality


o Hallucination trends


o Retrieval accuracy


6. System Thinking
• Structured problem-solving
• Metric-driven decision making
• Clear articulation of tradeoffs:


o Precision vs recall


o Cost vs performance


Engineering Requirements
• Strong Python (data + evaluation focus)
• Strong analytical thinking (more important than complex coding)


Nice to Have
• Cost optimization awareness
• Platform tradeoff understanding


Key Hiring Insight


Most candidates fail because they:
• Lack depth in evaluation frameworks
• Cannot clearly explain metrics or evaluation methodology

Job Tags

Contract work, Remote work

Similar Jobs

Cardinal Health

Pharmacist, Part Time Evening Job at Cardinal Health

 ...possibility of moving to every other weekend rotation. Work Environment: This is a work from home opportunity that will require reliable access to a...  ...and CPOE order evaluation and verification Ensure timely acquisition and renewal of team assigned pharmacy staff... 

PuebloDirect.com

Product Photographer Job at PuebloDirect.com

This is a part-time, on-site role based in Gilbert, AZ, for a Product Photographer. The Product Photographer will be responsible for capturing high-quality images of products, organizing and maintaining photo shoots, editing photographs, and ensuring all images meet brand... 

Aramark

Events Coordinator - Adventures on the Gorge- Seasonal Job at Aramark

 ...on the Gorge is the perfect place for adventurers and nature lovers alike. The Event Coordinator leads the planning, coordination, and execution of weddings, social events, and group functions at AOTG. This role partners closely with clients and internal teams (including... 

Asialocalize

Collaboration Opportunity - Linguists & Language Specialists Job at Asialocalize

 ...ongoing and future projects across European, Asian, and Arabic language combinations. Languages We Are Recruiting For: English Mongolian / Chinese (Taiwan) / Vietnamese / Portuguese (BR & PT) Italian Chinese / Japanese / Indonesian / Korean Spanish... 

Remote Jobs

Remote EdTech Sales Playbook Architect Job at Remote Jobs

 ...An educational technology organization is seeking an experienced EdTech Sales Consultant for a remote role. The consultant will design a comprehensive sales playbook to enhance their operations in the education sector. This role involves collaborating with leadership...