Founding AI Engineer, Agents & Evaluation

// What you'll own

• –Evaluation and simulation infrastructure: CI/CD for agent behavior. Run Casey against historical scenarios, measure fill-rate outcomes, catch regressions, A/B test prompt and orchestration changes before they hit production. This is the piece that lets us ship changes to a sophisticated multi-agent system with confidence instead of vibes.
• –Retrieval and grounding architecture: our agents operate over dense, agency-specific knowledge (SOPs, escalation policies, scheduling rules, compliance constraints). Design the retrieval layer that gets the right context to the right agent at the right turn, so the agent harness keeps scaling as the knowledge surface grows.
• –Alignment and outcome measurement: connect every ranking decision, outreach attempt, and agent choice to downstream business outcomes (fills, retention, utilization). The event data exists across multiple systems. Nobody has unified it. You will, and everything else compounds from it.
• –The agent intelligence roadmap: we have 100K+ scheduling decisions and growing with paired outcome data. You'll decide when fine-tuning, reward modeling, or new model architectures are the right next investment for the harness, and ship them.

// Who you are

• –An applied scientist who ships production code: your mental model of system behavior is probabilistic, but your output is working systems.
• –Fluent in the full stack of interventions (prompting, retrieval, evals, fine-tuning, RL) and have the judgment to know which to reach for.
• –Someone whose first instinct when something breaks is to design an experiment, not just read a stack trace.
• –Comfortable with the reactive trap dying. You stop treating symptoms and start building the systems that prevent whole classes of problem.
• –Indifferent to healthcare domain experience. The domain is learnable in weeks.

// How we build

// The process

• –Intro call (45 min, remote): Daniel walks you through the role and the reactive-trap diagnosis. You walk us through a reliability or evaluation system you've built for a production ML/LLM product.
• –Technical exercise (async, own time): we hand you a real slice of our data (ranking snapshots, outreach events, outcomes). You design and prototype an offline evaluation framework. Record a Loom walking through your thinking.
• –On-site day (half day, in-office): review + live extension of your exercise, architecture conversation, close with the founders.
• –Offer.

// What we offer

• –Competitive base and meaningful equity. If this works, it should work for you.
• –Equipment stipend: whatever setup makes you fast.
• –Luxury gym membership in NYC. We work in health care. Yours counts too.
• –Daily lunch, and dinner when work runs past dark. Big fan of late-night jams!
• –Time off as needed. We measure output, not hours at a desk.
• –Happy hours, poker nights, and builder events in our Soho office.
• –Snacks in office, including team favorites like Gruns gummies, sardines, protein bars, and midday froyo runs.

// Apply