AI Agent Performance Engineer
- Build and maintain observability and monitoring systems that track agent latency, accuracy, call outcomes, call-flow behavior, and failure modes in production.
- Develop rigorous QA and regression testing frameworks for Voice AI agents, including automated evaluation pipelines and scripted test scenarios.
- Define, instrument, and own key performance metrics and ensure agents consistently meet or exceed them.
- Analyze large volumes of agent interactions to identify trends, misclassifications, drift, prompt issues, or reliability gaps.
- Partner closely with Product, AI/ML, and Deployment teams to convert findings into concrete product, prompt, and architecture improvements.
- Build internal dashboards, alerts, and reports that make agent performance transparent for internal teams and customers.
- Conduct root-cause investigations for production issues and drive sustainable, long-term fixes.
- Document QA processes, performance baselines, and best practices to support fast scaling across specialties.
- 3–5+ years of experience in software engineering and Applied AI.
- Hands-on experience building, deploying, and maintaining LLM-based agentic applications in production is essential.
- Strong working knowledge of agentic AI frameworks
- Experience building observability, monitoring, or QA systems using logs, metrics, dashboards, or automated evaluation pipelines.
- Proficiency with Python, analytics tooling, and comfort working with large-scale data logs.
- Experience in fast-paced, zero-to-one environments and comfort operating with autonomy.
- Drive real impact. Your work will help clinics run more smoothly and ensure patients get the care they need—faster and with fewer headaches.
- Shape something meaningful. From early product decisions to UI details, you'll play a big role in crafting both the code and the overall user experience.
- High ownership. Join a team where you’re trusted to lead, build, think critically, and bring ideas to life.
- Work with purpose. We’re not here to throw more tech at the wall, we're solving real problems in healthcare with tools that people rely on every day.
- Find a way: We own outcomes—no excuses, no shortcuts. When obstacles appear, we remove them. When the path isn’t clear, we forge one. We take extreme ownership, stay scrappy, and do what it takes to get the job done. Whatever the challenge, we find a way.
- Dream big: We refuse to accept that healthcare has to be broken. We dream boldly and act deliberately to build a future where healthcare organizations are free to focus on what truly matters: caring for people. We challenge what’s possible, rethinking every mundane barrier standing in the way of better care. Our vision is ambitious—some might call it impossible—but we believe patients and providers deserve nothing less.
- Obsess over the customer: We put our customers at the center of everything we do. We anticipate their needs, solve their problems, and design experiences that make them say, “Wow, I love this.” Every decision starts with empathy: How will this help them? How will they use it? We make our customers the hero—while never compromising on ethics, privacy, or trust.
- Always be learning: We’re relentless about growth—our own and each other’s. We challenge each other to level up, not just in skill, but in mindset, awareness, and character. We attract and empower “A” players, raise the bar, and multiply the impact of those around us. We’re learning machines: always reading, asking, improving. Because when we grow as people, we become better teammates, leaders, and humans - that lifts the whole team with us.
- Move Fast. Deliver Excellence. We prioritize rapid execution without sacrificing quality. Speed unlocks momentum—but only when paired with precision. We test relentlessly, ship often, and iterate fast to drive the results we want. Whether it’s responding to a message or launching a product, we take bold action, measure impact, and refine quickly. Velocity is our competitive edge—excellence is our standard.
Recommended Jobs
Estate Planning Paralegal
Estate Planning Paralegal – This is a Remote Opportunity We are a forward-thinking, rapidly growing company specializing in estate planning and trust creation. We empower our clients by guiding th…
J.P. Morgan Wealth Management - Private Client Investment Associate - Tempe, AZ
Job Description If you are passionate about client experience and goals driven planning, then our team is the right place for you. As part of Consumer Banking, you will provide client service ope…
Costco Free Sample Representative
We want you to help us shape the future of shopping experiences and deliver on our purpose of connecting people with the products and experiences that enrich their lives. Joining Advantage Solutions …
Medical Records Associate
Description The Medical Records Associate will be responsible for maintaining and managing patient medical records in compliance with regulatory requirements and practice policies. The ideal candi…
Land Surveyor, PLS
Webuild ( ) is actively seeking an experienced Professional Land Surveyor (PLS), Project Manager to take a leadership role in directing oil and gas, power, and civil / utility development - land surv…
A-10/9 - 8377 - Software QA Automation Lead - - Contract to Hire - Must be Local to AZ Only
*** Contract to Hire - This position is not eligible for visa sponsorship, either now or in the future. *** ** Hybrid work model, mostly remote; on-site in Phoenix, AZ for design/whiteboard sessio…
Merchandiser (Full Time)
The Full-Time Merchandiser (FTM) is ensuring proper level of Red Bull products is stock, maintained, displayed and rotated correctly in large and small format stores. The primary role of the Merchan…
Ironworker
Journeyman Ironworker – Job Description A Journeyman Ironworker performs skilled ironwork in accordance with project plans, safety standards, and structural requirements. Responsibilities include …
Cyber Systems Engineer - Level 4
RELOCATION ASSISTANCE: No relocation assistance available CLEARANCE TYPE: None TRAVEL: No Description At Northrop Grumman, our employees have incredible opportunities to work on revoluti…