Teams at Robots & Pencils

Lead development of a production-grade evaluation framework for planner-executor agents. Build user simulation, isolated experiment execution, grading, statistical analysis, tracing, optimization-lever plugins, and a locally hosted dashboard. Run experiments comparing models and inference strategies, quantify quality, latency, and cost tradeoffs, generate evidence-backed insights, and deliver client recommendations. Partner with engineers and business stakeholders, communicate findings, establish evaluation standards, and mentor others.

Designs and delivers cloud-native applications on AWS, including microservices, serverless functions, APIs, event-driven systems, infrastructure as code, and CI/CD pipelines. The role emphasizes scalability, resilience, observability, security, and cost optimization, while integrating AI and LLM services using RAG and inference workflows. Responsibilities include architecture documentation, technical collaboration, code and design reviews, end-to-end ownership, and mentoring junior engineers.

Leads large-scale AI, digital, and cloud transformation programs from strategy through launch and realized business value. The role coordinates concurrent Agile teams, technical direction, architecture discussions, AI lifecycle governance, risk management, KPIs, stakeholder communication, staffing, and executive decision-making. It requires translating business goals into technical plans, managing complex dependencies and trade-offs, mentoring delivery leaders, and shaping delivery practices while partnering with clients on growth opportunities.