Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Tailor your resume to this Lead Software Engineer - Agent Safety role.
Rezi rewrites your resume against Kraken's job description. Free.
Tailor your resume to this Lead Software Engineer - Agent Safety role.
Rezi rewrites your resume against Kraken's job description. Free.
Don't guess if your resume is good enough.
See how it scores against the Lead Software Engineer - Agent Safety posting at Kraken — free, in seconds.
Don't guess if your resume is good enough.
See how it scores against the Lead Software Engineer - Agent Safety posting at Kraken — free, in seconds.
About the Role
We are seeking a Lead Software Engineer to head our newly formed Agent Safety/Evals team. This role is critical to ensuring AI agents operate securely and predictably as they take on more autonomous tasks across Kraken's internal workflows. You will define the technical strategy for evaluating models, enforcing safety guardrails, and governing AI behavior across internal platforms, skills, and harnesses, working closely with the broader AI Foundations team and engineers across Kraken.
Responsibilities
- Lead the technical direction for internal agent safety, designing and implementing systems for reliability, security, and determinism of LLMs and autonomous agents.
- Build scalable harnesses and evaluation frameworks tailored to internal workflows and skills, engineering systems to measure quality and reproducibility.
- Create and enforce pre- and post-generation guardrails, managing the governance of AI models within internal tools and platforms.
- Build out dedicated auth/permissions for internal AI agents, establish monitoring/observability pipelines, and define incident response protocols for AI-specific anomalies.
- Lead continuous verification efforts and red teaming exercises to proactively identify vulnerabilities, prompt injections, or unpredictable behaviors.
- Deploy, run, and support high-throughput, low-latency safety services for internal use cases on AWS, making architecture/cost tradeoffs and partnering with stakeholders.
Requirements
- Proven experience leading technical initiatives or teams, capable of setting technical vision for complex, ambiguous domains.
- Senior/advanced capability in designing secure components end-to-end, thorough testing, and reasoning about system design, concurrency, and architecture tradeoffs.
- A critical thinker comfortable interrogating AI output quality and experienced in building harnesses to measure it reproducibly.
- Experience with threat modeling, authentication, red teaming, or building guardrails for internal systems, tooling, or platforms.
- Comfortable running internal services, owning reliability/scalability, and collaborating with platform/techops/security partners on AWS.
- Able to synthesize complex safety and eval metrics into actionable insights for technical and non-technical stakeholders.
- Python preferred.
- Experience with specific LLM evaluation and safety frameworks (e.g., Inspect AI, Ragas, OpenAI Evals, NeMo Guardrails) is a plus.
- Django experience and strong backend engineering patterns (security, performance, maintainability) is a plus.
- Experience with Datadog for complex observability, tracing, and monitoring in AI environments is a plus.
- Familiarity with foundational AI engineering tooling like Pydantic AI, LiteLLM, or LangChain is a plus.
- Must have the right to work in the country of application; visa sponsorship is not provided.
Skills
- AI Evaluation
- AI Safety
- LLM Behavior
- System Design
- Concurrency
- Architecture Tradeoffs
- Threat Modeling
- Authentication
- Red Teaming
- Guardrails
- AWS
- Reliability
- Scalability
- Observability
- Monitoring
- Incident Response
- Python
- Django
- Datadog
- Pydantic AI
- LiteLLM
- LangChain
Location
- AWS
Work Type
- Full-time
Experience Level
- Lead
- Senior
- Advanced
About the Company
- Kraken powers innovative global developments in energy, creating technology that redefines utilities and unlocks a new energy system.
- We optimize renewable generation, build intelligent grids, and empower utilities to deliver exceptional customer experiences.
- Our mission is to improve the lives of one billion humans within the decade and shape a cleaner, better future.
- AI is a key investment area for Kraken Technologies, expanding foundational infrastructure to enable effective AI use across the organization.
Equal Opportunity
- As an equal opportunity employer, we do not discriminate on the basis of any protected attribute. We consider all applicants without regard to race, colour, religion, national origin, age, sex, gender identity or expression, sexual orientation, marital or veteran status, disability, or any other legally protected status.
- We encourage individuals to apply even if they don't meet 100% of the job requirements.
- We are creating a team of genuine, honest, empathetic people whose unique skills and perspectives drive our success.