Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Target Resume Now
Tailor your resume to this Data Scientist role.
Rezi rewrites your resume against Commonwealth Bank's job description. Free.
Tailor your resume to this Data Scientist role.
Rezi rewrites your resume against Commonwealth Bank's job description. Free.
Don't guess if your resume is good enough.
See how it scores against the Data Scientist posting at Commonwealth Bank — free, in seconds.
Don't guess if your resume is good enough.
See how it scores against the Data Scientist posting at Commonwealth Bank — free, in seconds.
About the Role
Independently test and challenge machine learning, Generative AI, and agentic AI use cases. Build expertise in emerging AI architectures, evaluation methods, and responsible AI. Create reusable evaluations, test harnesses, and automation to strengthen AI validation at scale.
Responsibilities
- Independently deliver defined components of AI and ML validation, including technical analysis, testing and evaluation.
- Review data, methodologies, assumptions, and model outcomes, investigating unexpected behaviour and potential weaknesses.
- Design and execute evaluations for Generative AI and agentic systems, including architectures involving RAG, tool use, guardrails, routing, memory, and LLM-as-a-Judge approaches.
- Build reproducible test harnesses, reusable evaluations, and automation that improve the efficiency and consistency of validation.
- Navigate unfamiliar codebases, reproduce evaluation pipelines, and inspect trace data to understand model and agent behaviour.
- Provide constructive, evidence-based technical challenge and bring findings into broader validation conclusions.
- Communicate complex technical findings clearly to both technical and non-technical stakeholders.
- Share knowledge and contribute to the continuing development of AI validation practices, tooling, and standards.
Requirements
- Advanced Python skills and experience with GitHub, version control, and CI/CD.
- SQL, R, or TypeScript exposure is valuable.
- Experience with frameworks and tools such as LangChain, LangGraph, OpenAI Agents SDK, Pydantic, and evaluation or observability platforms such as Langfuse or HoneyHive.
- Practical knowledge of Generative AI and agentic architectures, including orchestrators, sub-agents, RAG pipelines, tool use, guardrails, multi-agent routing, and memory.
- Hands-on experience building or evaluating AI and Generative AI solutions, including synthetic data.
- Transformer fine-tuning experience would be beneficial.
- Experience using AI coding agents such as Claude Code or Codex, including building reusable skills, agents, hooks, or plugins to automate technical workflows.
- Working knowledge of traditional machine-learning approaches such as regression, random forests, and XGBoost, together with appropriate evaluation methods.
- Confidence reading unfamiliar code, reproducing evaluation pipelines, writing test harnesses, and analysing trace data.
- Strong communication skills and the ability to explain complex analysis and technical risks clearly to different audiences.
- Curiosity, hands-on approach, sound analytical judgement, attention to detail, and confidence to investigate unexpected results.
- Ability to stay curious as Generative AI and agentic techniques continue to evolve.
Skills
- Python
- GitHub
- Version Control
- CI/CD
- SQL
- R
- TypeScript
- LangChain
- LangGraph
- OpenAI Agents SDK
- Pydantic
- Langfuse
- HoneyHive
- Generative AI
- Agentic Architectures
- RAG Pipelines
- Tool Use
- Guardrails
- Multi-agent Routing
- Memory
- Synthetic Data
- Transformer Fine-tuning
- AI Coding Agents
- Claude Code
- Codex
- Regression
- Random Forests
- XGBoost
- Evaluation Methods
- Test Harnesses
- Trace Data Analysis
- Communication Skills
About the Company
- AI & General Model Validation (AIGMV) provides independent technical challenge and review of important machine learning and AI models across the Group.
- The role involves working closely with model developers, product owners, and business stakeholders while maintaining the independence expected of a validation function.
Equal Opportunity
- If AI safety matters to you, you enjoy learning independently and you bring strength across some rather than all of these areas, we’d like to hear from you.
- If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application.
- We’re keen to support you with the next step in your career.
- We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.