Principal Software Engineer, AI Quality and Testing
Where
In short AI
Principal Software Engineer, AI Quality and Testing, in the AI Delivery & Engineering team. Location: Lisbon, Portugal / Krakow, Poland. Working pattern: fully remote (Portugal); hybrid (Poland). Permanent local contract.
Description
a long ad (3 302 characters): the AI summarised it; the full text is in the next tab.
- Role
- Provide principal-level technical leadership for AI quality, testing and evaluation of AI assets and solutions. Define risk-based quality and evaluation standards; review supplier test plans, code, methods and results; guide decisions on test strategy, coverage, repeatability and residual limitations; improve representative evaluations and evidence traceability; set standards for regression, automation, drift and adversarial testing.
- Requirements
- Degree in Computer Science, IT, Software Engineering or a related discipline
- Enterprise software engineering experience in cloud-native and distributed systems
- Hands-on enterprise AI evaluation and integrated-solution testing experience
- Knowledge of deterministic testing and probabilistic AI or agent assessment
- Experience with risk-based strategy, representative datasets and reproducible evaluations
- Ability to assess groundedness, factual correctness, task completion and tool actions
- Experience with CI/CD quality gates, evaluation harnesses and regression evidence
- Knowledge of dataset provenance, baselines, drift and change-impact assessment
- Proven technical leadership through standards, independent review and supplier assurance
- Strong communication, cross-team influence, mentoring and documentation skills
- Preferred
- advanced study in AI/ML, distributed systems or cloud/platform engineering
- Conditions
- Contract: permanent local
- Salary: competitive; for Poland, no less than 327660 PLN gross per year
- Hours: not stated
- Location: Lisbon, Portugal (remote) / Krakow, Poland (hybrid)
- Benefits
- Life and health insurance
- Employee pension plan
- 26 holiday days
- National and international growth opportunities
- Tabaqueira employee benefits
Location: Lisbon, Portugal / Krakow, Poland
Working Pattern: Fully Remote (Portugal); Hybrid (Poland)
Join our AI Delivery & Engineering team and provide principal-level technical leadership for AI quality, testing and evaluation. You'll help ensure reusable AI assets and enterprise solutions are assessed through credible, risk-based and reproducible evidence.
Impact You'll Own
• Define quality and evaluation standards for AI assets and end-to-end solutions.
• Strengthen supplier quality through independent reviews of test plans, code, methods and results.
• Guide decisions on test strategy, coverage, repeatability, cost and residual limitations.
• Improve representative evaluations, evidence traceability and use-case-specific assurance.
• Advance regression, automation, drift and adversarial testing standards.
What You'll Do
• Define risk-based standards for software testing and probabilistic AI behavior.
• Review supplier test plans, designs, test artifacts, evaluation methods and results.
• Assess evidence at component, integration and end-to-end business use-case levels.
• Guide partners on evaluation trade-offs, asset reuse and justified exceptions.
• Review dataset fitness, provenance, baselines, groundedness and reproducibility.
• Set standards for CI/CD quality gates, regression, drift and failure scenarios.
• Document readiness recommendations, residual limitations and required remediation.
Skills and Capabilities
Must-haves
• Degree in Computer Science, IT, Software Engineering or a related discipline.
• Enterprise software engineering experience in cloud-native and distributed systems.
• Hands-on enterprise AI evaluation and integrated-solution testing experience.
• Knowledge of deterministic testing and probabilistic AI or agent assessment.
• Experience with risk-based strategy, representative datasets and reproducible evaluations.
• Ability to assess groundedness, factual correctness, task completion and tool actions.
• Experience with CI/CD quality gates, evaluation harnesses and regression evidence.
• Knowledge of dataset provenance, baselines, drift and change-impact assessment.
• Proven technical leadership through standards, independent review and supplier assurance.
• Strong communication, cross-team influence, mentoring and documentation skills.
Nice-to-have
• Advanced study in AI/ML, distributed systems or cloud/platform engineering.
Rewards and Benefits
Permanent local contract with a competitive salary plus Tabaqueira employee benefits including:
• Life and Health insurance
• Employee Pension Plan
• 26 holiday days
• National and international growth opportunities
• Diverse and international work environment
For Poland: In this position, you will earn no less than 327660 PLN gross per year.
Equal Opportunities Statement
We believe that diverse perspectives and experiences make us stronger. As an equal opportunity employer, we are committed to creating a workplace where every individual feels welcomed, respected, valued, and empowered to reach their full potential.
We encourage applications from people of all backgrounds and identities, and provide reasonable accommodations to ensure an accessible recruitment process. If you need support or adjustments during your application, let us know. We're here to help you succeed.
How to apply
Prepare your application
A cover letter and a short pitch for this job, written from your profile, and what to learn next.
Sign in and create your profile from your CV. It is private: we share it with no one.
Something wrong or unlawful in this ad? Report it
Similar jobs
- Principal Software Engineer, AI Data EngineeringPhilip Morris (Tabaqueira) · Lisboa
- Principal Software Engineer, AI Platform EnablementPhilip Morris (Tabaqueira) · Lisboa
- Principal Software Engineer, AI DevOpsPhilip Morris (Tabaqueira) · Lisboa
- Principal Tech Software Engineer AI - Agentic AIPhilip Morris (Tabaqueira) · Lisboa
- Principal Product Owner AI, Agent Marketplace and AI-Assisted SDLCPhilip Morris (Tabaqueira) · Lisboa
- Principal Product Owner AI, Agentic Solutions and OrchestrationPhilip Morris (Tabaqueira) · Lisboa
Salary compared ⓘ
32 jobs · Software testers and ICT quality specialists · Portugal
see the best paid →Training for this job
- AI+ Quality Assurance Practitioner™Galileuself-paced · flexible hours · 40 h · 695 € + IVAAsked for in this ad: AI / machine learning
- ISTQB® Testing with Generative AIRumosonline · daytime · 14 h · 1320 € + IVA · starts 09/12/2026Asked for in this ad: AI / machine learning
- Utilizar Ferramentas da Ia na OrganizaçãoServiço de Formação Profissional de Santarémonline · daytime · 25 h · free · starts 02/12/2026Asked for in this ad: AI / machine learning
Languages AI
Published on (1)
- Philip Morris (Tabaqueira) seen 18 Oct – 21 Nov
Where the company is see on the map →
- Sintra (Lisboa) 33 jobs
- Lisboa (Lisboa) 20 jobs
- Porto (Porto) 6 jobs
- Coimbra (Coimbra) 3 jobs
- Aveiro (Aveiro) 2 jobs
- Braga (Braga) 2 jobs
- Castelo Branco (Castelo Branco) 2 jobs
- Leiria (Leiria) 2 jobs
- Vila Real (Vila Real) 2 jobs
- Viseu (Viseu) 2 jobs
- Albufeira (Faro) 1 job
- Bragança (Bragança) 1 job
- and 7 more municipalities · all on the map
according to the company's other jobs; not necessarily where this job is.
First seen on 18 Oct, 04:44