Staff Engineer (Core & MLOps) - Remote
Em resumo IA
Staff Engineer (Core & MLOps), posição remota para a Zyte, empresa de extração de dados da web. Lidera a arquitetura das camadas de controlo e contexto da plataforma Zyte API. Requer mais de 10 anos de experiência em sistemas distribuídos. Contrato, salário e localização não indicados.
Descrição
anúncio longo (8 307 caracteres): a IA resumiu-o; o texto integral está no separador ao lado.
- Função
- Função de Staff Engineer na equipa Core & MLOps. Responsável por arquitetar e construir as camadas de controlo (registos de serviços e schemas, routing, canary releases) e de contexto (agregação de sinais operacionais e ciclos de feedback automáticos). Define bibliotecas cliente Java e Python, Helm charts e pipelines de deployment, e contratos entre serviços em gRPC e Protocol Buffers. Opera a infraestrutura (Kubernetes, Terraform, Kafka, bases de dados), lidera RFDs sobre orquestração de workflows, routing multi-cluster e failover, define SLOs e participa na rotação de on-call.
- Requisitos
- Mais de 10 anos de experiência a construir sistemas backend distribuídos e escaláveis
- Experiência comprovada a criar plataformas internas ou bibliotecas core adotadas por equas de engenharia
- Experiência em Java com frameworks reativas (Vert.x ou Netty) e forte proficiência em Python
- Experiência profunda em gRPC e Protobuf, incluindo evolução de schemas e compatibilidade retroativa
- Experiência prática com Kubernetes em produção à escala, Terraform e streaming de eventos com Kafka
- Experiência a desenhar pipelines de telemetria automáticos, materialized views ou feature stores
- Experiência a definir SLOs/SLIs, analisar blast radius e desenhar sistemas tolerantes a falhas
- Capacidade de escrita técnica para articular desenhos complexos e alinhar equipas multifuncionais
- Comunicação assíncrona forte, escrita e interpessoal, em ambiente remoto e global
- …
- Valorizado
- Temporal ou DBOS, MLOps (serving de modelos, drift detection), SPIRE, mTLS, Cilium, Istio ou Envoy, criação de CLIs ou SDKs, experiência em web scraping e crawling, contribuições open source em sistemas distribuídos
- Condições
- Contrato: não indicado
- Salário: não indicado
- Horário: não indicado
- Local: remoto
- Benefícios
- Trabalho remoto (remote-first), a partir de qualquer parte do mundo
- Horário flexível
- Conferências e encontros com membros da equipa de vários países
- Trabalho com tecnologias e ferramentas open source
Why Zyte?
At Zyte, we don’t just collect web data—we solve complex web data challenges at scale. We are a globally distributed team that is bold, curious, and dedicated to building innovative ways to deliver clean, reliable web data from across the internet.
- Data is our passion: We unlock access to open web data, empowering organizations—from startups to global enterprises—to power competitive intelligence, analytics, and AI pipelines.
- Remote-first culture: Work from anywhere in the world. With over 260 Zytans across 33 countries, our strength comes from diverse backgrounds, perspectives, and skills.
- Engineering excellence: We love diving deep into complex code bases, evaluating emerging technologies, and pushing technical boundaries. If you thrive on creative problem-solving and reliably delivering production-grade solutions, you will fit right in.
- Established system design framework: We systematically analyze functional and product requirements alongside architectural trade-offs to make informed, sustainable design decisions.
- Real-world distributed systems challenges: Tackle rare engineering problems in custom networking, high-throughput streaming, and large-scale orchestration under real-world operational constraints.
- Strong product ownership: Work on a profitable, market-leading platform with dedicated time and resources allocated for architectural research and strategic feature engineering.
Why This Role?
Zyte API powers a suite of Java and Python microservices running across multiple cloud providers and data centers. Every engineering team—spanning Core API, Browser, Edge, Antiban, AI, and Agentic products—relies on foundational infrastructure managed by our team: Kubernetes clusters, high-throughput Kafka pipelines, billing engines processing per-request metrics, and robust inter-service contracts.
We are evolving Zyte API into an automated software factory: a platform where developers and AI agents seamlessly create, validate, deploy, and maintain data extractors and workflows through governed interfaces. Two foundational layers drive this evolution. The control plane (service/schema registries, health-aware routing, automated canary releases) is actively under deployment. The context plane, which synthesizes observational signals to guide automated repair and optimizations, represents our next major design effort. As a Staff Engineer, you will own both architectures, setting the engineering standard for how services at Zyte are designed, built, and operated. You will be the technical lead in the Core & MLOps squad, collaborating closely with the Team Lead, Senior Engineers, DevOps, and QA, while directly influencing the release velocity and architecture of five neighboring squads.
Requirements
What You’ll Do
- Architect the control and context planes. Advance the control plane (service registry, schema registry, SLO enforcement, CLI tooling) into a robust production substrate. Design and build the context plane to aggregate operational signals—such as domain extraction histories, IP reputation scores, and BigQuery cost/performance telemetry—creating automated feedback loops between intent, execution, and self-healing maintenance.
- Own the service chassis and golden path. Maintain and refine multi-language client libraries (Java and Python), standardized workload specifications, Helm charts, and deployment pipelines, driving seamless adoption across all product teams.
- Define inter-service contracts. Establish gRPC and Protocol Buffer definitions, API gateway transcoding, versioning policies, and schema evolution rules to enable autonomous, safe deployments across teams.
- Operate the platform substrate. Partner with infrastructure engineers to run Kubernetes (across OCI, Hetzner, Servers.com, and GCP), Terraform, HAProxy/Nginx ingresses, Confluent Kafka, real-time event-billing pipelines, Valkey, and ongoing database modernizations (MySQL to PostgreSQL).
- Drive architectural strategy via RFDs. Lead Requests for Discussion (RFDs) on critical platform initiatives, including durable workflow orchestration (Temporal/DBOS), per-request gateway orchestration, multi-cluster routing, and automated failover.
- Institutionalize reliability engineering. Establish clear SLOs and error budgets for core capabilities, implementing health-aware traffic isolation and automated weighted canary deployments to eliminate manual release bottlenecks.
- Share production ownership. Participate in the shared infrastructure on-call rotation, lead incident post-mortems, and translate operational lessons into platform improvements.
- Elevate engineering standards. Mentor engineers across squads on distributed systems design, review system proposals, and establish intuitive best practices that make building reliable software effortless.
Who You Are
- 10+ years of experience building scalable distributed backend systems, with a proven track record of authoring internal platforms or core libraries widely adopted by engineering teams.
- Expertise in Java (using reactive frameworks like Vert.x or Netty) alongside strong proficiency in Python.
- Deep gRPC & Protobuf experience, with demonstrated success managing schema evolution and backward compatibility in mission-critical environments.
- Hands-on infrastructure expertise with production Kubernetes at scale, Terraform, and event streaming with Kafka.
- Feedback loop designer: Experience designing automated telemetry pipelines, materialized views, or feature stores that dynamically adapt system behavior based on production data.
- Reliability mindset: Proven success defining SLOs/SLIs, analyzing blast radius, and designing fault-tolerant systems built around rigorous service contracts.
- Technical leadership: Exceptional technical writing skills with the ability to articulate complex designs clearly and drive technical alignment across multi-functional teams.
- Async communication: Strong interpersonal and written communication skills tailored for a globally distributed, remote-first environment.
- Curious problem solver: Passionate about continuous learning and evaluating novel tools, architectures, and techniques.
Our Tech Stack
- Languages: Java 21, Python.
- Frameworks & Tools: Vert.x, Netty, gRPC and protobuf, Kubernetes, Helm, Terraform, CircleCI.
- Infrastructure: Multi-cloud (OCI, GCP) and data centres (Hetzner, Servers.com), blue/green deployments everywhere.
- Data & Messaging: Confluent Kafka, BigQuery, Valkey, MySQL (Postgres next), HBase, Google Pub/Sub.
- Monitoring & Observability: Prometheus, Grafana, Loki, OpenTelemetry.
Bonus Points
- Durable execution engines: Hands-on experience with Temporal, DBOS, or similar workflow platforms.
- MLOps: Experience in model serving, performance monitoring, and drift detection in production environments.
- Zero-trust networking & service meshes: Familiarity with SPIRE, mTLS, Cilium, Istio, or Envoy.
- Developer tooling: Track record of building CLIs, SDKs, or project generators adopted across an engineering organization.
- Web scraping & crawling experience: Familiarity with the technical challenges of large-scale web data extraction.
- Open source contributions: History of contributing to or maintaining open-source projects in distributed systems or data extraction.
Why Join Us?
- Impact at scale: Architect core infrastructure powering web-scale data pipelines for top-tier global enterprises.
- Autonomy and flexibility: Enjoy a genuine remote-first culture with flexible working hours and high organizational trust.
- Continuous innovation: Solve emerging, complex technical challenges as AI and web data extraction rapidly evolve.
- Global community: Collaborate with a passionate, diverse team of distributed systems engineers and data specialists across the world.
If you are a Staff Engineer excited by the prospect of building core platform substrates that empower multiple engineering teams, we would love to hear from you. Join us in shaping the control plane behind web data extraction at Zyte.
Benefits
By joining the Zyte team, you will:
- Become part of a self-motivated, progressive, multi-cultural team
- Have the freedom and flexibility to work from where you do your best work
- Attend conferences and meet with team members from across the globe
- Work with cutting-edge open source technologies and tools
Originally posted on Himalayas
Como me candidatar
Candidatar no anúncio original ↗
Preparar candidatura
Uma carta de apresentação e uma apresentação curta para esta oferta, escritas a partir do teu perfil, e o que aprender a seguir.
Entra e cria o teu perfil a partir do CV. É privado: não o partilhamos com ninguém.
Algo errado ou ilegal neste anúncio? Reportar
Ofertas semelhantes
- Staff Software EngineerZencoder
- Staff Software Engineer - AI Copilot (Backend)Zendesk
- Staff Software EngineerStartup Talents
- Staff Security EngineerZencoder
- Staff Software EngineervCluster Labs
- Staff Software EngineerZenRows
Salário comparado ⓘ
68 ofertas · Programadores de software · teletrabalho
ver as que pagam mais →Formação para esta vaga
- DevOps with Docker & KubernetesGalileuonline · horário laboral · 28 h · 1620 € + IVA · início 15/02/2027Pedido neste anúncio: Docker e Kubernetes
- Running Containers on Amazon Elastic Kubernetes Service (Amazon EKS)Rumosonline · horário laboral · 21 h · 2100 € · início 18/01/2027Pedido neste anúncio: Docker e Kubernetes
- Pós-Graduação Online em Data Analytics & Big DataUniversidade Europeiaonline · 35 semanas · início 03/2027Pedido neste anúncio: Engenharia de dados (Spark, Databricks)
Idiomas IA
Publicado em (1)
- Himalayas visto 17 nov
Vista pela primeira vez a 17 nov, 00:11