Backend engineering ownership on the speech platform – build audio pipelines, real-time inference services, GPU infrastructure, and on-prem deployment.

About the Company

Rezo.ai is a fast-growing AI startup building a conversational intelligence platform that helps enterprises analyze and act on voice interactions. The company’s speech platform powers real-time transcription, sentiment analysis, and agent coaching for contact centers. With a strong focus on engineering excellence and low-latency serving, Rezo.ai is backed by prominent venture capital firms and operates in a high-growth segment of enterprise AI.

Role Overview

We are looking for a Senior Backend Engineer to join our speech platform team. You will be the engineering owner on the speech team, working alongside two ML engineers to build everything around the models—ingestion, training infrastructure, real-time serving, and integration into our existing platform. This is a pure backend engineering role; you don’t need to train models, but you need to build the systems that make trained models useful in production. You will own the audio data pipeline, real-time inference services, GPU scheduling, on-prem deployment, and cost economics.

Key Responsibilities

• Build the audio data pipeline: mine our call archive, transcode, resample, segment, deduplicate, and quality-filter at scale.
• Build and operate real-time inference services with hard latency targets, including streaming, cancellation, and mid-utterance interruption.
• Integrate speech services into our existing Java-based platform and telephony infrastructure.
• Instrument the full latency budget end to end and find where the milliseconds go.
• Stand up training infrastructure—GPU scheduling, checkpointing, experiment tracking, reproducibility.
• Own compute cost and concurrency economics: how many simultaneous calls per GPU must be determined and improved.
• Support on-premise deployment for clients who cannot send data outside their network.
• Collaborate with ML engineers to ensure models are production-ready and optimized for inference.
• Monitor and debug performance issues, profiling latency percentiles and throughput under load.

Tech Stack

Requirements

• 3–4 years building and operating production backend systems.
• Strong Java expertise—you have owned services in production, not just contributed to them.
• Working Python—enough to build data pipelines and integrate with ML tooling.
• Real-time or low-latency systems experience: streaming APIs, WebSocket or gRPC streaming, concurrency, backpressure.
• Data pipelines at scale (Airflow, Dagster, Spark or equivalent).
• Docker and Kubernetes in production.
• Cloud infrastructure (AWS/GCP/Azure).
• Comfortable debugging performance: profiling, latency percentiles, throughput under load.
• Nice to have: serving ML models in production (Triton, vLLM, TorchServe); audio tooling; telephony experience; MLOps tooling; GPU-aware infrastructure work.

3–4 years building and operating production backend systems. Strong Java — you've owned services in production, not just contributed to them. Working Python — enough to build data pipelines and integrate with ML tooling. Real-time or low-latency systems experience: streaming APIs, WebSocket or gRPC streaming, concurrency, backpressure. Data pipelines at scale (Airflow, Dagster, Spark or equivalent). Docker and Kubernetes in production. Cloud infrastructure (AWS/GCP/Azure). Comfortable debugging performance: profiling, latency percentiles, throughput under load.

Not specified in input.