Member of Technical Staff, ML & Inference Infra

About the company

Our client is a seed-stage AI startup in the San Francisco Bay Area, founded in 2025 by two ex-Google engineers who led search infrastructure and reinforcement-learning post-training for Google's AI search products. The team of four has just closed an $8M seed round. The product is post-training and inference infrastructure that turns what production agents actually do into evaluations, reward signals and specialised models that keep improving.

The role

You would be one of the first engineers and the infrastructure expert in the room, owning the serving and training infrastructure for specialised language models across both experimentation and production. The day-to-day is profiling and improving latency, throughput, GPU utilisation and cost — batching, caching, memory and model execution — and evaluating techniques such as quantisation and speculative decoding. You would run inference for evaluations, data generation and reinforcement-learning rollouts, split GPU capacity between training and serving, and deploy post-trained models into customer environments. The priority is building this infrastructure from scratch so the company stops depending on third-party providers, on real workloads rather than toy benchmarks.

What you'll do

• Profile and improve latency, throughput, GPU utilisation and cost across batching, caching, memory and model execution
• Evaluate and ship low-level techniques such as quantisation, speculative decoding and multi-token prediction
• Run inference for evaluations, data generation and reinforcement-learning rollouts
• Split GPU capacity between training and serving, and own deployment and monitoring
• Deploy post-trained models into customer environments, and build the serving stack from scratch rather than on third-party providers

Nice to have

• Experience at an AI-native company that trains its own models
• A technical degree from a top programme
• Publications at NeurIPS, ICML, ICLR or CVPR, or public writing about your systems work
• Low-level optimisation: CUDA or Triton kernels, quantisation, speculative decoding, or distributed and multi-model serving
• Having tuned inference for one specific workload, or managed a GPU fleet

Tech stack

Large language model serving frameworks such as vLLM, SGLang and TensorRT-LLM, or a comparable custom engine, in PyTorch or JAX; GPU orchestration across training and serving; quantisation and speculative decoding; and the evaluation, reward-signal and post-training pipelines that sit on top.

Compensation & logistics

• $200k–$350k base, plus 0.1–2% equity; the client would rather weight an offer towards equity
• On-site five days a week in Mountain View; the office may move towards Palo Alto
• The client can take over an existing work visa, such as an H-1B transfer or OPT, but is not filing new, first-time visa sponsorships for this role
• One to two hires on this seat
• Interview process: a 30-minute founder intro chat, a one-hour technical deep dive on past projects with no coding, and an optional one-hour discussion of how you would build the post-training and inference infrastructure. The founders want to meet every candidate in person at least once
• Full-time