Senior Software Architect, AI Networking

Design and evolve scalable architectures for multi-node LLM inference across GPU clusters. Develop infrastructure to optimize latency, throughput, and cost-efficiency of serving large models in production. Collaborate with model, systems, compiler, and networking teams to ensure holistic, high-performance solutions. Prototype novel approaches to KV cache handling, tensor/pipeline parallel execution, and dynamic batching. Evaluate and integrate new software and hardware technologies relevant to model inference (e.g., memory hierarchy, network topology, modern inference architectures). Work closely with internal teams and external partners to translate high-level architecture into reliable, high-performance systems. Author design documents, internal specs, and technical blog posts and contribute to open-source efforts when appropriate. Experience working on LLM inference pipelines, transformer model optimization, or model-parallel deployments.