https://plumephp.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E6%8E%A8%E7%90%86/Recent content in 分布式推理 on PlumePHPHugozh-CNSun, 27 Sep 2026 09:30:00 +0800https://plumephp.com/ai-distributed-inference-gpu-cluster/Sun, 27 Sep 2026 09:30:00 +0800https://plumephp.com/ai-distributed-inference-gpu-cluster/<blockquote> <p>当单卡放不下一个 70B 模型，或单卡无法满足线上吞吐时，分布式推理就从&quot;可选优化&quot;变成&quot;必选项&quot;。本文系统讲解张量并行、流水线并行与专家并行三大策略的通信代价与适用场景，介绍连续批处理在集群中的落地，并深入 KServe 与 Ray Serve 两类 GPU 调度器的架构与配置。</p>