Overview
- Full name
docker.io/lmcache/lmstack-router- Registry
- docker.io
- Namespace
- lmcache
- Repository
- lmstack-router
- Cached tags
- 146
- Last synced
- 07/05/2026, 09:47 PM
Introduction
LMCache LLM 推理路由镜像，转发请求至 KV cache 后端与模型 worker，优化大模型推理延迟与吞吐。
Details
lmcache/lmstack-router 运行 LMStack 请求路由层，将 LLM 推理流量分发至 LMCache 缓存节点与模型 worker。定位为无状态 LLM 推理 API 网关，配置 upstream worker 地址；K8s Deployment + Service 多副本水平扩展推理入口。
快速启动
docker run -d --name lmstack-router \
-p 8080:8080 \
-e WORKER_URL=http://model-worker:8000 \
docker.io/lmcache/lmstack-router:latest
推荐实践
docker run -d --name lmstack-router \
--restart unless-stopped \
-p 8080:8080 \
-e WORKER_URL=http://vllm-worker:8000 \
-e LMCACHE_URL=http://lmcache:8081 \
docker.io/lmcache/lmstack-router:0.2.0
核心参数说明
-p 8080:8080— 推理路由 API 端口-e WORKER_URL— 模型 worker 地址-e LMCACHE_URL— KV cache 后端- 依赖 — GPU worker 独立部署
Kubernetes
apiVersion: apps/v1
kind: Deployment
metadata:
name: lmstack-router
spec:
replicas: 2
selector:
matchLabels:
app: lmstack-router
template:
metadata:
labels:
app: lmstack-router
spec:
containers:
- name: router
image: docker.io/lmcache/lmstack-router:0.2.0
ports:
- containerPort: 8080
env:
- name: WORKER_URL
value: http://vllm-worker:8000
---
apiVersion: v1
kind: Service
metadata:
name: lmstack-router
spec:
selector:
app: lmstack-router
ports:
- port: 8080
targetPort: 8080
GPU worker 独立 NodePool；Ingress 限流与鉴权。