Real-time LLM Inference on Standard GPUs: 3k tokens/s per requestblog.kog.ai
10 points by NicoConstant 49 minutes ago | 0 comments