French startup Kog promises 30x faster LLM inference on standard GPUs

Kog's demo hit 3,000 tokens/sec single-request decoding on AMD MI300X and Nvidia H200 GPUs, toward its promised 30x faster inference. CEO Gaël Delalleau says 200 business leads came in, with software engineering the likely first use case. Since customers won't fine-tune small models, the startup now targets larger ones.

Featured · Gaël Delalleau