Language Models

OVHcloud's 7B Olala Mini Runs on an 8 GB GPU, With Benchmark Results Still Missing

November 18, 20261 views阅读中文原文

OVHcloud announced Olala Mini 7A1B, its first in-house language model. The mixture-of-experts model has seven billion parameters in total and activates about one billion per token. OVHcloud says it can run on an 8 GB GPU and reports CPU-only throughput of up to 50 tokens per second.

What OVHcloud disclosed

The vendor reports up to 160 tokens per second on an RTX A4500 and a 32K context window. It describes 128K as a planned update, so it should not be treated as a current capability. These figures describe different test setups and are not guaranteed on other hardware.

What is still unknown

No comparable quality score accompanied the announcement covered here. Throughput indicates how quickly the model can produce tokens, but it does not establish accuracy, instruction following, or quality in a particular language. Use the model card to confirm the license, format, and supported deployment targets before testing.

A useful first comparison

Put Olala Mini and an established model that fits the same hardware through actual prompts. Compare correctness, latency, memory use, and long-context behavior at the same batch size.

Sources

Comments

0 published
Log inorsign up to leave a comment.

Loading comments...