Are closed embedding APIs worth paying for in Portuguese?
The top of the MTEB-BR leaderboard is a closed API. Google’s Gemini-Embedding-001 posts the highest 22-task mean, $0.682$, ahead of OpenAI, Voyage, Cohere, Mistral, and Amazon. So the deployment question looks settled: pay for the best. It isn’t, and the reason is the shape of the cost–quality frontier.
Twenty of our 93 models are closed commercial APIs, each publishing a single per-million-token list price. We place all twenty on a single frontier of price against quality.
Four of twenty are Pareto-optimal
Reading the frontier from cheap to expensive, four endpoints are not dominated by any other:
| Endpoint | Price / 1M | 22-task mean |
|---|---|---|
| Voyage-3.5-lite | $0.02 | 0.620 |
| Voyage-4 | $0.06 | 0.643 |
| Voyage-context-4 | $0.12 | 0.668 |
| Gemini-Embedding-001 | $0.15 | 0.682 |
The other sixteen priced endpoints are dominated: some cheaper model scores at least as well.
The frontier is shallow
Here is the part that should change the decision. Going from the cheapest endpoint to the best is a 7.5× increase in price for a gain of just $0.062$ in the 22-task mean. And $0.062$ is only about three times the width of the statistical-tie band on this benchmark, the range within which paired-bootstrap tests cannot tell two models apart.
Pricing, not just scale, shapes the frontier. OpenAI’s text-embedding-3-large ($0.13, $0.645$) is dominated outright by Voyage-context-4 ($0.12, $0.668$), which is both cheaper and better — you can pay a premium for a model another vendor beats on price and quality at once.
And a free model ties the leader
The closed leaders are not even uniquely good. Where self-hosting is acceptable, the open-weight Qwen3-Embedding-8B (Apache 2.0, $0.670$) is statistically tied with the top closed APIs on the 22-task mean, at zero per-token cost. It trades a per-call fee for self-hosting infrastructure, which is an operations question, not an accuracy one.
How to actually choose
Put together, the picture is clear: on native Brazilian Portuguese, the closed APIs are clustered tightly in quality, the price spread between them is large, and an open model reaches the same tier for free. So the right way to read this frontier is in tiers, not ranks. Pick a cost tier you can afford, and within it decide on the things the leaderboard does not measure:
- Latency and throughput: round-trip time, rate limits, batch support.
- Context length: how long your documents are.
- Region and data residency: where the data is allowed to go.
- License and self-hosting: whether an open-weight model removes the per-token fee entirely.
The leaderboard tells you which models are in contention. For Portuguese, that set is wider, and cheaper, than the top rank suggests. Explore it on the live leaderboard.