BroadRouter
Get started

Embeddings

Updated

Embeddings are lists of floating-point numbers (vectors) that a model produces to represent a piece of text, an image or other input. Inputs with similar meaning get vectors that are close together, so distance between vectors can rank search results, cluster documents or find related items.

Also called: text embeddings, vector embeddings, embedding vectors, embedding model, sentence embeddings

An embedding is how a computer measures meaning: two texts that say similar things get two vectors that point in similar directions.

How it works

You send text to an embedding model and get back one fixed-length vector. OpenAI’s guide puts it plainly: “An embedding is a vector (list) of floating point numbers.” The length is fixed per model. OpenAI’s text-embedding-3-small returns 1,536 numbers by default and text-embedding-3-large returns 3,072.

To compare two texts, you compare their vectors. OpenAI recommends cosine similarity. Its vectors are normalised to length 1, so a plain dot product gives the same ranking. A query vector compared against millions of stored vectors is the core of semantic search. Doing that fast at scale is the job of a vector database.

Embedding models are a different kind of model from chat models. They read a whole input and return numbers; they do not generate text. They are billed per input token, and there is no output text to pay for.

Why it matters

  • Retrieval. Embeddings are the usual first step of retrieval-augmented generation: embed your documents once, embed each question, fetch the closest chunks and pass them to the chat model.
  • Model choice changes results. The MTEB benchmark tested 33 models across 8 task types, 58 datasets and 112 languages and found that “no particular text embedding method dominates across all tasks.” Test on your own queries, especially for Arabic or mixed-language text.
  • Lock-in. Vectors from one model cannot be compared with vectors from another. Switching embedding models means re-embedding the whole corpus.

On BroadNet Inference

BroadNet’s docs list /v1/chat/completions, /v1/audio/transcriptions and GET /v1/models (a model list). There is no embeddings endpoint. For a search or RAG system, pair a separate embedding model (a hosted API or an open model you run yourself) with a BroadNet chat model for the answer. The Arabic RAG guide shows this split with a multilingual embedding model.

If your documents fit, a long context window can sometimes skip retrieval entirely. The trade-offs are in long context vs RAG.

Common mistakes

  • Mixing vectors from two embedding models in one index.
  • Embedding whole documents instead of chunks. One vector for 50 pages blurs every topic in it.
  • Normalising documents but not queries, or the reverse. Apply the same text cleaning to both.

Questions

Are embeddings the same as tokens?

No. A token is a piece of text the model reads. An embedding is the vector the embedding model outputs for a whole input, such as a sentence or a paragraph.

How many dimensions does an embedding have?

It depends on the model. OpenAI's text-embedding-3-small returns 1,536 numbers by default and text-embedding-3-large returns 3,072, and both can be shortened with a dimensions parameter.

Does BroadNet Inference have an embeddings endpoint?

No. BroadNet's docs list only /v1/chat/completions and /v1/audio/transcriptions. Use a separate embedding model for retrieval and BroadNet for the generation step.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity