|ARTIFICIAL INTELLIGENCE| EMBEDDING| CODING|

Lord of Vectors: One Embedder to Rule Them All

Embedders are back in vogue, so why not have a universal one?

Photo by Roman Derrick Okello on Unsplash

Embeddings are used today to retrieve knowledge or memories for LLMs, to build content moderation filters, or for many other applications. These tasks can be not only in English but also in different languages, not to mention that they are now also applied to code and programming languages. How can we have an embedding that works in all these cases?

Are the embedding back in the game?

Photo by Riho Kroll on Unsplash

In Natural Language Processing, embeddings are a key component. Word2vec was the first model, in which discrete text sequences were mapped into continuous vectors. The embedding layer is a fundamental component of large language models (generally the first layer).

In any case, embeddings are not only a component of Large Language Models (LLMs), but they are also used for various applications such as semantic textual similarity (STS) and search systems. Today, embeddings have gained popularity again because they are used…