What are vector embeddings? An interactive guide

tl;dr

This guide walks you through the history, design, theory, and practical applications of vector embeddings. For the first time, we bring this abstract subject to life through interactive visualizations that let you see and manipulate the invisible geometry of meaning underpinning everything from Google Search and your Twitter feed to ChatGPT.

We begin with the problem embeddings were created to solve: how to represent meaning numerically so that computers can work with it. From there, we build an intuitive understanding of semantic space, vector arithmetic, and semantic similarity.

We then put these principles to work, showing how embeddings can power real business systems through techniques such as semantic search, hierarchical retrieval, zero-shot classification, clustering, and deduplication.

No programming experience is required. For readers who want to recreate the techniques, we have included short Python snippets demonstrating how each one works.

For a taste of what lies ahead, try the GeoGuessr/Wordle-style word-trivia game below. It projects 43,339 English words onto a 2D map using real embeddings from Kanon 2 Embedder, then selects one as the target. Enter words or vector expressions to place markers, then use them to trace the smallest possible loop around the target word. We explain how embeddings power the map and its mechanics later in the guide; for now, let the spirit of exploration guide you as you take your first plunge into the hidden geometry of language.

Loading word vectors

Introduction

Vector embeddings are among the most common and versatile tools in modern natural language processing. They power everything from Google Search and Netflix recommendations to the memory and retrieval capabilities of AI chatbots. At their core, embeddings allow computers to work with not only the characters in a piece of text, but also the meaning those characters cumulatively convey.

The underlying problem is simple. Humans intuitively understand words and sentences, while computers ultimately work with binary digits represented as 0s and 1s.

For a computer to process text, its characters must first be expressed in this binary form. In UTF-8, for example, the capital letter C is represented by the byte 01000011. Once encoded, it can be stored, copied, and processed. This translation into a machine-readable form is known as encoding.

Character encoding tells a computer which symbols appear, but not what those symbols mean. What if we could encode the meaning of words and sentences in a similar way?

An embedding model attempts this transformation at a higher level. Rather than encoding individual characters, it encodes the meaning of a word, sentence, or document as a vector, which is simply a list of numbers. The sentence a warm summer evening, for example, might become [0.18, -0.42, 0.73, ...]. Once meaning has a numerical form, computers can compare texts, search by meaning, and identify relationships that their exact wording alone cannot reveal.

Despite their importance, embeddings remain poorly understood. Generative models expose their behavior through the text they produce, while embedding models return long lists of numbers that are far more difficult to interpret directly. Most people therefore encounter embeddings indirectly, beneath search engines, recommendation systems, and AI chat services.

This lack of visibility affects engineers too. Because generative models are easy to prompt and work with, they are often chosen over faster, cheaper, and more accurate model alternatives. The recent attention around Jev, a non-generative model designed to make fast, structured decisions inside software, reflects a growing interest in alternatives to using chat models for every AI task. Embedding models are another important part of this broader non-generative toolkit.

Before we can use embeddings to automate sophisticated decision-making tasks, we need to start with the fundamentals: what embeddings are, how they work, and how they came to be.

A short history of vector embeddings

Humans have been grappling with the relationship between symbols and meaning for as long as we have been writing.

As far back as 3000 BC, writing systems such as cuneiform and Egyptian hieroglyphs used visual forms to represent objects, sounds, and ideas. A sun could be represented by a disc 𓇳, water by a series of ripples 𓈗, and a house by its floor plan 𓉐. Other signs depicted a human being 𓀀 or the head of an ox 𓃾.

Representing a sun or an ox is relatively easy. Representing the past, fear, intention or sadness is much harder. These concepts have no physical form that can be reduced to a simple picture. So how do we represent abstract meaning without turning every symbol into a Rorschach test?

One approach is to arrange words along scales that capture part of their meaning. Temperature-related words, for example, have a natural order that we can encode numerically.

WordValue4-bit encoding
frigid−31101
cold−21110
cool−11111
lukewarm00000
warm10001
muggy10001
hot20010
torrid30011

The final column shows how each value can be stored in signed 4-bit binary. The numbers are illustrative, but assigning each word a numerical position allows a computer to perform arithmetic whose results reflect our intuition about how the words relate.

cold (−2) + hot (+2) = lukewarm (0)

This is a rudimentary one-dimensional embedding. It captures temperature, but nothing else. Warm and muggy, for example, occupy the same position despite having different meanings. Adding humidity as a second dimension would separate them, while a third could capture register, distinguishing an everyday word such as hot from a more literary word such as torrid.

Adding more dimensions would produce richer representations, but deciding what every dimension should mean and assigning every word a position by hand would quickly become impractical. Modern embeddings grew out of attempts to learn this structure automatically from how language is used.

In 1990, latent semantic analysis introduced one way to do this. Instead of manually defining properties such as temperature or humidity, it represented words according to how often they appeared across a collection of documents. Each document initially acted as a dimension, producing a large but crude map of word usage.

That map was then compressed into a smaller set of latent dimensions. Words that appeared across similar documents received similar positions, allowing relationships to emerge from patterns in the text rather than being defined by hand.

Neural language models later learned these representations directly from unlabeled text through self-supervised learning. Word2vec helped popularize this approach in 2013 by learning relationships between words from their surrounding context. Modern embedding models extend the same principle to phrases, sentences, and entire documents.

Unlike our temperature example, the dimensions learned by these models rarely correspond to a single property that we can name. Meaning is distributed across the vector as a whole. Taken together, a few hundred dimensions can capture distinctions as subtle as differences in legal wording between jurisdictions or the passive aggression buried in a midday text from your partner.

Vectorization

The process of converting plain text into a vector embedding is known as vectorization. Modern embedding models such as Isaacus’ Kanon 2 Embedder can vectorize anything from a single word to an entire document in milliseconds.

Kanon 2 represents every input as a fixed-length list of 1,792 numbers. Each number is a coordinate along one of the model’s learned dimensions, giving every embedding a dimensionality of 1,792. Together, these coordinates determine the text’s location within a high-dimensional vector space, where passages with similar meanings tend to appear closer together.

To see this structure in practice, the visualization below projects a small set of word embeddings into three dimensions. Each point represents a word, while the distances and directions between points reflect how the model relates them.

Input
→
Model Kanon 2 Embedder
→
Vector dimensions 1,792 dimensions −0.007+0.027−0.001+0.031−0.020−0.007−0.018+0.010 …

Reducing 1,792 dimensions to three inevitably loses information and distorts the exact distances between points. Even so, enough of the original structure remains for several relationships to be visible.

The horizontal displacement from king to queen follows roughly the same direction as the displacement from male to female. This suggests that the model represents the distinction between these gendered concepts consistently.

Switching the visualization to 2D makes a second relationship easier to see. Once depth is removed, the horizontal and vertical displacement from English to French resembles that from London to Paris. In this projection, one direction has come to encode a separation between classes of cities and nationalities, while another preserves the distinction between English-associated concepts and French-associated ones.

Vector arithmetic

An embedding represents each word as a point in semantic space, or equivalently as a vector from the origin to that point. Subtracting two word vectors produces another vector: the displacement between them. A word vector tells us where a concept is, while a displacement vector describes how meaning changes as we move from one concept to another.

Because word and displacement vectors share the same coordinate space, we can apply a relationship found between one pair of concepts to another. Subtracting man from king, for example, captures the shift from a man to the gender-neutral concept of a monarch. Adding woman to that displacement vector reintroduces gender and produces a vector close to queen.

king − man + woman ≈ queen

To build a better intuition for vector arithmetic, try some of the following expressions in the interactive visualization below:

  • london − english + french
  • puppy − dog + cat
  • running − land + water
  • avg(breakfast, lunch, dinner)
  • 0.4 × noodles + 0.6 × pasta
Loading word vectors
≈ —

For most arithmetic operations, the results will not always resolve as neatly as king − man + woman. Embedding arithmetic reflects statistical patterns learned from language rather than a fixed system of symbolic rules.

Nevertheless, using vector arithmetic through an exercise like this is helpful for understanding what knowledge embedding models are potentially encoding.

Semantic similarity

So far, we have seen how embeddings map words and phrases into a high-dimensional semantic space, and how vector arithmetic can reveal relationships within it. The embeddings themselves, however, are still lists of 1,792 numbers. Computers can process these lists easily, but they are largely unintelligible to humans. To make practical decisions with them, we need to reduce the relationship between two vectors to a single score.

Linear algebra gives us several ways to make this comparison. Each examines a different aspect of the vectors’ geometry and returns a scalar, which is simply a single number. We can then use this value to estimate semantic similarity, or how closely two pieces of text are related in meaning. Estimates of semantic similarity include:

  • Euclidean distance measures the straight-line distance between two points. A smaller distance indicates greater similarity.
  • Dot product measures both the alignment and magnitude of two vectors. Greater alignment produces a higher score, although longer vectors can dominate the result.
  • Cosine similarity measures the angle between two vectors while ignoring their length. Vectors pointing in the same direction score close to 1, perpendicular vectors score 0, and opposing vectors approach -1.

Cosine similarity is commonly used for text retrieval because it compares semantic direction while normalizing away differences in vector length that may be unrelated to relevance. Mathematically, it is written as

cos ( A , B ) = A · B ‖ A ‖ · ‖ B ‖

The visualization below shows how cosine similarity turns this geometry into a score. Choose a reference sentence to compare its vector with the other examples. The more closely two sentences are related in meaning, the more closely their vectors align, producing a smaller angle and a higher similarity score.

Similar to

With ‘A bat is flying out of a cave’ selected as the reference, ‘A flying fox slipped out of the grotto’ receives the highest similarity score of the remaining sentences at 0.55. A flying fox is a type of bat, while a grotto is similar to a cave. The model recognizes this relationship despite the differences in wording.

‘His bat went flying from his hands’ shares the words bat and flying with the reference, but scores lower at 0.48 because bat now refers to sporting equipment. When this sentence becomes the reference, ‘His racket slipped from his grasp’ scores 0.60. Both describe a piece of sporting equipment leaving someone’s grip, even though they use different nouns and verbs.

This is what makes semantic similarity useful. It can connect different phrases that express similar ideas while distinguishing between different meanings of the same word.

A map of meaning

Semantic similarity allows us to compare two vectors at a time. When we apply the same comparison across an entire vocabulary, a much larger structure emerges. Words with related meanings gather into neighborhoods, while unrelated words move further apart.

The map below contains embeddings for 43,339 English words generated using Kanon 2 Embedder. Stored at the model’s original 1,792 dimensions using full floating-point precision, the vectors alone would occupy around 311 MB of memory. By reducing them to 512 dimensions and quantizing each value to 4 bits, we have compressed the complete atlas into a 12.3 MB file that can be loaded directly in your browser.

To make the map navigable, we project its 512-dimensional vectors into 2D. The original 512-dimensional representations remain in the browser for more precise vector arithmetic, while the projection is used to draw the map. It cannot preserve every relationship, but broader patterns remain visible: names, places, actions, emotions, and other related concepts form recognizable regions across the space.

To put your understanding of embeddings to the test, we have turned the map into a game inspired by Wordle and GeoGuessr. Each round randomly selects one of its words as the target. Enter words or vector expressions to place markers, then use those markers to trace a loop around it. Fewer moves and a smaller enclosed area produce a better score.

To explore the map freely, select the flask icon in the top-right corner to enable sandbox mode.

Loading word vectors

Even in this compressed form, the map preserves a surprising amount of nuance. Before being projected into 2D, the words were clustered using their higher-dimensional vectors, a process we will introduce later. Reading a word’s position alongside its cluster label can therefore reveal relationships that the projection alone cannot fully preserve.

The word butler, for example, belongs to the surnames cluster but sits close to people and social roles. Both signals make sense: Butler is a surname, while a butler is also an occupation.

Similar patterns appear among band names. Beatles, Nirvana, and Interpol are associated so strongly with their respective bands that they appear within cultural names and references, alongside names such as PlayStation, Netflix, Oreo, and Zelda. By contrast, oasis, blur, eagles, and pavement remain closer to their ordinary meanings despite also being band names.

A single word embedding must reconcile all the senses encountered during training. In the 2D projection, the stardom of Beatles and Nirvana appears strong enough to overcome the semantic gravity of their non-musical associations. Their higher-dimensional vectors still retain both sets of associations, just as our earlier bat example showed how context can distinguish the animal from the sporting equipment.

From theory to practice

So far, we have treated embeddings primarily as a way of representing and comparing meaning. We have turned text into vectors, explored their geometry, and reduced the relationship between two vectors to a single similarity score. The next step is to put those ideas to work.

For the rest of this guide, we will follow a single case study showing how embeddings can support a range of natural language processing tasks within a business.

Our case study is Chronos Pty Ltd, a fictional company offering time travel as a service, or TTaaS. Chronos lets customers book holidays in different historical periods and transports them using its proprietary time machines. As the service has grown, so has the volume of customer support requests. Its team is now overwhelmed with everything from declined payments and scheduling issues to refund requests and malfunctioning machinery.

You have been brought in as a machine learning engineer to help. Using embeddings, you will rebuild Chronos’ support system one stage at a time, making its growing volume of requests easier to search, route, and resolve.

Although Chronos and its time-travel premise are fictional, the solutions are anything but. The same approaches apply far beyond customer support; indeed, the unusual setting demonstrates how readily an embedding model can recognize useful relationships within a domain it was never specifically trained to handle.

We begin where many customers first turn when something goes wrong: the help center.

A help center is usually the first line of defense for a support team. It allows customers to resolve common problems without opening a ticket by directing them to relevant documentation and support articles. This only works, however, if its search system can connect each query with the right material. Even the best support content is useless if customers cannot find it.

Many help centers rely heavily on keyword search, ranking articles according to the words they share with a customer’s query. This works when the customer uses the same terminology as the article’s author, but becomes less reliable when they describe the same problem differently.

Imagine that a Chronos customer searches for ‘It wouldn’t let me pay for my trip’. The relevant help article is titled ‘Why your payment was declined’. A person can immediately see the connection, but a keyword search may struggle because the query and title share little vocabulary.

Finding relevant material within a collection is known as information retrieval. Semantic search performs this task by comparing the meaning of queries and documents rather than relying on the keywords they share.

Semantic search is surprisingly straightforward to implement using vector embeddings. We embed each article in the help center once and store the resulting vectors. Together, these articles form our searchable collection, or corpus. Each corpus item now occupies a point in semantic space, much like the words in our earlier map.

When a customer enters a query, we embed it using the same model and compare its vector with every article vector using cosine similarity. The results can then be ranked from most to least semantically relevant.

"""Configure the in-memory vector database and define the embedding, indexing, and search helpers reused by the Chronos applications."""

import os

from isaacus import Isaacus
from qdrant_client import QdrantClient, models


isaacus = Isaacus(api_key=os.environ["ISAACUS_API_KEY"])

# These examples use an in-memory vector database so the same storage and search interface can be reused throughout the guide.
db = QdrantClient(":memory:")


# Kanon 2 Embedder distinguishes between incoming query embeddings and document embeddings, allowing for more accurate semantic comparisons.
def _embed(texts, task):
    response = isaacus.embeddings.create(
        model="kanon-2-embedder",
        texts=texts,
        task=task,
    )

    return [item.embedding for item in response.embeddings]


# Embed each corpus item once, then store its vector and original metadata in a cosine similarity collection.
def _index_records(collection, records, *, texts, task="retrieval/document"):
    vectors = _embed(texts, task)

    if not db.collection_exists(collection):
        db.create_collection(
            collection_name=collection,
            vectors_config=models.VectorParams(size=len(vectors[0]), distance=models.Distance.COSINE),
        )

    db.upsert(
        collection_name=collection,
        points=[
            models.PointStruct(id=i, vector=vector, payload=record)
            for i, (record, vector) in enumerate(zip(records, vectors, strict=True))
        ],
    )

    return vectors


# Compare one vector with the stored vectors using cosine similarity and return the closest matches first.
def _search_vector(collection, vector, *, limit=3, threshold=None):
    return db.query_points(
        collection_name=collection,
        query=vector,
        limit=limit,
        score_threshold=threshold,
    ).points

def _search_text(collection, text, *, limit=3, threshold=None, task="retrieval/query"):
    vector = _embed([text], task)[0]

    return _search_vector(collection, vector, limit=limit, threshold=threshold)

For a small help center, we can compare the query with every document directly. As the corpus grows to millions of passages, checking every vector becomes expensive. Approximate nearest-neighbor indexes make this more efficient by narrowing the search to the most promising candidates. These indexes are often managed through a vector database.

A conventional search interface might return only the highest-scoring result or a short ranked list. A Retrieval-Augmented Generation system, commonly known as RAG, usually retrieves the top k results and supplies them to a generative model as additional context.

The demo below compares semantic search with a simple keyword-based approach. Try changing the query and observe how each system ranks the same collection of Chronos support articles.

"""Use the helpers from Cell 1 to embed the Chronos help center articles, search them by meaning, and rank the results using cosine similarity."""
articles = [
    {
        "title": "Why your payment was declined",
        "text": "Your card issuer may have blocked the charge because of insufficient funds, an expired card or a fraud warning.",
    },
    {
        "title": "Ending a scheduled trip",
        "text": "Cancel an upcoming time jump before departure to release your booking and avoid further charges.",
    },
    {
        "title": "Locked out of your account",
        "text": "Use the recovery link sent to your email address to reset your password and regain access.",
    },
    {
        "title": "Getting reimbursed for a bad jump",
        "text": "You may request a refund if Chronos sends you to the wrong historical period.",
    },
    {
        "title": "Stranded in the past",
        "text": "Activate the emergency beacon if your time machine breaks down and leaves you in another century.",
    },
]

# Embed every help center article once and store its vector in the searchable corpus.
_index_records(
    "help_articles",
    articles,
    texts=[f"{article['title']}\n\n{article['text']}" for article in articles],
)

query = "It wouldn't let me pay for my trip"

# Embed the customer query, compare it with the article vectors, and rank the closest matches.
results = _search_text(
    "help_articles",
    query,
)

for result in results:
    print(f"{result.score:.3f}  {result.payload['title']}")

For the query ‘It wouldn’t let me pay for my trip’, semantic search correctly ranks ‘Why your payment was declined’ first. The keyword search approach, on the other hand, is distracted by the word trip and instead favors an article about ending a scheduled journey.

The same pattern appears with a conversational query such as ‘Make my login harder to break into’. Semantic search recognizes that the customer is asking about account security, even though the query does not use the exact terminology found in the relevant article.

Chunking and hierarchical retrieval

Semantic search works best when each candidate passage is short and focused. Longer documents present a greater challenge because the appropriate amount of text to retrieve depends on the query. A broad question may concern an entire webpage, while a specific one may be answered by a single sentence. Returning only whole pages forces the user to locate the answer themselves, even when the search system has identified the relevant detail.

The usual solution is to divide each document into smaller units called chunks. This process is known as segmentation or, more commonly, chunking. Each chunk is embedded separately, allowing shorter passages to be retrieved and ranked by relevance.

The difficulty lies in choosing the right chunk size. Large chunks may contain unrelated material, while small ones can split a coherent unit across several chunks. Fixed-length chunking may also cut across boundaries deliberately established by the author, such as combining the end of one chapter with the beginning of the next.

Hierarchical retrieval avoids choosing a single chunk size by representing a document as a tree. A Wikipedia article, for example, can be divided into sections, subsections, and paragraphs, with each part linked to its parent and children.

Every part of this hierarchy is embedded and stored in the same searchable index. When a query arrives, semantic search compares it with every level and returns the passage whose granularity best matches the question, while retaining the surrounding structure needed to interpret it.

Our hierarchical graphitization model, Kanon 2 Enricher, can construct this tree before embedding. Kanon 2 Embedder is then used to represent and retrieve passages across its different levels.

A specific query about the geology of Australia could therefore retrieve the relevant subsection from its Wikipedia article, while a broader question about Australia’s geography could return the parent section and its related subtopics.

The same approach can be applied to Chronos’ terms of service. Select a question below to see how the system searches across the hierarchy and isolates the most relevant passage.

"""Extract a small document hierarchy and retrieve its most relevant passage."""

terms_text = """
Chronos Traveller Terms of Service

1. Bookings and payment

1.1 When payment is taken

Chronos authorizes your payment method when you submit a booking.

(a) The full fare is charged only after Chronos confirms your booking.

(b) Additional purchases are charged when they are added to your itinerary.

1.2 Failed payments

A booking is not confirmed when the payment authorization fails.

2. Cancellations and refunds

2.1 Traveller cancellations

You may cancel a scheduled journey before its departure window closes.

(a) Cancellations made within 24 hours of departure incur a 10% fee.

(b) The fee is waived when Chronos cancels or materially reschedules the journey.
""".strip()

# Kanon 2 Enricher chunks the document, sections, subsections, and clauses based on their hierarchical structure.
response = isaacus.enrichments.create(
    model="kanon-2-enricher",
    texts=terms_text,
)

document = response.results[0].document

# Store the complete document and every extracted segment as separate candidates.
passages = [{"scope": "document", "text": document.text}]
passages.extend(
    {
        "scope": segment.type or segment.kind,
        "text": segment.span.decode(document.text),
    }
    for segment in document.segments
)

# Kanon 2 Embedder places every level of the hierarchy in the same searchable index.
_index_records(
    "terms_hierarchy",
    passages,
    texts=[passage["text"] for passage in passages],
)

# Return only the passage with the highest cosine-similarity score.
question = "What happens to the cancellation fee if Chronos materially reschedules my journey?"

result = _search_text(
    "terms_hierarchy",
    question,
    limit=1,
)[0]

print("Question:")
print(question)
print("\nAnswer:")
print(result.payload["text"].strip())
chronos.help/traveller-terms
chronos-traveller-terms.pdf
100%
100%

For a precise question such as ‘Exactly when is my payment method charged after I make a booking?’, the search engine retrieves the individual clause containing the answer rather than returning the entire agreement. For the broader question ‘What subjects are covered by the Chronos Traveller Terms of Service overall?’, it retrieves the document as a whole.

This allows Chronos customers to search not only across documents and webpages, but within them. Instead of first locating the correct document and then manually scanning it or guessing the right keywords, a single semantic query can identify both the relevant source and the appropriate level of detail.

Zero-shot classification

So far, we have used semantic search to help customers find relevant passages across and within Chronos’ documentation. Embeddings can also work behind the scenes to automate the support process itself.

Most organizations divide support work among specialist teams. A travel company might direct payment disputes to its accounts team and account-access problems to technical support. Asking customers to make this routing decision themselves can cause confusion, misdirected tickets, and longer wait times. In a large organization, triaging incoming requests can become a substantial workload of its own.

Classification is the process of assigning an input to a class or category. Traditional classifiers are trained using labeled examples from each class. Embeddings allow us to perform zero-shot classification, where categories can instead be defined in natural language and altered without retraining the model.

The process closely resembles semantic search. Instead of comparing a query with documents, we compare an incoming ticket with descriptions of each category. Adding a new class is as simple as writing a description and embedding it alongside the others. A class for abusive content, for example, might be described as ‘complaints containing slurs, offensive language, or hate speech’.

At runtime, an incoming ticket is compared with every category description using cosine similarity. It is assigned to the highest-scoring category when that score passes a minimum threshold. Low-confidence tickets can instead be sent to a general queue or human reviewer.

For Chronos, this means tickets can be routed according to their meaning, whether they concern billing, bookings, account access, or malfunctioning time machines.

"""Route a Chronos support ticket by comparing it with natural-language category descriptions."""

categories = {
    "Billing & credits": "A customer needs help with a payment, invoice, card charge, overcharge, or account credit.",
    "Booking & itinerary": "A customer needs help with a booking, destination, departure, itinerary, or schedule.",
    "Malfunction & rescue": "A time machine failed, a jump malfunctioned, or a traveler needs emergency rescue.",
    "Account & access": "A customer cannot sign in or needs help with a password, identity, or account security.",
    "Refunds & claims": "A customer requests a refund, reimbursement, compensation, or travel claim.",
}

category_records = [
    {"name": name, "description": description}
    for name, description in categories.items()
]

# Treat each category description as a query and store its vector for reuse.
_index_records(
    "support_categories",
    category_records,
    texts=list(categories.values()),
    task="retrieval/query",
)

ticket = "My time machine broke down and stranded me in the past."

# Embed the ticket as a document and retrieve its closest category description.
minimum_score = 0.45
matches = _search_text(
    "support_categories",
    ticket,
    limit=1,
    threshold=minimum_score,
    task="retrieval/document",
)

# Send tickets without a sufficiently similar category to the fallback queue.
route = matches[0].payload["name"] if matches else "Miscellaneous"
score = f"{matches[0].score:.2f}" if matches else f"below {minimum_score:.2f}"

print("Ticket:")
print(ticket)
print("\nRoute:")
print(route)
print("\nSimilarity:")
print(score)
chronos.help/support/routing
28% Similarity threshold
Best 0%100%
14Correct 0Misrouted 2Manual
3
3
3
3
2
2

The minimum similarity score acts as a decision threshold: the highest-scoring category must exceed it before a ticket is routed automatically. If no category clears the threshold, the ticket is sent to a fallback queue instead.

By testing different thresholds against a labeled validation set, we can balance the number of tickets routed automatically against the risk of incorrect assignments. This process is known as threshold tuning.

A fallback class is essential because every ticket will have a highest-scoring category, even when none of the available options is a good match. Sending these low-confidence cases for human review prevents weak matches from becoming confident misclassifications. Reviewing a small queue of uncertain tickets is far easier than correcting errors after they have been routed to the wrong teams.

Clustering and deduplication

Until now, we have compared each piece of text with something defined in advance: a document, a chunk, or a category description. We can also compare the items within a collection directly and use their relationships to discover groups. This process is known as clustering.

Unlike classification, clustering does not begin with predefined labels or category descriptions. The groups emerge from similarities within the data itself, making clustering useful when we do not know in advance what patterns the collection contains.

Deduplication addresses a narrower problem: identifying records that describe the same underlying item or event. Although clustering and deduplication are not the same task, clustering can be used to achieve deduplication. Each cluster can be treated as a set of potential duplicates, while items that do not belong to a group remain unique.

GitHub issues provide a familiar example. Several users may describe the same bug in different language, but because defects emerge unpredictably in production, an organization is unlikely to have a predefined label for each one. Clustering groups these reports around the same underlying problem without requiring that problem to be described in advance. Developers can then avoid investigating the same bug repeatedly while keeping everyone who reported it informed through a single thread.

To deduplicate an issue tracker, we first embed the title and description of every issue, then calculate the cosine similarity between each pair. Issues whose similarity exceeds a chosen threshold are connected, and each connected group becomes a potential duplicate cluster. One issue can then be retained as the canonical record while the others are linked to it. Reports with no sufficiently similar match remain separate.

The visualization below applies this process to Chronos’ fictional software repository. Adjust the threshold to see the clusters change, then select an issue to open its full report and inspect the other issues connected to it. The category labels are shown for context, but are not included in the embeddings used to form the clusters.

"""Cluster semantically similar issue reports and identify candidate duplicates without relying on predefined categories."""

issues = [
    {
        "id": 1842,
        "title": "Login loops after enabling two-factor authentication",
        "body": (
            "After I enter a valid verification code, the dashboard appears briefly "
            "before returning me to the sign-in page."
        ),
    },
    {
        "id": 1836,
        "title": "2FA users are redirected to the sign-in page",
        "body": (
            "The one-time code is accepted, but authentication never completes "
            "and the application redirects back to login."
        ),
    },
    {
        "id": 1804,
        "title": "Cannot access account after turning on 2FA",
        "body": (
            "My password and security code both succeed, but I am returned to "
            "the login screen and asked to authenticate again."
        ),
    },
    {
        "id": 1765,
        "title": "CSV export corrupts accented customer names",
        "body": (
            "Names such as José, François, and Müller display correctly in the app "
            "but become garbled in downloaded CSV files."
        ),
    },
    {
        "id": 1749,
        "title": "Non-ASCII characters are mangled in downloaded CSV files",
        "body": (
            "Exporting a report to CSV breaks Unicode text, including customer "
            "names containing accents and Japanese characters."
        ),
    },
    {
        "id": 1933,
        "title": "Dark mode makes code blocks unreadable",
        "body": (
            "Code blocks use nearly identical foreground and background colors "
            "in dark mode, making examples difficult to read."
        ),
    },
]

# Embed each issue's title and body.
issue_vectors = _index_records(
    "issue_reports",
    issues,
    texts=[f"{issue['title']}\n\n{issue['body']}" for issue in issues],
)

# Connect every pair whose cosine similarity passes the chosen threshold.
similarity_threshold = 0.55
connections = {i: set() for i in range(len(issues))}

for issue_index, vector in enumerate(issue_vectors):
    matches = _search_vector(
        "issue_reports",
        vector,
        limit=len(issues),
        threshold=similarity_threshold,
    )

    for match in matches:
        match_index = int(match.id)

        if match_index != issue_index:
            connections[issue_index].add(match_index)
            connections[match_index].add(issue_index)

# Each connected group becomes a potential duplicate cluster.
clusters = []
remaining = set(connections)

while remaining:
    seed = min(remaining)
    remaining.remove(seed)

    cluster = {seed}
    frontier = [seed]

    while frontier:
        issue_index = frontier.pop()
        neighbors = connections[issue_index] & remaining
        remaining -= neighbors
        cluster |= neighbors
        frontier.extend(neighbors)

    clusters.append(sorted(cluster))

# Retain the first issue in each cluster as its canonical record.
cluster_number = 0

for cluster in clusters:
    heading = "Unique"

    if len(cluster) > 1:
        cluster_number += 1
        heading = f"Cluster {cluster_number}"

    print(f"{heading}:")

    for position, issue_index in enumerate(cluster):
        role = "Unique"

        if len(cluster) > 1:
            role = "Canonical" if position == 0 else "Duplicate"

        issue = issues[issue_index]
        print(f"- {role}: #{issue['id']} {issue['title']}")

    print()
codehub.com/chronos-labs/chronos-web/issues
Similarity threshold
53%
Unmatched issuesOpen issues without a match3

At the recommended threshold, ‘Login loops after enabling two-factor authentication’ is grouped with ‘2FA users are redirected to the sign-in page’ and ‘Cannot access account after turning on 2FA’. Although the reports use different language, they describe the same underlying defect. An unrelated report such as ‘Dark mode makes code blocks unreadable’ remains unmatched.

By connecting sufficiently similar reports, we can deduplicate the issue tracker without defining every possible bug in advance. The cluster names are not predefined either. Once a cluster has formed, representative keywords can be extracted from its reports to generate a descriptive label automatically.

This is the same basic approach used to construct the semantic map introduced earlier in the guide. Before the words were projected into 2D, they were grouped in higher-dimensional space according to the proximity of their embeddings. These clusters revealed dense regions of semantically related vocabulary. Extracting representative terms from each region allowed us to identify groups such as surnames, countries, regions, and geography, and numbers and mathematics.

The same process can reveal structure in support tickets, documents, and many other forms of data, not just individual words. Clustering is a form of unsupervised learning, a family of methods that discovers patterns without labeled examples or predefined target categories. These methods are used extensively online, including within recommendation and content-ranking systems.

By grouping users according to their behavior, preferences, or interactions, a platform can identify clusters, or communities, of people with similar interests and personalize what each person sees. This might mean recommending a shoe brand based on the purchasing patterns of similar customers. More worryingly, the same approach could be used to target gambling promotions at users whose behavior suggests they may be particularly susceptible to them. The mathematics is indifferent to the outcome; whether it is used responsibly depends on the objectives, safeguards, and people behind the system.

As the old IBM adage goes: ‘A computer can never be held accountable, therefore a computer must never make a management decision.’ Embeddings can reveal patterns and automate decisions, but responsibility for their consequences ultimately rests with the humans behind the machine. Now that you understand how embeddings work, the final question is no longer what they can do, but what you will choose to do with them.

Beyond just embeddings

Embeddings are only the tip of the iceberg. They are powerful on their own, but become even more capable when combined with other machine-learning technologies. A generative model can turn retrieved passages into answers or summaries, while an enrichment model can organize unstructured documents into hierarchies and knowledge graphs before they are embedded. Together, these components can support everything from retrieval-augmented generation and question answering to personalized recommendations and large-scale knowledge discovery.

Explaining how these technologies work together falls outside the scope of this guide. If you would like to explore this topic further, our guide to building a legal RAG application shows how they can be assembled in practice, while Legal RAG Bench examines how retrieval and generation shape the performance of complete RAG systems.

If you enjoyed this guide and would like to stay up to date on our latest releases, follow us on LinkedIn, Twitter/X, or Reddit.

Citation

@misc{butler2026vectorembeddings,
      title={What are vector embeddings? An interactive guide},
      author={Abdur-Rahman Butler},
      year={2026},
      month={October},
      publisher={Isaacus},
      url={https://isaacus.com/blog/what-are-vector-embeddings},
}