Between the era of pure exact matching and the arrival of transformer-based contextual encoders, there was a decade in which a single idea reshaped how machines compared text: the dense word vector. Word2Vec, introduced by Mikolov and colleagues at Google in 2013, demonstrated that the meaning of a word could be captured to a useful approximation as a point in a few hundred dimensions of real-valued space, learned from nothing more than the company a word kept in a large corpus. GloVe, from Pennington, Socher and Manning at Stanford in 2014, offered a mathematically cleaner formulation of the same insight. FastText, from Bojanowski, Grave, Joulin and Mikolov at Facebook in 2016, extended the approach below the word level by representing each word as a bag of character n-grams.
These three models – and the toolkit of techniques built around them – remain in active use today. They are not state-of-the-art for most NLP tasks; that distinction belongs to contextual encoders like BERT and the embedding heads of large language models. But for plagiarism detection specifically, they retain a set of practical advantages that keep them in production pipelines: they are fast, they are cheap to deploy, their output is stable and inspectable, and – in the case of FastText – they handle the misspellings, morphological variation and out-of-vocabulary text that defeats both exact matching and many heavier alternatives. This article is a working comparison of what each of these classic embedding approaches actually does, where each excels, and where the subword model in particular earns its place in the modern detector’s stack.
What distributional meaning means
Every method discussed here is built on a single hypothesis, articulated by the linguist J. R. Firth in 1957: you shall know a word by the company it keeps. The distributional hypothesis says that words appearing in similar contexts tend to have similar meanings, and that – by extension – the meaning of a word can be approximated by some summary of the contexts in which it occurs. Latent Semantic Analysis, introduced in 1990, was an early implementation: it built a term-document matrix, applied singular value decomposition, and represented each word as a row of the resulting low-rank approximation. Words used in similar documents ended up with similar vectors.
Word2Vec, GloVe and FastText are all refinements of this basic move. They replace the document-level co-occurrence of LSA with word-level co-occurrence in a sliding context window; they replace SVD with neural or log-bilinear training objectives that scale to billions of tokens; and they produce vectors typically 100 to 300 dimensions in size, in which semantic relationships emerge as geometric ones. The standard demonstration is the analogy:
vector("king") − vector("man") + vector("woman") ≈ vector("queen")The fact that simple vector arithmetic recovers gendered, geographical and grammatical relationships is the most visible evidence that these models are capturing something more than surface frequency. For plagiarism detection, the relevant consequence is more prosaic: synonyms cluster.
Quick and rapid, large and huge, purchase and buy end up near each other in vector space, which means a paraphrase that substitutes one for the other no longer evades detection at the word level. This is the door that exact matching cannot open, and that distributional embeddings opened first.
Word2Vec: the foundation
Word2Vec was not the first distributional model, but it was the one that made the technique practical at scale. Its two architectures – Continuous Bag-of-Words (CBOW), which predicts a target word from its surrounding context, and Skip-gram, which predicts the surrounding context from a target word – are both shallow neural networks with a single hidden layer. The trick that made them tractable was negative sampling: rather than computing a softmax over the entire vocabulary at every training step, Word2Vec samples a handful of “negative” words at random and trains the model to distinguish them from the true positive. The result is a model that trains in hours rather than weeks on commodity hardware, and produces vectors of competitive quality.
The skip-gram objective, in its negative-sampling form, can be written as a sum over training pairs of the log-sigmoid of the dot product between target and context vectors:
J = Σ [ log σ(v_c · v_w) + Σ log σ(−v_n · v_w) ]where v_w is the target word, v_c is a true context word, and the inner sum is over k negative samples v_n. The geometry that emerges is the geometry of co-occurrence: words that share contexts end up with similar vectors, and the angle between two vectors – measured by cosine similarity – becomes a proxy for semantic relatedness.
For plagiarism detection, the typical pipeline using Word2Vec embeddings is straightforward. Each document is reduced to a vector by averaging (or weighted-averaging, often with TF-IDF weights) its constituent word vectors. Two documents are then compared by computing the cosine similarity of their vectors. Passages with high similarity but limited literal overlap are surfaced as candidate paraphrases for human review. The approach is crude – it discards word order entirely, it loses the contribution of any single distinctive phrase in a sea of common words, and it is essentially helpless on documents that share topic but not content – but it does, robustly, catch synonym-substitution paraphrase that exact matching misses.
The serious limitations of Word2Vec, for any application including plagiarism detection, are well-documented. It is a window-based model that captures only local context and does not exploit the global statistics of the training corpus. It assigns one vector per word, collapsing polysemes – bank the financial institution and bank the river edge – into a single point. And it cannot handle out-of-vocabulary words at all: any token not seen during training simply has no vector, and is either ignored or replaced with a zero vector that contributes nothing.
GloVe: global vectors and what they buy you
GloVe, introduced by Pennington, Socher and Manning at EMNLP 2014, addresses the first of these limitations. Where Word2Vec trains on local context windows in sequence, GloVe constructs the global word-word co-occurrence matrix for the entire corpus once, then fits a weighted least-squares model to its log-counts. The model is described in the paper as a “global log-bilinear regression model that combines the advantages of the two major model families” – the global matrix factorisation of LSA and the local context window approach of Word2Vec.
The GloVe objective, dropping constants, is:
J = Σ f(X_ij) · ( w_i · w_j + b_i + b_j − log X_ij )²where X_ij is the number of times word j appears in the context of word i, w_i and w_j are the word vectors, b_i and b_j are bias terms, and f is a weighting function that downweights very rare and very frequent co-occurrences. The intuition behind the model is that ratios of co-occurrence probabilities – not raw counts – encode meaningful relationships between words. The Stanford team’s own description on the project page is concise: GloVe produces “a vector space with meaningful sub-structure,” reaching 75% accuracy on the standard word-analogy benchmark and outperforming Word2Vec on word similarity and named-entity recognition tasks given the same corpus and training budget.
For plagiarism detection, the practical question is whether the global-statistics approach actually buys you better paraphrase detection than skip-gram, and the empirical answer is: usually, slightly, but not dramatically. The 2021 corpus-based paraphrase detection study by Vrbanec and Meštrović in MDPI Mathematics, which evaluated eight embedding methods including GloVe and FastText on the Microsoft Research Paraphrase Corpus, the Clough and Stevenson corpus, and the Webis Crowd Paraphrase Corpus 2011, found that GloVe and Word2Vec performed at broadly similar levels across all three benchmarks, with neither consistently dominant. The choice between them in practice is usually driven by which pre-trained vectors are available for the relevant language and domain, not by any decisive performance gap.
GloVe inherits two of Word2Vec’s limitations. It still produces one vector per word, so polysemy collapses; and it still has no way to vectorise out-of-vocabulary words. The third limitation – being memory-heavy because the co-occurrence matrix has to be constructed and stored – is real but largely a one-time training cost. Once the vectors are released, deploying them is no more expensive than deploying Word2Vec.
FastText: subwords and why they matter
FastText, introduced by Bojanowski, Grave, Joulin and Mikolov in Enriching Word Vectors with Subword Information, takes a different and, for plagiarism detection, decisive step. Instead of treating each word as an atomic unit with its own vector, FastText represents each word as a bag of character n-grams, typically of length 3 to 6, plus the word itself. The vector for the word where, with n ranging from 3 to 6, is the sum of vectors for:
<wh, whe, her, ere, re>, <whe, wher, here, ere>, <wher, where, here>, <where>, <where>where < and > are special boundary markers distinguishing prefixes and suffixes from internal n-grams. Two consequences follow immediately. First, words sharing roots share parameters:
run, running, runner and runs all draw on overlapping sets of character n-grams, and their vectors end up structurally related rather than independently learned.
Second, and more importantly for our purposes, FastText can produce a vector for any word, including ones it never saw during training, by summing the vectors of its constituent n-grams. The original paper’s evaluation on word-similarity tasks across nine languages found that the subword model outperformed Word2Vec on essentially every dataset, with the gap widening on morphologically complex languages such as German, Czech and Russian.
For plagiarism detection, the practical implications are direct.
Misspellings and obfuscation
A long-standing tactic for evading exact-matching detection is the deliberate introduction of small misspellings – recieve for receive, seperate for separate, occured for occurred – that preserve meaning and readability while breaking literal hash equality. Earlier embedding models could not help, because the misspelled form was simply absent from the vocabulary and had no vector. FastText, by construction, gives the misspelled form a vector that is very close to the correctly spelled one, because the two share most of their character n-grams. The detector sees a vector close to receive‘s vector, not a missing token, and the obfuscation fails.
The same logic applies to deliberate character-level obfuscation: zero-width characters, homoglyph substitution (Latin a replaced with Cyrillic а), or stylised respellings. None of these defeats a subword model as completely as they defeat exact matching, because the core character n-grams of the original word remain intact and dominate the resulting vector.
Morphological variation
The harder, more interesting case is genuine morphological variation. A copied passage that has been mechanically inflected – the experiment demonstrated rewritten as the experiments have demonstrated, or analysing the data as analysis of the data – produces tokens that Word2Vec and GloVe treat as unrelated. FastText sees the shared roots and produces vectors that remain close. For morphologically rich languages, the effect is dramatic. The tutorial example in Dive into Deep Learning notes that in French and Spanish many verbs have over forty inflected forms; in Finnish a single noun may have more than fifteen. A static word-level model has to learn each form independently. A subword model gets most of them for free.
Empirical performance on paraphrase
The most directly relevant evidence is from work by Chawla and colleagues, cited in the literature on paraphrase-detection evaluation, which found FastText to outperform both Word2Vec and GloVe on short-text paraphrase identification specifically – including on the kind of paraphrased student answers that approximate real plagiarism cases. The same finding appears, with caveats, in the Vrbanec and Meštrović comparison: FastText shows particular strength on the Webis Crowd Paraphrase Corpus, the dataset most representative of natural paraphrasing behaviour rather than constructed test cases. The general pattern across the empirical literature is that FastText’s edge over GloVe and Word2Vec is small on clean, well-spelled, in-vocabulary text and grows rapidly as the test data becomes noisier, more morphologically diverse, or more dependent on rare words.
Where each fits in a plagiarism pipeline
The classic embeddings are not, in practice, used in isolation. They sit in detection pipelines as the semantic-similarity layer – the component that runs after exact matching has surfaced its hits and looks for the paraphrased material that literal overlap missed. Each of the three has a characteristic role.
Word2Vec is the lightest-weight option and the one most often used as a default. Pre-trained vectors for English (Google News, 100 billion tokens, 300 dimensions, 3 million words) are widely available, well-understood, and small enough to load into memory on modest hardware. For a detection system whose primary load is exact matching and whose semantic layer needs only to be acceptable rather than exceptional, Word2Vec embeddings averaged over sentences and compared by cosine similarity are a reasonable choice.
GloVe is the option preferred when interpretability and stability matter. Because GloVe’s training objective is a deterministic least-squares fit to a global statistic, its vectors are reproducible across runs in a way that Word2Vec’s negative-sampling stochasticity does not guarantee. For systems whose outputs feed into adjudication processes – where being able to say “this similarity score would have been the same yesterday and will be the same tomorrow” carries weight – GloVe’s determinism is a real advantage. The Stanford GloVe project continues to release updated pre-trained vectors, with the most recent refresh in July 2025.
FastText is the option chosen when the input is dirty, multilingual, or technical. Misspellings, code-switching, technical vocabulary not present in standard pre-trained corpora, hashtags and product names – all of these are areas where the subword model’s ability to construct a vector for any string of characters pays for the additional storage and slightly slower inference. Facebook’s release of pre-trained FastText vectors for 157 languages on Wikipedia and Common Crawl made it the de facto choice for any detection system that needs to work across languages or against user-generated text. For detecting plagiarism in student writing in highly inflected languages – Turkish, Finnish, Arabic, Hindi, Tamil – FastText is rarely the wrong choice.
What none of them can do
A fair account of the classic embedding toolkit must be honest about the ceiling.
All three are static embeddings: each word gets exactly one vector, regardless of context. Bank in river bank and bank in investment bank collapse to the same point, and any paraphrase that turns on word-sense disambiguation is invisible to them. This was the limitation that contextual models – ELMo in 2018, BERT later the same year, and the entire transformer lineage that followed – were designed to address. For a detection system that needs to distinguish “the company filed for bankruptcy” from “the company filed its return” as semantically distinct in context, static embeddings are insufficient.
All three operate at the word or subword level, with no native notion of sentence or document structure. The standard workaround – averaging word vectors over a passage – discards word order entirely. The dog bit the man and the man bit the dog produce identical document vectors. For paraphrase that turns on syntactic transformation rather than lexical substitution, this is a serious blind spot, and it is the one that sentence-level encoders such as Sentence-BERT and the Universal Sentence Encoder were specifically designed to fill.
All three inherit the biases of their training corpora. The well-documented gender associations in Word2Vec and GloVe – doctor close to man, nurse close to woman – are not bugs but accurate reflections of the statistical patterns in the underlying text. For a detection system whose output is used to make consequential decisions about students or authors, the implication is that any threshold-based judgement based on these embeddings inherits those statistical biases as well. This is not a fatal objection – exact matching has its own biases, and so do humans – but it is a fact about the technique that the operator of any detection system needs to understand.
And, of course, none of them detect plagiarism on their own. They produce a similarity score. The score, combined with the threshold the system operator has chosen, surfaces a candidate match. The candidate match is then either confirmed by literal-overlap evidence at a lower granularity, or – if no literal overlap exists – passed to a human for the kind of judgement that no embedding model is in a position to make.
What this means for the field
The classic embedding toolkit occupies a specific and durable niche in the detection stack. It is the layer that catches paraphrase – synonym substitution, mild rephrasing, light morphological variation – at a cost that exact matching cannot reach and a speed that contextual transformers cannot match. Among the three, FastText is the option that has aged best, because the subword mechanism that made it useful in 2016 has become more useful as the volume of noisy, multilingual, AI-influenced text in the detection pipeline has grown. Word2Vec and GloVe remain in service primarily through their pre-trained vectors and through the inertia of pipelines built around them in the late 2010s; neither offers a capability that FastText does not also offer, and FastText’s handling of out-of-vocabulary and morphological cases is decisively better.
The deeper point is that the classic embeddings are not in competition with the methods that surround them. They complement exact matching at the layer above and contextual encoders at the layer below. A well-built detection pipeline runs all three in sequence, using each to catch what the previous layer missed, and reserving human judgement for the cases where evidence from multiple layers converges. The newer methods that get the attention are extensions of this same architecture, not replacements for it.
For builders, the practical conclusion is that investing in the embedding layer is still worthwhile – particularly in coverage (more languages, better pre-trained vectors for the relevant domain) and in subword robustness (FastText-style models, or the byte-pair encoding tokenisers used by modern transformers). For users, the conclusion is the same as elsewhere in this series: a similarity score from an embedding model is a prompt for closer inspection, not a finding. The model knows that two passages are close in vector space. It does not know why. That second question is the one detection software was never designed to answer alone.
GloVe and FastText showed that meaning lives in vector space – that two passages can match even when they share almost no surface words, and that subword models can handle morphologically rich languages a token-level system would fumble. Viper Plagiarism and AI Checker applies that thinking. Paraphrase-aware matching catches reworded passages pure n-gram overlap would miss. Cross-language detection spans 100+ languages. The scan runs against 60 trillion web pages, 16,000+ open-access journals, 1M+ internal documents and 20+ code repositories, alongside the highest-accuracy AI detector available – 99.12% across Claude, GPT-4, Gemini and the rest. Credits from 0.16p, no subscription. Run a plagiarism and AI scan →
References and further reading:
- Bojanowski, P., Grave, E., Joulin, A. and Mikolov, T. (2017) ‘Enriching word vectors with subword information’, Transactions of the Association for Computational Linguistics, 5, pp. 135–146.
- Chawla, A., Mulay, N., Bishnoi, V. and Dhama, G. (2021) ‘Improving the performance of paraphrase identification using FastText embeddings’. (As cited in the plagiarism-detection literature reviewing FastText’s paraphrase-detection performance.)
- Firth, J. R. (1957) ‘A synopsis of linguistic theory, 1930–1955’, in Studies in Linguistic Analysis. Oxford: Blackwell, pp. 1–32.
- Grave, E., Bojanowski, P., Gupta, P., Joulin, A. and Mikolov, T. (2018) ‘Learning word vectors for 157 languages’, in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
- Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013) ‘Efficient estimation of word representations in vector space’, arXiv preprint arXiv:1301.3781.
- Mikolov, T., Sutskever, I., Chen, K., Corrado, G. and Dean, J. (2013) ‘Distributed representations of words and phrases and their compositionality’, in Advances in Neural Information Processing Systems 26.
- Pennington, J., Socher, R. and Manning, C. D. (2014) ‘GloVe: Global vectors for word representation’, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha: Association for Computational Linguistics, pp. 1532–1543.
- Vrbanec, T. and Meštrović, A. (2021) ‘Corpus-based paraphrase detection experiments and review’, Mathematics, 9(24), 3231.