Runs the corpus-level work that does not depend on the number of topics — embedding every document and tokenizing it for term scoring — a single time, so that fitting many models (a [select_topics()] sweep, a manual loop over topic counts, or repeated [topics()] calls) reuses it instead of repeating it. Pass the returned object to [topics()] or [select_topics()] in place of the raw `text`.
Arguments
- text
Character vector containing one document per element.
- column
When `text` is a data frame, the name of the column holding the documents to model. Every other column is carried into `$documents`, and rows whose text is missing, blank, or a bibliographic placeholder are dropped. Leave `NULL` for a character vector.
- model
A loaded [sbert_model][load_model()], a pinned model name from [models()], or `NULL` for the default model. Ignored when `embeddings` are supplied.
- embeddings
Optional numeric matrix with one row per document; when supplied, no model is loaded or used.
- batch_size
Batch size passed to [encode()] when `model` is used.
- stop_words
Character vector excluded from topic terms. Use `character()` to disable stop-word filtering.
- min_token_length
Minimum Unicode character length for a topic token.
- stem
Whether to collapse inflected forms (for example `animals` and `animal`) onto a shared Porter stem before scoring, displaying the most frequent surface form of each stem. Requires the `SnowballC` package.
- cores
Number of forked worker processes used to tokenize documents for term scoring. Default `1` (serial). Values above one use `parallel::mclapply` on Unix-alikes and fall back to serial on Windows or for small corpora; the tokenization — and therefore every result — is identical regardless of the count. Ignored when a prepared [topic_corpus()] is supplied (its tokens are already computed).
- numbers
How to treat purely numeric tokens (years, counts) in topic terms. `"keep"` (default) keeps them; `"remove"` drops tokens made only of digits, while retaining alphanumerics such as `covid19`.
- roman_numerals
How to treat Roman-numeral tokens (chapter, section, and list markers such as `ii`, `iv`, `xii`). `"keep"` (default) keeps them; `"remove"` drops canonical Roman numerals up to 100. The bound is deliberate: larger Roman numerals collide with common abbreviations that are also valid numerals (`ml`, `mm`, `cc`, `ci`, `cv`), which are always kept. A few small numerals that are also words (`iv`, `vi`, `xl`) are removed when this is on.
- section_numbers
How to treat section, reference, and list numbering. `"keep"` (default) keeps it; `"remove"` strips, before tokenizing, both multi-level indices (`1.2.3`, `4.5.6.7` — three or more dot-separated groups) and enumeration or list markers (`1.`, `2.`, `figure 12.` — a standalone one- or two-digit number followed by a period). Decimals (`3.14`), four-digit years (`2020.`), hyphenated numbers (`covid-19.`), and larger counts are left untouched, so genuine values survive even when `numbers = "keep"`. These filter only the topic terms; the document text and its embedding are untouched. To clean the source text itself — list markers, reference noise, junk characters — before encoding, see [clean_corpus()].
Value
An object of class `sbert_topic_corpus`: a list with the prepared `text`, carried `metadata`, document `embeddings`, cached `token_lists`, `model` information, and the fixed tokenization `settings`.
Details
The results are byte-identical to calling [topics()] on the raw text: the corpus only *caches* the embedding and tokenization steps, it does not change them. The corpus fixes the embedding source (`model` or `embeddings`) and the tokenization settings (`stop_words`, `min_token_length`, `stem`, `numbers`, `roman_numerals`, `section_numbers`); [topics()] then refuses conflicting overrides for those, while per-model settings (`n_topics`, `n_terms`, `min_term_frequency`, `weighting`, `reduce_frequent_words`, `seeds`, ...) are still chosen per call.
Examples
text <- c(
"Cats chase mice", "Dogs chase balls",
"Stocks and bonds trade", "Markets price shares"
)
embeddings <- rbind(c(1, 0), c(0.9, 0.1), c(0, 1), c(0.1, 0.9))
corpus <- topic_corpus(text, embeddings = embeddings)
topics(corpus, n_topics = 2)$topics
#> topic label n_documents proportion withinss
#> 1 1 chase / balls / cats 2 0.5 0.006116265
#> 2 2 bonds / markets / price 2 0.5 0.006116265
select_topics(corpus, n_topics = 2:3)
#> <sbert_topic_sweep> 2 candidates, coherence measure: npmi
#> n_topics coherence topic_diversity explained
#> 2 -0.1000000 1.0000000 0.9931506
#> 3 0.3777778 0.9166667 0.9965753
#>
#> Fitted models retained: fitted(x, n_topics = 3)