Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English version of the page.

Note: This page has been translated by MathWorks. Click here to see
To view all translated materials including this page, select Country from the country navigator on the bottom of this page.

Text Data Preparation

Import text data into MATLAB®and preprocess it for analysis.

Text Analytics Toolbox™ includes tools for processing raw text from sources such as equipment logs, news feeds, surveys, operator reports, and social media. Use these tools to extract text from popular file formats, preprocess raw text, extract individual words, convert text into numerical representations, and build statistical models.

Text Analytics Toolbox supports English language. Most Text Analytics Toolbox functions will work with text from other languages. For more information, see Language Support.

Functions

expand all

extractFileTextRead text from PDF, Microsoft Word, HTML, and plain text files
extractHTMLTextExtract text from HTML
readPDFFormDataRead data from PDF forms
writeTextDocumentWrite documents to text file
eraseTagsErase HTML and XML tags from text
eraseURLsErase HTTP and HTTPS URLs from text
erasePunctuationErase punctuation from text and documents
decodeHTMLEntitiesConvert HTML and XML entities into characters
normalizeWordsReduce words to common stems using the Porter stemmer
removeLongWordsRemove long words from documents or bag-of-words model
removeShortWordsRemove short words from documents or bag-of-words model
removeWordsRemove selected words from documents or bag-of-words model
stopWordsList of stop words
splitSentencesSplit text into sentences
tokenDetailsDetails of tokens in tokenized document array
addSentenceDetailsAdd sentence numbers to documents
topLevelDomainsList of top-level domains
abbreviationsTable of common abbreviations
upperConvert documents to uppercase
lowerConvert documents to lowercase
plusAppend documents
docfunApply function to words in documents
replaceFind and replace substrings in documents
regexprepReplace text in words of documents using regular expression
addDocumentAdd documents to bag-of-words or bag-of-n-grams model
removeDocumentRemove documents from bag-of-words or bag-of-n-grams model
removeEmptyDocumentsRemove empty documents from tokenized document array, bag-of-words model, or bag-of-n-grams model
removeInfrequentWordsRemove words with low counts from bag-of-words model
removeInfrequentNgramsRemove infrequently seen n-grams from bag-of-n-grams model
removeNgramsRemove n-grams from bag-of-n-grams model
topkwordsMost important words in bag-of-words model or LDA topic
topkngramsMost frequent n-grams
encodeEncode documents as matrix of word or n-gram counts
tfidfTerm Frequency–Inverse Document Frequency (tf-idf) matrix
joinCombine multiple bag-of-words or bag-of-n-grams models
contextSearch documents for word occurrences in context
doclengthLength of documents in document array
doc2cellConvert documents to cell array of string vectors
joinWordsConvert documents to string by joining words
stringConvert scalar document to string vector

Objects

expand all

tokenizedDocumentArray of tokenized documents
bagOfWordsBag-of-words model
bagOfNgramsBag-of-n-grams model

Topics

Extract Text Data from Files

This example shows how to extract the text data from text, HTML, Microsoft® Word, PDF, CSV, and Microsoft Excel® files and import it into MATLAB for analysis.

Prepare Text Data for Analysis

This example shows how to create a function which cleans and preprocesses text data for analysis.

Create Simple Text Model for Classification

This example shows how to train a simple text classifier on word frequency counts using a bag-of-words model.

Featured Examples

Was this topic helpful?