Text Analytics Toolbox™ includes tools for processing raw text from sources such as equipment logs, news feeds, surveys, operator reports, and social media. Use these tools to extract text from popular file formats, preprocess raw text, extract individual words or multiword phrases (n-grams), convert text into numerical representations, and build statistical models. For an example showing how to get started, see Prepare Text Data for Analysis.
Text Analytics Toolbox supports the languages English and Japanese. Most Text Analytics Toolbox functions work with text from other languages. For more information, see Language Support.
|Array of tokenized documents for text analysis|
|Erase punctuation from text and documents|
|Erase HTML and XML tags from text|
|Erase HTTP and HTTPS URLs from text|
|Remove stop words from documents|
|Remove short words from documents or bag-of-words model|
|Remove long words from documents or bag-of-words model|
|Remove selected words from documents or bag-of-words model|
|Stem or lemmatize words|
|List of stop words|
|Convert HTML and XML entities into characters|
|Convert documents to lowercase|
|Convert documents to uppercase|
|Search documents for word occurrences in context|
|Details of tokens in tokenized document array|
|Add sentence numbers to documents|
|Add part-of-speech tags to documents|
|Add lemma forms of tokens to documents|
|Add language identifiers to documents|
|Add token type details to documents|
|Split text into sentences|
|Detect language of text|
|Table of common abbreviations|
|List of top-level domains|
|Add documents to bag-of-words or bag-of-n-grams model|
|Remove documents from bag-of-words or bag-of-n-grams model|
|Remove words with low counts from bag-of-words model|
|Remove infrequently seen n-grams from bag-of-n-grams model|
|Remove n-grams from bag-of-n-grams model|
|Remove empty documents from tokenized document array, bag-of-words model, or bag-of-n-grams model|
|Most important words in bag-of-words model or LDA topic|
|Most frequent n-grams|
|Encode documents as matrix of word or n-gram counts|
|Term Frequency–Inverse Document Frequency (tf-idf) matrix|
|Combine multiple bag-of-words or bag-of-n-grams models|
|Apply function to words in documents|
|Find and replace substrings in documents|
|Replace text in words of documents using regular expression|
|Length of documents in document array|
|Convert documents to cell array of string vectors|
|Convert documents to string by joining words|
|Convert scalar document to string vector|
This example shows how to extract the text data from text, HTML, Microsoft® Word, PDF, CSV, and Microsoft Excel® files and import it into MATLAB® for analysis.
This example shows how to create a function which cleans and preprocesses text data for analysis.
This example shows how to parse HTML code and extract the text content from particular elements.
This example shows how to analyze text data containing emojis.