textrankScores

Document scoring with TextRank algorithm

Syntax

scores = textrankScores(documents)

scores = textrankScores(bag)

Description

scores = textrankScores(documents) scores documents for importance according to pairwise similarity values using the TextRank algorithm. To compute similarities and importance scores, the function uses the BM25 and PageRank algorithms, respectively.

scores = textrankScores(bag) scores documents encoded by a bag-of-words or bag-of-n-grams model bag.

Examples

Importance of Documents

Create an array of tokenized documents.

str = [
    "the quick brown fox jumped over the lazy dog"
    "the fast brown fox jumped over the lazy dog"
    "the lazy dog sat there and did nothing"
    "the other animals sat there watching"];
documents = tokenizedDocument(str)

documents = 
  4x1 tokenizedDocument:

    9 tokens: the quick brown fox jumped over the lazy dog
    9 tokens: the fast brown fox jumped over the lazy dog
    8 tokens: the lazy dog sat there and did nothing
    6 tokens: the other animals sat there watching

Calculate the TextRank scores.

scores = textrankScores(documents);

Visualize the scores in a bar chart.

figure
bar(scores)
xlabel("Document")
ylabel("Score")
title("TextRank Scores")

Scores Using Bag-of-Words Model

Create a bag-of-words model from the text data in sonnets.csv.

filename = "sonnets.csv";
tbl = readtable(filename,'TextType','string');
textData = tbl.Sonnet;
documents = tokenizedDocument(textData);
bag = bagOfWords(documents)

bag = 
  bagOfWords with properties:

          Counts: [154x3527 double]
      Vocabulary: [1x3527 string]
        NumWords: 3527
    NumDocuments: 154

Calculate the TextRank scores.

scores = textrankScores(bag);

Visualize the scores in a bar chart.

figure
bar(scores)
xlabel("Document")
ylabel("Score")
title("TextRank Scores")

Input Arguments

documents — Input documents

tokenizedDocument array | string array of words | cell array of character vectors

Input documents, specified as a tokenizedDocument array, a string array of words, or a cell array of character vectors. If documents is not a tokenizedDocument array, then it must be a row vector representing a single document, where each element is a word. To specify multiple documents, use a tokenizedDocument array.

bag — Input model

bagOfWords object | bagOfNgrams object

Input bag-of-words or bag-of-n-grams model, specified as a bagOfWords object or a bagOfNgrams object. If bag is a bagOfNgrams object, then the function treats each n-gram as a single word.

Output Arguments

scores — TextRank scores

vector

TextRank scores, returned as a N-by-1 vector, where scores(i) corresponds to the score for the ith input document and N is the number of input documents.

References

[1] Mihalcea, Rada, and Paul Tarau. "Textrank: Bringing order into text." In Proceedings of the 2004 conference on empirical methods in natural language processing, pp. 404-411. 2004.