Sentence parsing can be done one word at a time using strtok. However, sometimes it is useful to (efficiently) extract all words into a cell array in one function call. The function allwords.m does exactly this.
Spaces, white space (tabs), carriage returns, and punctuation characters are all valid separator characters by default. In this example, I had a period at the end, as well as multiple spaces between some words.
str = 'The quick brown fox jumped over the lazy dog.';
words = allwords(str)
'The' 'quick' 'brown' 'fox' 'jumped' 'over' 'the' 'lazy' 'dog'
This utility can also work on any integer vector. The default separators for numeric vectors are [-inf inf NaN], but you can assign any separators you desire. Here, parse a string of integers, with only NaN elements as the separator.
str = [1 2 4 2 inf 3 3 5 nan 4 6 5];
words = allwords(str,nan);
1 2 4 2 Inf 3 3 5
4 6 5
Finally, allwords is efficient. For example, on a random numeric string of length 1e6, allwords parses it into over 90000 distinct "words" in less than 0.5 seconds.
str = round(rand(1,1000000)*10);
words = allwords(str,[0 10]);
Elapsed time is 0.455194 seconds.
There were over 90000 different words that were extracted
The longest word had length 104.
John D'Errico (2023). allwords (https://www.mathworks.com/matlabcentral/fileexchange/27184-allwords), MATLAB Central File Exchange. Retrieved .
MATLAB Release Compatibility
Platform CompatibilityWindows macOS Linux
Inspired by: wordcount
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!
Speed enhancement for character strings, plus I added a reference to the wordcount function.