How do you do regular expressions at the character level?

Hello all,

I am trying to find words in a text with a set of rules and then extract them. I am looking for words with a certain structure. The words themselves have different lengths and letters.

for example:

term_1 = "TER";
term_2 = "ZTnE";
term_3 = "ZEnP";

...

Since I have a lot of terms, I tried to create a pattern with character-level rules. To do this, I split up the terms and always looked to see which character could occur at which position in the string.

For the simple example above:

1st place:

seg_1 = '[TZ]'

2nd place:

seg_2 = '[ET]'

3rd digit:

seg_3 = '[nR]'

4th digit:

seg_4 = '[EP]'

seg = seg_1 + seg_2 + seg_3 + seg_4;
result = extract(term_2, seg)

This now works for a term with the same length, but term_1 is not recognised.

Therefore, I have now made the following adjustment and declared the 4th seg as optionalPattern:

seg_4 = optionalPattern("E" | "P");

This is how the extraction works now. However, terms are now also extracted that skip an optionalPattern in the meantime.

Does anyone have any other ideas on how I can easily and safely include terms of different lengths?

Thank you very much!

Answer 1

[Walter Roberson]

"+" on character vectors is not a pattern operation.

seg_1 = '[TZ]'

seg_1 = '[TZ]'

seg_2 = '[ET]'

seg_2 = '[ET]'

seg = seg_1 + seg_2

seg = 1×4

182 153 174 186

char(seg)

ans = '¶□®º'

p_1 = characterListPattern('TZ')

p_1 = pattern
  Matching:

    characterListPattern("TZ")

p_2 = characterListPattern('ET')

p_2 = pattern
  Matching:

    characterListPattern("ET")

p_1 + p_2

ans = pattern
  Matching:

    characterListPattern("TZ") + characterListPattern("ET")

[Walter Roberson]

When you use extract and specify a string array or cell array of character vectors or character vector as the second parameter, then no pattern matching is done. Pattern matching is only done if you pass in a pattern array. But you cannot construct a regexp style text pattern like '[TZ][ET]' using most of the pattern operators: you either need to use operations like characterListPattern() or you have to use regexpPattern

... Or you can use strcat() of character vectors, or strjoin() of character vectors or string objects, or you can use the + operator between string objects, to construct either '[TZ][ET]' or "[TZ][ET]" and pass that as the second parameter to regexp() with the 'match' option . regexp() does not work with pattern array functions such as optionalPattern

[Tom Bernand]

Thank you!

You are exactly right here. I had used the regexpPattern() function and not directly the extract() function.

One question I still have is how to take spaces in the string into account?

For example

term_4 = 'ZT E'

should also be recognised.

[Walter Roberson]

In terms of your original patterns, 'ZT E' would require that seg_3 match space instead of [nR]

To allow space instead of one of the characters, include space in the [] if you are using regexp

seg_1 = "[ TZ]"
seg_2 = "[ ET]"
seg_3 = "[ nR]"
seg_4 = "[ EP]";
seg = seg_1 + seg_2 + seg_3 + seg_4;

result = regexp(term_2, seg, 'match');

If you want to more generally include "whitespace" (such as tab) then instead of putting a space in the [], use \s such as "[\sTZ]"

[Tom Bernand]

Perfect, that worked out great.

I must have been thinking in a complicated way.

Thank you very much!