Thread Subject: Words Segmentation

Hello guys......
I want to Extract words fro a text line,
then what is the best method using MATLAB???

All replies Are Appreciated...................

"Andrew Wayne" <ics2008_contact@yahoo.com> wrote in message
<fu1mnb$lld$1@fred.mathworks.com>...
> Hello guys......
> I want to Extract words fro a text line,
> then what is the best method using MATLAB???
>
> All replies Are Appreciated...................

Many ways. For example assuming only spaces, this one is
reasonably short to type
str = 'word1 word2 word3';
words = regexp(str,' ','split')

"Pekka " <pekka.nospam.kumpulainen@tut.please.fi> wrote in
message <fu26l9$d0p$1@fred.mathworks.com>...
> "Andrew Wayne" <ics2008_contact@yahoo.com> wrote in message
> <fu1mnb$lld$1@fred.mathworks.com>...
> > Hello guys......
> > I want to Extract words fro a text line,
> > then what is the best method using MATLAB???
> >
> > All replies Are Appreciated...................
>
> Many ways. For example assuming only spaces, this one is
> reasonably short to type
> str = 'word1 word2 word3';
> words = regexp(str,' ','split')
>
NOOOOO!!!! this is very slow!!!!

check out "split" or "split2" on the MATLAB file exchange.
The implementation in split2 uses textscan which is much
much faster than "strtok" or matlab regexp's.

of course if you only want to parse a few lines then who
cares about the speed? But if you're going to parse 100000
lines then textscan is your ONLY option!

You also need to be careful about typecasting strings to
numbers via str2double or even slower str2num. These
functions are very slow. So if you can use textscan to
parse strings to strings and numbers to numbers that'll save
you alot of time!


Example:

    cell_array = textscan(string,'%s');
OR
    cell_array =textscan(string,'%s','delimiter',':');

    cell_array =textscan(string,'%s','delimiter','/');

    ...

    %reshape
    cell_array = cell_array{1};

In article <fu1mnb$lld$1@fred.mathworks.com>,
Andrew Wayne <ics2008_contact@yahoo.com> wrote:
>Hello guys......
>I want to Extract words fro a text line,
>then what is the best method using MATLAB???

>All replies Are Appreciated...................

What is a "word" ?

The below is an example I used a week ago for someone wanting
to separate out paragraphs. Which are the words in this?

  Mr. Todd E. Jones gave $3000. (!) in nickels, dimes, etc. to his
  No. 1 son at 4 7th Ave. N., NY. NY. USA., who exclaimed "What joy!
  Now I can buy 3 lbs. of St. Tropiz bananas... or can I?!"


--
  "Nothing recedes like success." -- Walter Winchell

roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) wrote in
message <fu2jlf$aia$1@canopus.cc.umanitoba.ca>...
> In article <fu1mnb$lld$1@fred.mathworks.com>,
> Andrew Wayne <ics2008_contact@yahoo.com> wrote:
> >Hello guys......
> >I want to Extract words fro a text line,
> >then what is the best method using MATLAB???
>
> >All replies Are Appreciated...................
>
> What is a "word" ?
>
> The below is an example I used a week ago for someone wanting
> to separate out paragraphs. Which are the words in this?
>
> Mr. Todd E. Jones gave $3000. (!) in nickels, dimes,
etc. to his
> No. 1 son at 4 7th Ave. N., NY. NY. USA., who exclaimed
"What joy!
> Now I can buy 3 lbs. of St. Tropiz bananas... or can I?!"
>
>
> --
> "Nothing recedes like success." -- Walter
Winchell

words are [a-zA-Z0-9] That's pretty simple yeah.

or more specifically \w+ in perl-speak

In article <fu2pcr$150$1@fred.mathworks.com>,
Abel Brown <brown.2179@osu.edu> wrote:
>roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) wrote in
>message <fu2jlf$aia$1@canopus.cc.umanitoba.ca>...

>> What is a "word" ?

>words are [a-zA-Z0-9] That's pretty simple yeah.

>or more specifically \w+ in perl-speak

It isn't that simple in English. In English, there are
contractions, possessives, abbreviations, and some symbols,
all treated as words.

For example, for "its'", "it's", "etc.", and "&" the word
is everything inside the double-quote marks. The apostrophes
in "its'" and "it's" are not punctuation: they are part of the word.
Similarily, the period at the end of "etc." is part of the word.

If you are reading a document (especially a financial document)
and it has a capital N and a slightly-raised small o, then that
is the word "number" (or, depending on the context, "numbering").
--
   "Any sufficiently advanced bug is indistinguishable from a feature."
   -- Rich Kulawiec

roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) wrote in
message <fu2q56$jl7$1@canopus.cc.umanitoba.ca>...
> In article <fu2pcr$150$1@fred.mathworks.com>,
> Abel Brown <brown.2179@osu.edu> wrote:
> >roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) wrote in
> >message <fu2jlf$aia$1@canopus.cc.umanitoba.ca>...
>
> >> What is a "word" ?
>
> >words are [a-zA-Z0-9] That's pretty simple yeah.
>
> >or more specifically \w+ in perl-speak
>
> It isn't that simple in English. In English, there are
> contractions, possessives, abbreviations, and some symbols,
> all treated as words.
>
> For example, for "its'", "it's", "etc.", and "&" the word
> is everything inside the double-quote marks. The apostrophes
> in "its'" and "it's" are not punctuation: they are part of
the word.
> Similarily, the period at the end of "etc." is part of the
word.
>
> If you are reading a document (especially a financial
document)
> and it has a capital N and a slightly-raised small o, then
that
> is the word "number" (or, depending on the context,
"numbering").
> --
> "Any sufficiently advanced bug is indistinguishable
from a feature."
> -- Rich Kulawiec

then use perl to pre-process the data before using matlab.
You can even call your perl from within matlab (im sure you
already know :).

I understand what you're talking about. It's just the
original poster had a very basic question. Hence, a simple
answer ...

In article <fu2qog$g5t$1@fred.mathworks.com>,
Abel Brown <brown.2179@osu.edu> wrote:

>I understand what you're talking about. It's just the
>original poster had a very basic question. Hence, a simple
>answer ...

We don't know what the original poster meant by "word", so clarification
had to be sought. In this newsgroup, we -often- get "simple" questions
that are not so simple after all, because the poster has a particular
meaning of some phrase in mind and doesn't realize that the phrase
can mean a number of other things as well. It is not uncommon here
for people to change their questions and want to do something different
than they originally wanted, once alternative shades of meaning
are pointed out to them.

--
  "To all, to each! a fair good-night,
   And pleasing dreams, and slumbers light" -- Sir Walter Scott

roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) wrote in
message <fu2ra5$lab$1@canopus.cc.umanitoba.ca>...
> In article <fu2qog$g5t$1@fred.mathworks.com>,
> Abel Brown <brown.2179@osu.edu> wrote:
>
> >I understand what you're talking about. It's just the
> >original poster had a very basic question. Hence, a
simple
> >answer ...
>
> We don't know what the original poster meant by "word",
so clarification
> had to be sought. In this newsgroup, we -often-
get "simple" questions
> that are not so simple after all, because the poster has
a particular
> meaning of some phrase in mind and doesn't realize that
the phrase
> can mean a number of other things as well. It is not
uncommon here
> for people to change their questions and want to do
something different
> than they originally wanted, once alternative shades of
meaning
> are pointed out to them.
>
> --
> "To all, to each! a fair good-night,
> And pleasing dreams, and slumbers light" -- Sir
Walter Scott
I meant by word that for example we have the following text
line:
I Want tomorrow to go for shopping as usual

Then I want to extract each word so that I get:
I
Want
tomorrow
to
go.....etc

"Andrew Wayne" <ics2008_contact@yahoo.com> wrote in message
<fu352f$in$1@fred.mathworks.com>...
> roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) wrote in
> message <fu2ra5$lab$1@canopus.cc.umanitoba.ca>...
> > In article <fu2qog$g5t$1@fred.mathworks.com>,
> > Abel Brown <brown.2179@osu.edu> wrote:
> >
> > >I understand what you're talking about. It's just the
> > >original poster had a very basic question. Hence, a
> simple
> > >answer ...
> >
> > We don't know what the original poster meant by "word",
> so clarification
> > had to be sought. In this newsgroup, we -often-
> get "simple" questions
> > that are not so simple after all, because the poster has
> a particular
> > meaning of some phrase in mind and doesn't realize that
> the phrase
> > can mean a number of other things as well. It is not
> uncommon here
> > for people to change their questions and want to do
> something different
> > than they originally wanted, once alternative shades of
> meaning
> > are pointed out to them.
> >
> > --
> > "To all, to each! a fair good-night,
> > And pleasing dreams, and slumbers light" -- Sir
> Walter Scott
> I meant by word that for example we have the following text
> line:
> I Want tomorrow to go for shopping as usual
>
> Then I want to extract each word so that I get:
> I
> Want
> tomorrow
> to
> go.....etc

%do this
cell_array = textscan(string,'%s');

hi,

Did you find any way or any code to segment the text?

Thanks,
Sara


"Mohammed Ahmed" wrote in message <fu1mnb$lld$1@fred.mathworks.com>...
> Hello guys......
> I want to Extract words fro a text line,
> then what is the best method using MATLAB???
>
> All replies Are Appreciated...................