Thread Subject: [OT] What information can be gleaned from a usenet post?

I cannot find the thread, but there was one in CSSM by someone who's
identity was being hijacked. He posted a URL to check the identity
of the poster.

What info can be gotten about a post?

Can I download CSSM for offline viewing?

Can I access CSSM effectively using URLREAD to download CSSM history?

TIA

Mike.

Michael Robbins wrote:
>
>
> I cannot find the thread, but there was one in CSSM by someone
> who's
> identity was being hijacked. He posted a URL to check the identity
> of the poster.
>
> What info can be gotten about a post?
>
> Can I download CSSM for offline viewing?
>
> Can I access CSSM effectively using URLREAD to download CSSM
> history?
>
> TIA
>
> Mike.

Hi Mike,

To download the complete archives of CSSM, you can use wget to mirror
the Archives at <http://www.mathforum.com/epigone/comp.soft-sys.matlab>
and then either browse the html pages or grep them.

I'm not at my computer which has the command, but later I will post
the wget command I use to mirror the site.

You can get wget from either 1) Cygwin or 2) gnuwin32.sf.net

The install at gnuwin32 also comes with extensive documentation.

Stephen

> To download the complete archives of CSSM, you can use wget to
> mirror
> the Archives at <http://www.mathforum.com/epigone/comp.soft-sys.matlab>
> and then either browse the html pages or grep them.

Thanks, I will give that a try.

As for the contents of a post, I found this website:

 <http://www.geocities.com/ski_usenet/article.html#nntp>

> To download the complete archives of CSSM, you can use wget to
> mirror
> the Archives at <http://www.mathforum.com/epigone/comp.soft-sys.matlab>
> and then either browse the html pages or grep them.

Hi. Thanks, I downloaded WackGet, a GUI windows version of wget, but
it seems to only have dowloaded the titles of the threads. It
doesn't seem to drill down into the actual posts.

Michael Robbins wrote:
>
>
>> To download the complete archives of CSSM, you can use wget to
>> mirror
>> the Archives at <http://www.mathforum.com/epigone/comp.soft-sys.matlab>
>> and then either browse the html pages or grep them.
>
> Hi. Thanks, I downloaded WackGet, a GUI windows version of wget,
> but
> it seems to only have dowloaded the titles of the threads. It
> doesn't seem to drill down into the actual posts.

 <http://www.httrack.com/>
s another free option. I used wget from <http://gnuwin32.sf.net>
nd mine was able to get the articles. I didn't keep up with mine
since I've been pretty happy with the archive the mathworks has.

It sounds like you might need to change the depth level.

Stephen

StephenLL wrote:
>
>
> Michael Robbins wrote:
>>
>>
>>> To download the complete archives of CSSM, you can use wget
> to
>>> mirror
>>> the Archives at <http://www.mathforum.com/epigone/comp.soft-sys.matlab>
>>> and then either browse the html pages or grep them.
>>
>> Hi. Thanks, I downloaded WackGet, a GUI windows version of
wget,
>> but
>> it seems to only have dowloaded the titles of the threads. It
>> doesn't seem to drill down into the actual posts.
>
> <http://www.httrack.com/>
> s another free option. I used wget from <http://gnuwin32.sf.net>
> nd mine was able to get the articles. I didn't keep up with mine
> since I've been pretty happy with the archive the mathworks has.
>
> It sounds like you might need to change the depth level.

HTTrack worked well. It was easy to install and easy to use.

Thank you.

I have one remaining issue with this process. It takes quite a bit
of time to download the CSSM archive and I'd like to update it daily.
 Is there a way for me to update my downloaded data, or do I have to
re-download the whole archive?
>
> Stephen

Michael Robbins wrote:
>
>
> StephenLL wrote:
>>
>>
>> Michael Robbins wrote:
>>>
>>>
>>>> To download the complete archives of CSSM, you can use
> wget
>> to
>>>> mirror
>>>> the Archives at <http://www.mathforum.com/epigone/comp.soft-sys.matlab>
>>>> and then either browse the html pages or grep them.
>>>
>>> Hi. Thanks, I downloaded WackGet, a GUI windows version of
> wget,
>>> but
>>> it seems to only have dowloaded the titles of the threads.
> It
>>> doesn't seem to drill down into the actual posts.
>>
>> <http://www.httrack.com/>
>> s another free option. I used wget from <http://gnuwin32.sf.net>
>> nd mine was able to get the articles. I didn't keep up with
mine
>> since I've been pretty happy with the archive the mathworks
has.
>>
>> It sounds like you might need to change the depth level.
>
> HTTrack worked well. It was easy to install and easy to use.
>
> Thank you.
>
> I have one remaining issue with this process. It takes quite a bit
> of time to download the CSSM archive and I'd like to update it
> daily.
> Is there a way for me to update my downloaded data, or do I have
> to
> re-download the whole archive?
>>
>> Stephen

There is an option to Update an Existing download. I don't have the
software in front of me, but I think that is an option when you open
the software. I think the help talks about it.

Stephen

> There is an option to Update an Existing download. I don't have
> the
> software in front of me, but I think that is an option when you
> open
> the software. I think the help talks about it.
>
> Stephen

Ah, there it is. Thank you again!

>> There is an option to Update an Existing download. I don't
have
>> the
>> software in front of me, but I think that is an option when you
>> open
>> the software. I think the help talks about it.
>>
>> Stephen
>
> Ah, there it is. Thank you again!

Hmmmm. It still takes many hours.

I think the problem is that the websites (both mathforum.com and
mathworks.com) that display CSSM use a FIFO method of displaying the
threads. That is to say, today, there may be 90 threads and 45 per
page so threads 90-46 are on page one, and 45-1 are on page 2.
Tomorrow, there may be a total of 100 so 100-56 will be on page 1,
55-11 on page 2, 10-1 on page three. That way all the pages must be
updated, instead of just the new threads.

Do you know of another website or download tool that will store
threads by name or use a LIFO storage method so only new additions
will be updated?

Hi Michael,

In the MATLAB Central CSSM reader, you can also click on 'view
original format' link to view the IP address where the post
originated. This can help in investigating the identity of the
poster.

-Min

The MATLAB Central Team
The MathWorks, Inc.

Michael Robbins wrote:
>
>
>>> There is an option to Update an Existing download. I don't
> have
>>> the
>>> software in front of me, but I think that is an option when
> you
>>> open
>>> the software. I think the help talks about it.
>>>
>>> Stephen
>>
>> Ah, there it is. Thank you again!
>
> Hmmmm. It still takes many hours.
>
> I think the problem is that the websites (both mathforum.com and
> mathworks.com) that display CSSM use a FIFO method of displaying
> the
> threads. That is to say, today, there may be 90 threads and 45 per
> page so threads 90-46 are on page one, and 45-1 are on page 2.
> Tomorrow, there may be a total of 100 so 100-56 will be on page 1,
> 55-11 on page 2, 10-1 on page three. That way all the pages must
> be
> updated, instead of just the new threads.
>
> Do you know of another website or download tool that will store
> threads by name or use a LIFO storage method so only new additions
> will be updated?

As far as downloading CSSM for offline reading, I recommend using a
newsreader that has this as a feature rather than trying to crawl
some website. Most Usenet newsreaders will automatically download
the current articles to your local PC or handheld, keep track of
which articles you've read, and present it all in a nice interface.

Sometimes we're hampered as to what we can do with our newsgroup on
MATLAB Central because we're riding on top of Usenet. Providing
alternative access to the content, however, is one of the big
advantages.

If this is a repeated post, I'm sorry. I posted last night but I
don't see it.

I'd love to use a newsreader but I cannot, since I am always behind a
firewall or on a public network, with no personal ISP.

Maybe The MathWorks could provide a LIFO copy of CSSM so people like
me could download updates, using WGET or some similar tool, without
having to download the entire history of CSSM. MP?

Michael, we don't advertise it much because we'd rather have people
using the web interface, especially new people, but you can access
CSSM publicly via NNTP on newsreader.mathworks.com. If you have a
newsreader configured properly, this URL should do the trick:

news://newsreader.mathworks.com/comp.soft-sys.matlab

<snip>
> you can access
> CSSM publicly via NNTP on newsreader.mathworks.com.


Fan-tastic! This is really neat. Thank you.

"Michael Robbins" <michaelNOrobbinsSPAMusenet@yahoo.com> wrote in message
news:94DEB672F0E4E98FFC12ABFB84E2CFBB@in.webx.raydaftYaTP...
> <snip>
>> you can access
>> CSSM publicly via NNTP on newsreader.mathworks.com.
>
>
> Fan-tastic! This is really neat. Thank you.


I'm using MS Outlook Newsreader (2003) and I only see 300 entries (all from
today). I changed the "download headers" option from 300 to the maximum
(1000) but it doesn't change the number downloaded. Is that due to your
website (if so maybe you know another NNTP I can use) or is it Outlook? I'm
going to try to find another newsreader while I wait for your answer but I'd
rather use Outlook.

Michael Robbins <michaelNOrobbinsSPAMusenet@yahoo.com> wrote:
>
>I'm using MS Outlook Newsreader (2003) and I only see 300 entries (all from
>today). I changed the "download headers" option from 300 to the maximum
>(1000) but it doesn't change the number downloaded. Is that due to your
>website (if so maybe you know another NNTP I can use) or is it Outlook? I'm
>going to try to find another newsreader while I wait for your answer but I'd
>rather use Outlook.

The server probably only keeps hhat many messages. You can try using your
local server which may have better retention times.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

>>I'm using MS Outlook Newsreader (2003) and I only see 300 entries (all
>>from today). I changed the "download headers" option from 300 to the
>>maximum (1000) but it doesn't change the number downloaded. Is that
>>due to your website (if so maybe you know another NNTP I can use) or
>>is it Outlook? I'm going to try to find another newsreader while I
>>wait for your answer but I'd rather use Outlook.
>
> The server probably only keeps hhat many messages. You can try using
> your local server which may have better retention times.
> --scott

I downloaded Xnews and I can see articles back to march (about 18143
posts, I think).

Matthew Simoneau, can you tell me how far back your archive goes?

In article <Xns9600C9D59F5C0michaelNOrobbinsSPAM@64.124.46.160>, Michael Robbins <michaelNOrobbinsSPAMusenet@yahoo.com> wrote:
>>>I'm using MS Outlook Newsreader (2003) and I only see 300 entries (all
>>>from today). I changed the "download headers" option from 300 to the
>>>maximum (1000) but it doesn't change the number downloaded. Is that
>>>due to your website (if so maybe you know another NNTP I can use) or
>>>is it Outlook? I'm going to try to find another newsreader while I
>>>wait for your answer but I'd rather use Outlook.
>>
>> The server probably only keeps hhat many messages. You can try using
>> your local server which may have better retention times.
>> --scott
>
>I downloaded Xnews and I can see articles back to march (about 18143
>posts, I think).
>
>Matthew Simoneau, can you tell me how far back your archive goes?


I'm trying NewsXpress now, and it seems to be able to go back to 3/04, but I'd
like to go back much farther.

Still, this is better than sharks with friggin' laser beams on their heads.

In my version of Outlook Express, I can uncheck the "download
headers: 300" option (rather than changing the number) to get them
all. When I do this, it looks like I get all the messages back to
sometime in October, then they start to trickle off. I don't know
what policy this server uses to delete the old the messages it offers
via NNTP. It looks like you get at least a few months, which should
be plenty.

Matthew Simoneau wrote:

> In my version of Outlook Express, I can uncheck the "download
> headers: 300" option (rather than changing the number) to get them
> all. When I do this, it looks like I get all the messages back to
> sometime in October, then they start to trickle off. I don't know
> what policy this server uses to delete the old the messages it offers
> via NNTP. It looks like you get at least a few months, which should
> be plenty.

I've tried the following newsreaders and it seems most can retrieve
messages going back to 3/04. I think it's due to some arbitrary
restriction on the number of messages or threads and not the server but
I'm not yet sure.

XNews (xnews.zip)
NewsXpress (nx201.zip)
Forte Agent (agentenu200-652.exe,agentenu-spelleng200-100.exe)
NewsBin (nb432.exe)
XanaNews (xananews_setup.exe)

On Thu, 17 Feb 2005 21:07:03 -0500, Michael Robbins wrote:

> Matthew Simoneau wrote:
>
>> In my version of Outlook Express, I can uncheck the "download
>> headers: 300" option (rather than changing the number) to get them
>> all. When I do this, it looks like I get all the messages back to
>> sometime in October, then they start to trickle off. I don't know
>> what policy this server uses to delete the old the messages it offers
>> via NNTP. It looks like you get at least a few months, which should
>> be plenty.
<snip>

I am trying 40tude now (4d2b38.exe) and it goes back about the same amount
of time (or perhaps the same number of posts). I think the earliest post
is 3/8/04 6:42:18 AM. The status window says the newsgroup has 65,630
messages on the server. I think there's 18,158 threads.

I guess that six of the seven newsreaders I've tried, with MS Outlook as
the exception, downloaded all the messages on the server.

HTTrack was able to download the full history from
mathforum.org\epigone\comp.soft-sys.matlab which is fine. I can look at
that for my deep inquiries and update it periodically.

Does anyone know of a NNTP for the mathforum server?

In article <3824F360A3D4E00F94DB486847F185BF@in.webx.raydaftYaTP>,
 "Michael Robbins" <michaelNOrobbinsSPAMusenet@yahoo.com> wrote:

> Matthew Simoneau wrote:
>
> > In my version of Outlook Express, I can uncheck the "download
> > headers: 300" option (rather than changing the number) to get them
> > all. When I do this, it looks like I get all the messages back to
> > sometime in October, then they start to trickle off. I don't know
> > what policy this server uses to delete the old the messages it offers
> > via NNTP. It looks like you get at least a few months, which should
> > be plenty.
>
> I've tried the following newsreaders and it seems most can retrieve
> messages going back to 3/04. I think it's due to some arbitrary
> restriction on the number of messages or threads and not the server but
> I'm not yet sure.
>
> XNews (xnews.zip)
> NewsXpress (nx201.zip)
> Forte Agent (agentenu200-652.exe,agentenu-spelleng200-100.exe)
> NewsBin (nb432.exe)
> XanaNews (xananews_setup.exe)

Michael,

It doesn't matter which newsreader you use. What matters is the news
retention policy of the particular news server to which you are
connected. It is possible for the administrator of the news server to
set a policy according to several criteria. Some possibilities:

1. Retain articles for a specified number of days
2. Retain a certain number of articles.
3. Retain articles to occupy a certain amount of disk space.

It all depends on the news server software and the policy desired. It
is possible for the newsreader to further limit the articles that it
will fetch and display, but no newsreader can display articles that
aren't on the server.

News servers are not intended to archive articles forever, but just for
a reasonable time to give people a chance to read them.

I read news daily, but I don't bother to archive any articles myself --
I just depend on Google if I want to look up something old.

I hope this clears things up a bit.

--
Doug Schwarz
dmschwarz&urgrad,rochester,edu
Make obvious changes to get real email address.

<snip>
> I read news daily, but I don't bother to archive any articles
> myself --
> I just depend on Google if I want to look up something old.
>
> I hope this clears things up a bit.

Thanks. Please ignore the following if you like, it's just an
explaination for others who may have similar problems:

1) I DON'T HAVE LIVE ACCESS

My issue is that I don't have access to CSSM most of the time,
especially when I need it.

2) AND WHEN I DO, IT'S RARELY USEFUL

And when I have access to CSSM, I don't have access to MATLAB which
is why my posts have been so poor lately.

3) I'D LIKE TO USE WGET OR HTTRACK

I was hoping to download the archive to reference offline, but the
LIFO presentation of the web based newsreaders don't allow me to
update new messages; I must re-download the entire archive to update
which takes a very long time.

It was considering that I could download mathforum.org once a week,
and download newsreader.mathworks.com daily because it holds less
data and would be somewhat quicker. I'd still have to download the
entire year of CSSM from TMW each day which takes much longer than
newsreaders and is much less convenient.

I'm also concerned that newsreader.mathworks.com has links to FEX and
the Link Exchange, etc. WGet and HTTrack may crawl all over
mathworks.com, not just CSSM, because they might not know the
difference between CSSM and the rest.

3) SO I'M LOOKING AT NEWSREADERS

But most will only download the subject lines. XNews seems to be the
exception, which is good. Outlook can also do this but can only
store about a day's worth of posts.

The only problem here is that TMW's server only keeps about a year of
posts which isn't always enough. It *is* enough if I keep a copy of
mathforum's archive handy.

So, I guess I'll just do two searches when I need to dig deep. First
I'll search newsreader.mathworks.net via my daily saved copy, using
XNews, and then I'll search the slightly oudated copy of
mathforum.org that I save less frequently with HTTtrack.

<snip>

This is [OT].^2

I can't seem to find a way to get Xnews to display the threads sorted
by *last* post, like newsreader.mathworks.com does. I like Xnews
because it will store both the subject and body of the message for
offline viewing.

I tried to post on a few newsreader newsgroups but google won't
display my posts for some reason, and the research I did only shows
older requests and no solutions.

Does anyone know how to do this? Or, another NNTP newsreader that
will store entire messages and sort this way?

Michael Robbins wrote:
> <snip>
[...]
> Does anyone know how to do this? Or, another NNTP newsreader that
> will store entire messages and sort this way?

have a look at mozilla 1.7.5:

http://www.mozilla.org

No one wrote:
> Michael Robbins wrote:
>
>><snip>
>
> [...]
>
>>Does anyone know how to do this? Or, another NNTP newsreader that
>>will store entire messages and sort this way?
>
>
> have a look at mozilla 1.7.5:
>
> http://www.mozilla.org

This looks good!

There is a new adult search engine out there
http://dxstar.com
dxStar lets you seach through image and movie galleries, keeps an index
of popular adult web sites.
Search expirience can be customized with different view options.
Let me know what you think.