Thread Subject: urlread

Hi people,

sorry for perhaps a very stupid question , but if i write :

s=urlread('http://securities.stanford.edu/1010/SJK98');

it give me an error :

??? Error using ==> urlread
Error downloading URL.

Error in ==> creates_the_database at 7
s=urlread('http://securities.stanford.edu/1010/SJK98');

Y ???

<snip>
> s=urlread('http://securities.stanford.edu/1010/SJK98');
> it give me an error :
<snip>

That is a good URL. I don't have MATLAB with me now, but try:

[s,status]=urlread('http://securities.stanford.edu/1010/SJK98')

and see what it gets you. You might also try tracing it.

Unfortunately, I'd love to use URLREAD more often but It is deficient
for my purposes (secure sockets). I hope TMW improves it soon.

If you look at some previous threads (search for URLREAD ROBBINS) you
will find a Java script I posted that should do the trick.

Incidentally, once you do get it working, REGEXP is a fantastic way
to parse the text.

I tried your code snippet,

s=urlread('http://securities.stanford.edu/1010/SJK98')

and it works fine for me. I suspect you are behind a firewall. You
can use the "Web" panel in your preferences to set your proxy
settings.

Hi, thanks all !!! yes, i realised i was behind the firewall very
soon after have posted this msg.

thanks again for the replys . Alex.

Matthew,

I wonder if you know how to set the proxy preferences for user and
password authentication? I believe something like <http://username>:password@proxyhost.ac.uk
hould work but doesn't. Perhaps I have the syntax wrong?

Thanks,
Neil

Matthew Simoneau wrote:
>
>
> I tried your code snippet,
>
> s=urlread('http://securities.stanford.edu/1010/SJK98')
>
> and it works fine for me. I suspect you are behind a firewall.
> You
> can use the "Web" panel in your preferences to set your proxy
> settings.

> password authentication? I believe something like <http://username>:password@proxyhost.ac.uk
> hould work but doesn't. Perhaps I have the syntax wrong?

That is not sufficient for use with URLREAD.

Michael,

I'm hoping for a solution for anything behind a proxy server, not
just URLREAD, like the datafeed toolbox for instance.

Can you say why this is not sufficient for use with URLREAD in
particular?

Cheers, Neil

Michael Robbins wrote:
>
>
>> password authentication? I believe something like <http://username>:password@proxyhost.ac.uk
>> should work but doesn't. Perhaps I have the syntax wrong?
>
> That is not sufficient for use with URLREAD.

> I'm hoping for a solution for anything behind a proxy server, not
> just URLREAD, like the datafeed toolbox for instance.
>
> Can you say why this is not sufficient for use with URLREAD in
> particular?

I posted a Java script a few months ago that solves that problem. I
was asking for help converting it to MATLAB but I didn't find a
solution. You can use the Java, though.

I'm not quite sure what you're looking for. When you say "proxy
preferences for user and password authentication", do you mean that
your local proxy requires a username and proxy? If so, I don't think
I've ever heard of this and don't know how to help. Maybe you didn't
mean to use the term "proxy" at all and just meant that the site
you're trying to connect to requires a username and password. If so,
there are a few different ways to transmit them and I'd need to know
more about the website.

Matthew,

No, nothing that obscure. Sorry, I'll say it properly:

The Matlab Web Preferences dialog mentioned earlier in the thread
lets one specify a proxy server to connect to the internet, and
provides two fields: Proxy host, and Proxy port. The proxy server at
my institute also requires a Proxy username, and Proxy password. (I
didn't think this was uncommon, but would like to hear if it is.)

In previous versions of Matlab up to R13 I've been able to use a java
class using getPasswordAuthentication to set these. Something along
the lines of System.setProperty for proxyHost, proxyPort,
proxyUserName and proxyPassword.

This java approach no longer works in R14. Someone in our IT services
suggested I try something like <http://myusername>:mypassword@wwwcache.rdg.ac.uk
n the Proxy host field of the preferences dialog box, but this
doesn't seem to work either.

I'd really appreciate any suggestions.

Best regards,
Neil

Matthew Simoneau wrote:
>
>
> I'm not quite sure what you're looking for. When you say "proxy
> preferences for user and password authentication", do you mean that
> your local proxy requires a username and proxy? If so, I don't
> think
> I've ever heard of this and don't know how to help. Maybe you
> didn't
> mean to use the term "proxy" at all and just meant that the site
> you're trying to connect to requires a username and password. If
> so,
> there are a few different ways to transmit them and I'd need to
> know
> more about the website.

Michael,

Thanks, I would like to try your java script. I did a search for
everything under your name (thanks, found some other useful stuff)
but nothing about proxies. Could you perhaps post a link for it?

Cheers, Neil

Michael Robbins wrote:
>
...
> I posted a Java script a few months ago that solves that problem.
> I
> was asking for help converting it to MATLAB but I didn't find a
> solution. You can use the Java, though.

Neil Caithness wrote:
>
>
> Michael,
>
> Thanks, I would like to try your java script. I did a search for
> everything under your name (thanks, found some other useful stuff)
> but nothing about proxies. Could you perhaps post a link for it?
>
> Cheers, Neil
>
> Michael Robbins wrote:
>>
> ...
>> I posted a Java script a few months ago that solves that
problem.
>> I
>> was asking for help converting it to MATLAB but I didn't find a
>> solution. You can use the Java, though.

Michael Robbins, "URLread" #, 22 Apr 2004 11:45 am
 
Michael Robbins, "URLREAD and password protected url..." #, 22 Aug
2003 9:33 am
 
 
Michael Robbins, "Pls help me translate Java to MATLAB" #, 19 Apr
2004 2:53 pm

Neil, I think you were saying it properly. Thanks for saying it
again. I think I understand now.

I searched around and found this article on proxies:

 <http://www.javaworld.com/javaworld/javatips/jw-javatip42.html>

It looks like you need to hack urlread.m. Something like this should
work:

% Open a connection to the URL.
urlConnection = url.openConnection;
password = 'username:password';
encodedPassword =
org.apache.axis.encoding.Base64.encode(double(password));
urlConnection.setRequestProperty('Proxy-Authorization',encodedPassword
);

We don't have this sort of proxy, so I can't test it out. Let us
know how it works and I'll see about rolling this fix into the
shipping version.

Matthew Simoneau wrote:
>
>
> Neil, I think you were saying it properly. Thanks for saying it
> again. I think I understand now.
>
> I searched around and found this article on proxies:
>
> <http://www.javaworld.com/javaworld/javatips/jw-javatip42.html>
>
> It looks like you need to hack urlread.m. Something like this
> should
> work:
>
> % Open a connection to the URL.
> urlConnection = url.openConnection;
> password = 'username:password';
> encodedPassword =
> org.apache.axis.encoding.Base64.encode(double(password));
>
urlConnection.setRequestProperty('Proxy-Authorization',encodedPasswo
> rd
> );
>
> We don't have this sort of proxy, so I can't test it out. Let us
> know how it works and I'll see about rolling this fix into the
> shipping version.

Wow. This is a popular thread!

Let us know how it works out.

I too am suffering from this username/password issue in matlab.
I have tried changing the urlread.m file but it doesnt seem to work.
I am wondering if there is an alternative method of getting through
the firewall?
Is it possible to use a .cgi script?

Many Thanks

Bart wrote:
>
>
> A small change to Matthew's code was needed to make it work. In
> urlread.m, change/add the following:
>
> % Open a connection to the URL.
> urlConnection = url.openConnection;
> password = 'username:password';
> encodedPassword =
> org.apache.axis.encoding.Base64.encode(double(password));
> myPwd = ['Basic ' encodedPassword.toCharArray'];
> urlConnection.setRequestProperty('Proxy-Authorization',myPwd);
>
>>
>> We don't have this sort of proxy, so I can't test it out. Let
us
>> know how it works and I'll see about rolling this fix into the
>> shipping version.
> I am behind a proxy which requires authorisation, and modifying
> urlread.m as above works for me.
>
> Cheers
> Bart
>
>

William, did you try Bart's fix above?

William,
Make sure to also set the Web Preferences in Matlab correctly: File >
Preferences > Web. You have to enter the name + port of your proxy
there. And then edit the urlread script as per my previous e-mail.
Doesn't that work?
Cheers
Bart Buelens

William wrote:
> I too am suffering from this username/password issue in matlab.
> I have tried changing the urlread.m file but it doesnt seem to work.
> I am wondering if there is an alternative method of getting through
> the firewall?
> Is it possible to use a .cgi script?
>
> Many Thanks

Hi, thanks for the help.

I set the proxy server in websettings (http://proxyserver)
nd also the port number. I then altered the urlread function file
using Bart's mail.

I keep getting an encoded password error as it doesnt seem to like
the encoded password line of:
encodedPassword =
org.apache.axis.encoding.Base64.encode(double(password))

Any suggestions? Thanks

I don't think you what the "http://"
refix when you specify your proxy server in the preferences.

What version of MATLAB are you using? I think we've only been
shipping the Axis classes since R14sp1. If you're on an older
version, you'll have to find another encoder. You may be able to use
the undocumented classes in Sun's JVM. Try this:

% Open a connection to the URL.
urlConnection = url.openConnection;
password = 'username:password';
encoder = sun.misc.BASE64Encoder;
encodedPassword = encoder.encode(double(password));
myPwd = ['Basic ' encodedPassword.toCharArray'];
urlConnection.setRequestProperty('Proxy-Authorization',myPwd);

Does this work?

I know there are several submissions on the MATLAB File Exchange that
pull stock data directly from Yahoo. Search for "yahoo" here:

 <http://www.mathworks.com/matlabcentral/fileexchange/>