Thread Subject: URLREAD and password protected url...

Subject: URLREAD and password protected url...

From: Michael Robbins

Date: 22 Aug, 2003 09:33:20

Message: 1 of 18

I want to download data from a URL but it prompts for a username and
password.


I have the username and password, of course, but I cannot figure out
how to use URLREAD for this site.


I am running the current MATLAB version on Windows 2000.

Subject: URLREAD and password protected url...

From: Jeremie

Date: 22 Aug, 2003 09:56:37

Message: 2 of 18

Michael Robbins wrote:
> I want to download data from a URL but it prompts for a username
> and
> password.


if your url is <http://www.company.com/private/>,
your login 'login' and your password 'password', did you try to use
the URL
 <http://login>:password@www.company.com/private/


Jeremie

Subject: URLREAD and password protected url...

From: Marcus Vinicius Eiffle Duarte

Date: 22 Aug, 2003 10:00:48

Message: 3 of 18

Michael,


I do not have Matlab running here, so I can't test it.. but have you
tried to use urlread with method = 'post' and params = 'cell_array',
where cell_array = {username;password}?


Like this


urldoc = urlread('http://www.foo.com','post',{'mylogin';'mypasswd');


Hope it works.

Subject: URLREAD and password protected url...

From: Michael Robbins

Date: 22 Aug, 2003 10:05:58

Message: 4 of 18

>> I want to download data from a URL but it prompts for a
username
>> and password.
>
> if your url is <http://www.company.com/private/>,
> your login 'login' and your password 'password',
> did you try to use the URL
> <http://login>:password@www.company.com/private/


Thanks for the help. I did try that. My login has a "/" in it (ie.
abc/def), I don't know if that's the problem:


 <http://abc/def>:pwd@www.company.com/private/

Subject: URLREAD and password protected url...

From: Jeremie

Date: 22 Aug, 2003 10:14:38

Message: 5 of 18

Michael Robbins wrote:
> Thanks for the help. I did try that. My login has a "/" in it
> (ie.
> abc/def), I don't know if that's the problem:
>
> <http://abc/def>:pwd@www.company.com/private/


Yes, I think the '/' should be encoded using a '%xx' but I did a test
and the login:password@ solution does not seem to work with
urlread...
Sorry.

Subject: URLREAD and password protected url...

From: Doug Schwarz

Date: 22 Aug, 2003 12:57:23

Message: 6 of 18

In article <eec2a4f.0@WebX.raydaftYaTP>, Jeremie <invalid@nospam.com>
wrote:

>Michael Robbins wrote:
>> I want to download data from a URL but it prompts for a username
>> and
>> password.
>
>
>if your url is <http://www.company.com/private/>,
>your login 'login' and your password 'password', did you try to use
>the URL
> <http://login>:password@www.company.com/private/


Shouldn't that be

  <http://login:password@www.company.com/private/> ?

Note the placement of the '>'. I just used this syntax to get a file
via ftp and it worked just fine.

As for the '/' in the password, it might work as is because maybe the
'/' won't terminate the password part of the URL (since the '@' should
do that) or maybe you can try escaping it with a backslash or the %
notation.

--
Doug Schwarz
Eastman Kodak Company
douglas.schwarz@kodak.com

Subject: URLREAD and password protected url...

From: Peter Boettcher

Date: 22 Aug, 2003 13:02:09

Message: 7 of 18

Doug Schwarz <douglas.schwarz@kodak.com> writes:

> In article <eec2a4f.0@WebX.raydaftYaTP>, Jeremie <invalid@nospam.com>
> wrote:
>
>>Michael Robbins wrote:
>>> I want to download data from a URL but it prompts for a username
>>> and password.
>>
>>
>>if your url is <http://www.company.com/private/>,
>>your login 'login' and your password 'password', did you try to use
>>the URL
>> <http://login>:password@www.company.com/private/
>
>
> Shouldn't that be
>
> <http://login:password@www.company.com/private/> ?
>
> Note the placement of the '>'. I just used this syntax to get a file
> via ftp and it worked just fine.

The > was inserted courtesy of the perpetually broken URL handling of
the MATLAB Central newsreader.


--
Peter Boettcher <boettcher@ll.mit.edu>
MIT Lincoln Laboratory
MATLAB FAQ: http://www.mit.edu/~pwb/cssm/

Subject: URLREAD and password protected url...

From: Michael Robbins

Date: 22 Aug, 2003 13:02:50

Message: 8 of 18

> Shouldn't that be
<http://login:password@www.company.com/private/> ?
> Note the placement of the '>'. I just used this syntax to get a
> file
> via ftp and it worked just fine.
>
> As for the '/' in the password, it might work as is because maybe
> the
> '/' won't terminate the password part of the URL (since the '@'
> should
> do that) or maybe you can try escaping it with a backslash or the %
> notation.


Thanks, but it doesn't work for me.


Maybe it's the website, not the technique. I can access the website
fine via MS Internet Explorer.


>> [s,status]=urlread('<http://u1/u2:p@www.c.com/private/>')


s = ''
status = 0

Subject: URLREAD and password protected url...

From: Doug Schwarz

Date: 22 Aug, 2003 13:11:27

Message: 9 of 18

In article <eec2a4f.6@WebX.raydaftYaTP>,
 "Michael Robbins" <michael.robbins@us.cibc.com> wrote:

>Maybe it's the website, not the technique. I can access the website
>fine via MS Internet Explorer.
>
>
>>> [s,status]=urlread('<http://u1/u2:p@www.c.com/private/>')
>
>
>s = ''
>status = 0


I don't think you want the <> characters in there. Try

  [s,status]=urlread('http://u1/u2:p@www.c.com/private/')

--
Doug Schwarz
Eastman Kodak Company
douglas.schwarz@kodak.com

Subject: URLREAD and password protected url...

From: Jeremie

Date: 22 Aug, 2003 13:16:39

Message: 10 of 18

Doug Schwarz wrote:
<michael.robbins@us.cibc.com> wrote:
[s,status]=urlread('<http://u1/u2:p@www.c.com/private/>')
>>
>>s = ''
>>status = 0
>
>
> I don't think you want the <> characters in there. Try
>[s,status]=urlread('http://u1/u2:p@www.c.com/private/')


Yes, there shouldn't be any <>, but as I said in another post,
this way of handling login/password doesn't work with urlread, it
still need to be implemented by TMW I guess...
And I do not see any other solution...


Jeremie
PS: sorry for the > in the URL of my first post, it has been added
by the Matlab Central Newsreader as stated by Peter.

Subject: URLREAD and password protected url...

From: Michael Robbins

Date: 22 Aug, 2003 13:19:17

Message: 11 of 18

> I don't think you want the <> characters in there. Try
>
> [s,status]=urlread('http://u1/u2:p@www.c.com/private/')


That was my original attempt; it didn't work.


When I do this


[s,status]=urlread('http://www.c.com')


it works fine, so I think my connection is fine.


I wish I could give you my password, but I must not.

Subject: URLREAD and password protected url...

From: Doug Schwarz

Date: 22 Aug, 2003 13:27:52

Message: 12 of 18

In article <eec2a4f.8@WebX.raydaftYaTP>, Jeremie <invalid@nospam.com>
wrote:

>Doug Schwarz wrote:
><michael.robbins@us.cibc.com> wrote:
>[s,status]=urlread('<http://u1/u2:p@www.c.com/private/>')
>>>
>>>s = ''
>>>status = 0
>>
>>
>> I don't think you want the <> characters in there. Try
>>[s,status]=urlread('http://u1/u2:p@www.c.com/private/')
>
>
>Yes, there shouldn't be any <>, but as I said in another post,
>this way of handling login/password doesn't work with urlread, it
>still need to be implemented by TMW I guess...
>And I do not see any other solution...


But it does work (for me, at least). I was able to fetch a file (my
startup.m file located in my matlab directory) via ftp with

s = urlread('ftp://login:password@mymachine/matlab/startup.m');

--
Doug Schwarz
Eastman Kodak Company
douglas.schwarz@kodak.com

Subject: URLREAD and password protected url...

From: Jeremie

Date: 22 Aug, 2003 13:36:42

Message: 13 of 18

Doug Schwarz wrote:
> But it does work (for me, at least). I was able to fetch a file
> (my
> startup.m file located in my matlab directory) via ftp with
>
> s = urlread('ftp://login:password@mymachine/matlab/startup.m');


Well, and it does not work for me with HTTP, I obtain:
<TITLE>401 Authorization Required</TITLE>
</HEAD><BODY>
<H1>Authorization Required</H1>
This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.<P>


and I am sure of the argument passed to urlread because I copy-pasted
it from Matlab to wget and it works fine there...
I had a look to urlread code but I'm not a java expert. Maybe the
code should use the obect Authentificator from java.net:
 <http://java.sun.com/j2se/1.4.1/docs/api/java/net/Authenticator.html>


My two cents...

Subject: URLREAD and password protected url...

From: Doug Schwarz

Date: 22 Aug, 2003 13:49:48

Message: 14 of 18

In article <eec2a4f.11@WebX.raydaftYaTP>, Jeremie <invalid@nospam.com>
wrote:

>Doug Schwarz wrote:
>> But it does work (for me, at least). I was able to fetch a file
>> (my
>> startup.m file located in my matlab directory) via ftp with
>>
>> s = urlread('ftp://login:password@mymachine/matlab/startup.m');
>
>
>Well, and it does not work for me with HTTP, I obtain:

[snip]

>and I am sure of the argument passed to urlread because I copy-pasted
>it from Matlab to wget and it works fine there...
>I had a look to urlread code but I'm not a java expert. Maybe the
>code should use the obect Authentificator from java.net:
> <http://java.sun.com/j2se/1.4.1/docs/api/java/net/Authenticator.html>
>
>
>My two cents...


Ok, I'll buy that. I don't think I have any http sites here behind our
firewall that require authorization so I can't try anything else. It
looks like it works ok for ftp, but not for http. I guess Michael is
out of luck.

--
Doug Schwarz
Eastman Kodak Company
douglas.schwarz@kodak.com

Subject: URLREAD and password protected url...

From: Michael Robbins

Date: 22 Aug, 2003 14:04:07

Message: 15 of 18

Thanks guys, I'll `escalate' this to TMW developers...


meanwhile I guess I'll stick with perl :(

Subject: URLREAD and password protected url...

From: Jeremie

Date: 22 Aug, 2003 14:54:52

Message: 16 of 18

Michael Robbins wrote:
> Thanks guys, I'll `escalate' this to TMW developers...
>
> meanwhile I guess I'll stick with perl :(


I found a solution in hacking urlread...
you need to add a setRequestProperty sending your login and password:
 <http://www.zdnet.com.au/builder/webdesign/scripting/story/0,2000040414,20268689,00.htm>


in urlread, just add after line 67
urlConnection = url.openConnection;
>>> urlConnection.setRequestProperty('Authorization','Basic
xxxxxxxxxxxxxxxxxx');


where xxxxxxxxx is a base64 version of the string login:password
I used <http://www.opinionatedgeek.com/DotNet/Tools/Base64Encode/>
to obtain a base64 version of my login:password
.
If you want to automatize the process, this string can be extract
from your url using
url = java.net.URL('http://aa:bb@www.c.com/')
url.getUserInfo
then there must be a java function to do the conversion or you can
use Peter Acklam code to do so:
 <http://home.online.no/~pjacklam/matlab/software/util/datautil/base64encode.m>


Hope this helps !

Subject: URLREAD and password protected url...

From: Michael Robbins

Date: 22 Aug, 2003 15:02:28

Message: 17 of 18

> I found a solution in hacking urlread...
> you need to add a setRequestProperty sending your login and
> password:


It worked! Thank you. Good job.

Subject: URLREAD and password protected url...

From: Christian

Date: 18 Jan, 2010 12:37:03

Message: 18 of 18

Hi,

I'm also struggling with urlread and password protected websites. What if the user name is an email adress? So the code looks like

urlread('http://user@name.com:pass@www.company.com')

with the following error

??? Error using ==> urlread at 74
Either this URL could not be parsed or the protocol is not supported.


Maybe I got it wrong, but doesn't terminate the @ the password? How can this be solved? Or is this an error of the website (no protocol support?)?

Thanks for any help!
Chris

Tags for this Thread

Everyone's Tags:

Add a New Tag:

Separated by commas
Ex.: root locus, bode

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Tag Activity for This Thread
Tag Applied By Date/Time
urlread Matthew Simoneau 22 Aug, 2007 15:51:31
authentication Matthew Simoneau 22 Aug, 2007 15:51:31
password Matthew Simoneau 22 Aug, 2007 15:51:31
base64 Matthew Simoneau 22 Aug, 2007 15:51:31
rssFeed for this Thread

Contact us at files@mathworks.com