Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Auto downloading from a url on a regular basis

Subject: Auto downloading from a url on a regular basis

From: Kian

Date: 16 Apr, 2013 21:03:08

Message: 1 of 6

This is this EUMETSAT page which provides multisensor atmospheric imagery estimates on an hourly basis. Here is the link to one such estimate:
http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/index.htm

If you open the link above, you can choose some specific number of frames (2 to 24) and click on download and it will download the number of frames you have specified (starting from the latest image and going back in time) for you.

Is there any matlab code which I can put into an .m file that would run in the background (If I keep the program open) and automatically downloads from the link I have shown below (as for 24 frames of the above url) on a regular basis, like every 10 hours? I mean it download from the following url and then every 10 hours download from the same link again (as the new download will contain new imagery).

Here is the link for the above url (24 frames): http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/IMAGESDisplay/24FramesDUST-AMERICA.zip

I really appreciate any help.

Subject: Auto downloading from a url on a regular basis

From: TideMan

Date: 17 Apr, 2013 00:48:20

Message: 2 of 6

On Wednesday, April 17, 2013 9:03:08 AM UTC+12, Kian wrote:
> This is this EUMETSAT page which provides multisensor atmospheric imagery estimates on an hourly basis. Here is the link to one such estimate:
>
> http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/index.htm
>
>
>
> If you open the link above, you can choose some specific number of frames (2 to 24) and click on download and it will download the number of frames you have specified (starting from the latest image and going back in time) for you.
>
>
>
> Is there any matlab code which I can put into an .m file that would run in the background (If I keep the program open) and automatically downloads from the link I have shown below (as for 24 frames of the above url) on a regular basis, like every 10 hours? I mean it download from the following url and then every 10 hours download from the same link again (as the new download will contain new imagery).
>
>
>
> Here is the link for the above url (24 frames): http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/IMAGESDisplay/24FramesDUST-AMERICA.zip
>
>
>
> I really appreciate any help.

After much experimentation with Matlab, I found it is not good for this sort of application - too many times it locks up for no apparent reason.
I download data from NOAA databases daily and have found that cURL (Google it) is better suited.
As far as putting it in a schedule that runs in the background, I've found System Scheduler from Splinterware to be excellent for Windows. Of course, for Linux there is cron.

Subject: Auto downloading from a url on a regular basis

From: Kian

Date: 17 Apr, 2013 02:34:07

Message: 3 of 6

TideMan <mulgor@gmail.com> wrote in message <0b79dbb5-cb90-46ac-b228-37d1ba1fd9b7@googlegroups.com>...
> On Wednesday, April 17, 2013 9:03:08 AM UTC+12, Kian wrote:
> > This is this EUMETSAT page which provides multisensor atmospheric imagery estimates on an hourly basis. Here is the link to one such estimate:
> >
> > http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/index.htm
> >
> >
> >
> > If you open the link above, you can choose some specific number of frames (2 to 24) and click on download and it will download the number of frames you have specified (starting from the latest image and going back in time) for you.
> >
> >
> >
> > Is there any matlab code which I can put into an .m file that would run in the background (If I keep the program open) and automatically downloads from the link I have shown below (as for 24 frames of the above url) on a regular basis, like every 10 hours? I mean it download from the following url and then every 10 hours download from the same link again (as the new download will contain new imagery).
> >
> >
> >
> > Here is the link for the above url (24 frames): http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/IMAGESDisplay/24FramesDUST-AMERICA.zip
> >
> >
> >
> > I really appreciate any help.
>
> After much experimentation with Matlab, I found it is not good for this sort of application - too many times it locks up for no apparent reason.
> I download data from NOAA databases daily and have found that cURL (Google it) is better suited.
> As far as putting it in a schedule that runs in the background, I've found System Scheduler from Splinterware to be excellent for Windows. Of course, for Linux there is cron.


Thank you very much Tideman. I ll take your suggestions.

Subject: Auto downloading from a url on a regular basis

From: Kian

Date: 17 Apr, 2013 17:44:07

Message: 4 of 6

"Kian " <kianian@gmail.com> wrote in message <kkl1mv$8dt$1@newscl01ah.mathworks.com>...
> TideMan <mulgor@gmail.com> wrote in message <0b79dbb5-cb90-46ac-b228-37d1ba1fd9b7@googlegroups.com>...
> > On Wednesday, April 17, 2013 9:03:08 AM UTC+12, Kian wrote:
> > > This is this EUMETSAT page which provides multisensor atmospheric imagery estimates on an hourly basis. Here is the link to one such estimate:
> > >
> > > http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/index.htm
> > >
> > >
> > >
> > > If you open the link above, you can choose some specific number of frames (2 to 24) and click on download and it will download the number of frames you have specified (starting from the latest image and going back in time) for you.
> > >
> > >
> > >
> > > Is there any matlab code which I can put into an .m file that would run in the background (If I keep the program open) and automatically downloads from the link I have shown below (as for 24 frames of the above url) on a regular basis, like every 10 hours? I mean it download from the following url and then every 10 hours download from the same link again (as the new download will contain new imagery).
> > >
> > >
> > >
> > > Here is the link for the above url (24 frames): http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/IMAGESDisplay/24FramesDUST-AMERICA.zip
> > >
> > >
> > >
> > > I really appreciate any help.
> >
> > After much experimentation with Matlab, I found it is not good for this sort of application - too many times it locks up for no apparent reason.
> > I download data from NOAA databases daily and have found that cURL (Google it) is better suited.
> > As far as putting it in a schedule that runs in the background, I've found System Scheduler from Splinterware to be excellent for Windows. Of course, for Linux there is cron.
>
>
> Thank you very much Tideman. I ll take your suggestions.

Hi there again.
I have set up the scheduler to run cURL every 10 hours as you suggested. The problem is every time it runs it overwrites on the same already downloaded file. What did you do to overcome this?

Subject: Auto downloading from a url on a regular basis

From: TideMan

Date: 17 Apr, 2013 19:55:40

Message: 5 of 6

On Thursday, April 18, 2013 5:44:07 AM UTC+12, Kian wrote:
> "Kian " <kianian@gmail.com> wrote in message <kkl1mv$8dt$1@newscl01ah.mathworks.com>...
>
> > TideMan <mulgor@gmail.com> wrote in message <0b79dbb5-cb90-46ac-b228-37d1ba1fd9b7@googlegroups.com>...
>
> > > On Wednesday, April 17, 2013 9:03:08 AM UTC+12, Kian wrote:
>
> > > > This is this EUMETSAT page which provides multisensor atmospheric imagery estimates on an hourly basis. Here is the link to one such estimate:
>
> > > >
>
> > > > http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/index.htm
>
> > > >
>
> > > >
>
> > > >
>
> > > > If you open the link above, you can choose some specific number of frames (2 to 24) and click on download and it will download the number of frames you have specified (starting from the latest image and going back in time) for you.
>
> > > >
>
> > > >
>
> > > >
>
> > > > Is there any matlab code which I can put into an .m file that would run in the background (If I keep the program open) and automatically downloads from the link I have shown below (as for 24 frames of the above url) on a regular basis, like every 10 hours? I mean it download from the following url and then every 10 hours download from the same link again (as the new download will contain new imagery).
>
> > > >
>
> > > >
>
> > > >
>
> > > > Here is the link for the above url (24 frames): http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/IMAGESDisplay/24FramesDUST-AMERICA.zip
>
> > > >
>
> > > >
>
> > > >
>
> > > > I really appreciate any help.
>
> > >
>
> > > After much experimentation with Matlab, I found it is not good for this sort of application - too many times it locks up for no apparent reason.
>
> > > I download data from NOAA databases daily and have found that cURL (Google it) is better suited.
>
> > > As far as putting it in a schedule that runs in the background, I've found System Scheduler from Splinterware to be excellent for Windows. Of course, for Linux there is cron.
>
> >
>
> >
>
> > Thank you very much Tideman. I ll take your suggestions.
>
>
>
> Hi there again.
>
> I have set up the scheduler to run cURL every 10 hours as you suggested. The problem is every time it runs it overwrites on the same already downloaded file. What did you do to overcome this?

THIS is where Matlab is useful, but you could equally do it in a DOS script or with Python or Perl or ...............
You could write a Matlab script called, say, getFile.m containing this:
dos(['cURL http:/ etc -o ' filename]);
where you have defined filename as a unique name from the date or whatever, then in the scheduler you do this:
Application: matlab
Parameters: -r getFile;quit

Providing getFile.m is in your Matlab path, this will execute the .m file in background.

Subject: Auto downloading from a url on a regular basis

From: Kian

Date: 18 Apr, 2013 02:02:09

Message: 6 of 6

This is wonderful ... Thank you again Tideman!


TideMan <mulgor@gmail.com> wrote in message <1c661729-00f2-4737-b29a-c101958d021b@googlegroups.com>...
> On Thursday, April 18, 2013 5:44:07 AM UTC+12, Kian wrote:
> > "Kian " <kianian@gmail.com> wrote in message <kkl1mv$8dt$1@newscl01ah.mathworks.com>...
> >
> > > TideMan <mulgor@gmail.com> wrote in message <0b79dbb5-cb90-46ac-b228-37d1ba1fd9b7@googlegroups.com>...
> >
> > > > On Wednesday, April 17, 2013 9:03:08 AM UTC+12, Kian wrote:
> >
> > > > > This is this EUMETSAT page which provides multisensor atmospheric imagery estimates on an hourly basis. Here is the link to one such estimate:
> >
> > > > >
> >
> > > > > http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/index.htm
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > > If you open the link above, you can choose some specific number of frames (2 to 24) and click on download and it will download the number of frames you have specified (starting from the latest image and going back in time) for you.
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > > Is there any matlab code which I can put into an .m file that would run in the background (If I keep the program open) and automatically downloads from the link I have shown below (as for 24 frames of the above url) on a regular basis, like every 10 hours? I mean it download from the following url and then every 10 hours download from the same link again (as the new download will contain new imagery).
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > > Here is the link for the above url (24 frames): http://oiswww.eumetsat.org/IPPS/html/MSG/RGB/DUST/AMERICA/IMAGESDisplay/24FramesDUST-AMERICA.zip
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > > I really appreciate any help.
> >
> > > >
> >
> > > > After much experimentation with Matlab, I found it is not good for this sort of application - too many times it locks up for no apparent reason.
> >
> > > > I download data from NOAA databases daily and have found that cURL (Google it) is better suited.
> >
> > > > As far as putting it in a schedule that runs in the background, I've found System Scheduler from Splinterware to be excellent for Windows. Of course, for Linux there is cron.
> >
> > >
> >
> > >
> >
> > > Thank you very much Tideman. I ll take your suggestions.
> >
> >
> >
> > Hi there again.
> >
> > I have set up the scheduler to run cURL every 10 hours as you suggested. The problem is every time it runs it overwrites on the same already downloaded file. What did you do to overcome this?
>
> THIS is where Matlab is useful, but you could equally do it in a DOS script or with Python or Perl or ...............
> You could write a Matlab script called, say, getFile.m containing this:
> dos(['cURL http:/ etc -o ' filename]);
> where you have defined filename as a unique name from the date or whatever, then in the scheduler you do this:
> Application: matlab
> Parameters: -r getFile;quit
>
> Providing getFile.m is in your Matlab path, this will execute the .m file in background.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us