How to save multiple files from a URL list?

Hi,

I am trying to save multiple .tif files that come from a URL list. I was able to read the URL list but I am still having problems to save (as .tif format) the links with different consecutive names using the urlwrite function... any of you guys know another way to do the same: this is the code I have so far:

clear all
close all

% Load Data

URLs    = urlread('http://www.dataset.txt');
jj      = 504; % Must Define Number of URL Links (files) to be downloaded
jjj     = (1:jj)';
jjj     = num2str(jjj);
ii      = length(URLs(1,:))/jj; 

URLs    = reshape(URLs,ii,jj);
URLs    = URLs';

for x = 1:jj % As time goes from 1 to 10.
    for y = 1:ii;
        urlwrite(URLs(x,:),'treecover.tif','UserName',jjj(x,:));  
    end
end

any help or idea is welcome,

thanks,

Answer 1

[Geoff Hayes]

Rodrigo - it is not too clear to me how the URLs variable gets set or why it gets reshaped. What is the content from urlread('http://www.dataset.txt')? A simple list of URLs of images or is it something else entirely?

The code also has an inner for loop which iterate over y. What is the purpose of this, as y is not used in the urlwrite command? It is also my understanding from urlwrite that if the UserName option is specified, then so must the Password and Authentication. Is there a reason why you are including this, especially as the code seems to assign a string number for the UserName (from the variable jjj)?

Let us start with something simpler and go from there. Suppose we have a text file called urlsToRead.txt, and the content of it is just the URLs to various images. Something like

 http://www.mathworks.com/matlabcentral/profiles/4211382.jpg
 http://www.mathworks.com/matlabcentral/profiles/4211382.jpg
 http://www.mathworks.com/matlabcentral/profiles/4211382.jpg
 http://www.mathworks.com/matlabcentral/profiles/4211382.jpg
 http://www.mathworks.com/matlabcentral/profiles/4211382.jpg

All five lines point to the same image. The first thing we want to do, is to read this file into MATLAB. You can do this with importdata as

 urlsToImgs = importdata('urlsToRead.txt');

urlsToImgs is a cell array with 5 rows of URLs to images.

We can read each URL and save the image to file by iterating over each element in the variable and do something similar to what you did

 for k=1:size(urlsToImgs,1)

     % get the filename and extension
     [pathstr,name,ext] = fileparts(urlsToImgs{k});

     % create the target filename
     targFilename = [name '_' num2str(k) ext];

     % write the URL content to file
     urlwrite(urlsToImgs{k},targFilename);

 end

In the above code, all we are doing is taking each URL (that "points" to an image) and determining its path string pathstr (this is the http://www.mathworks.com/matlabcentral/profiles/), the name (just 4211382) and the extension ext (for example .jpg) and creating a target filename for the image we wish to write to file. In this example, since all URLs are the same, I just distinguish between each one by inserting a _X where X is an integer from 1 to 5. So the target file name on the first iteration becomes 4211382_1.jpg, the target file name on the second iteration becomes 4211382_2.jpg, etc. If your file names are distinct, then you do not need this.

Each URL is read, and the image content is written to one of the 4211382_X.jpg files. It should be no different for your tif files, so long as the dataset.txt file is written in such a way that the URLs can be read in individually as strings either through importdata or some other file reader.

Try integrating this with your code, and see what happens!

[Rodrigo Valdés-Pineda]

Hi Geoff,

thanks for the help, this worked perfectly.... !!!!

Best Regards,

Rodrigo

:-)