Downloading files from an internet directory

So I have a function that will grab files from a directory on my computer and iterate through each one and perform the task I want to each file.

I have to do this to a couple thousand files on one website and then later on I'll have to do it with several others so I want to try to automate the process so that I don't have to save each file into a directory on my computer first.

Here is what I thought could work, but it it doesn't. When I run it the dir() commmand returns a 0x1 struct. Therefore the second part of the program doesn't work. If I type the URL with file name directly into the urlwrite() command it works, but that only gets me one file at a time and I might as well save them like I have been.

So here is the code and let me know what you think or if there is a better way of doing this.

file = dir('http://blizzard.mtukrc.org/~weather/KRCwthr*.csv')
for i = 1:numel(file)
    urlwrite ('http://blizzard.mtukrc.org/~weather/',file(i).name)
end

The URL I am trying to get the files from is http://blizzard.mtukrc.org/~weather if it wasn't already obvious. I want to download all of the .csv files that begin with "KRCwrth*.csv." Thanks in advance for the help!

Answer 1

[Jan]

dir works with URLs, if you open a connection using the ftp command, see ftp.dir

I'd prefer mget.

[Joe]

I tried to do this, but I don't have access to the ftp. Is there anyway to do this without accessing the ftp server? I could possibly get access to this particular server as the organization is closely connected to my university and this is a university project, but that doesn't help me when I go to get data at other sites.

Thanks for the help.

[J Travis Hunsucker]

Jan- I cannot get dir to work with a url in R2018. Do you have any suggestions on how to query a folder using a url? Thank you

[dsmalenb]

Did you ever figure this out without the ftp server info? I have the same issue