Downloading a zipped file from S3 via AWS

We're trying to download files (one at a time) from AWS S3.

I've used the example code I found on the web to get the list of files (works great), and I am trying to use this code to fetch the file:

fp= ['s3://ourBucketGoesHere/‘ selectedFile];

ds=fileDatastore(fp,'ReadFcn',@AWSRead, 'FileExtensions', {'.zip','','.txt'});

mydata=ds.read;

saveName = [filename ext];

save (saveName, 'mydata');

(also use this function for actual fetching) ->

function data = AWSRead(fileName)

fid = fopen(fileName);

data= fread(fid,inf);

fclose(fid);

end

Which does fetch the file.

The problem is that I can't save it in a format recognized. It should be gzip format (as it is coming from the server) but I can't make it work.

It's probably a setting I have wrong, I expect. Searching the net didn't find anything.

Answer 1

[Jingyu Si]

Try this funciton

ds=fileDatastore(fp,'ReadFcn',@S3_download);
dataFile=read(ds);

function dataFile=S3_download(readFileName)

writeFileName=regexp(readFileName,'\','split');
writeFileName=writeFileName{end}
readFileId = fopen(readFileName, 'r'); 
assert(readFileId > 0);
writeFileId = fopen(writeFileName, 'w');
assert(writeFileId > 0);
%%
buffersize = 1024;
while ~feof(readFileId)
    fileData = fread(readFileId, buffersize, '*uint8');
    writeCount = fwrite(writeFileId, fileData, 'uint8');
end

dataFile= writeFileName; 

end