fileDatastore request to AWS S3 limited to 1000 files

6 views (last 30 days)
I have a bucket on an AWS S3 server which I can reach without any problems.
The issue is: The answer from the server is limited to 1000 files. How can I increase this number? My target is to get just the list of filenames.
I also tried different calls of fileDatastore.
imds = fileDatastore(['s3://',<bucket>,','/'],'FileExtensions','.csv','ReadFcn',@load,'IncludeSubfolders',true)
  4 Comments
Jhon Wine
Jhon Wine on 18 Apr 2021
I have the same issue. When using Matlab R2020b I cannot get more than 1,000 files.
However while using Matlab R2019b, fileDataStore seems to return all the files I need.
Using AWS CLI seems to work fine, but is not as elegent of a solution.

Sign in to comment.

Answers (1)

Div Tiwari
Div Tiwari on 19 Apr 2021
Hi all, this issue was resolved in MATLAB R2021a and R2020b Update 5: https://www.mathworks.com/support/bugreports/details/2407120.
If you're using R2020b, you can install updates by clicking on the notification icon towards the top-right of the MATLAB window.
  1 Comment
Jhon Wine
Jhon Wine on 22 Apr 2021
I just installed R2021a, problem persists.
In fact I have two versions of Matlab on the same computer. 2019a shows 2000 files, 2021a shows 1000 files...

Sign in to comment.

Products


Release

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!