Info

This question is closed. Reopen it to edit or answer.

Urlread accessing millions of records

1 view (last 30 days)
Muneer
Muneer on 19 Nov 2013
Closed: MATLAB Answer Bot on 20 Aug 2021
Hello,
I am accessing a password protected database using urlread. Here's what the beginning of my code looks like.
clc;
clear;
tic
url = 'http://assa104/eurofot/csv';
block = urlread(url, 'Authentication', 'Basic', 'Post', {'login', '****', 'passwd', '*****', 'trips', '1332416521', 'signals', '1.AVL_BRTORQ_SUM.AVL_BRTORQ_SUM_DVCH','content_type', 'text'});
%, 'limit', '20000' after signals
toc
The problem is that I am accessing so much data this way that if I don't put a limit on how much I'm accessing, I just get an error and I can't read any data into my block. I removed a few signals from my code (one is shown, originally bringing in ~130) above so it wouldn't clutter this post but I'm accessing millions of data records. I need to bring the full set however. Can anyone offer a suggestion for working around this problem?
The other thing I was thinking about is instead of bringing in all 130 signals at once, bring in one at a time using urlread and save them into a matrix. The problem is the data set uses Last Observation Carried Forward so calling each signal individually may mean that each matrix has a different number of elements and then I won't be able to combine them into one matrix.
Any help would be great, thanks in advance!

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!