MATLAB Examples

Contents

Load file and create datastore

ds = datastore('*.csv');
ds.SelectedVariableNames = {'Speed_OBD__mph_','DeviceTime'};

previewDS = preview(ds);
Warning: Variable names were modified to make them valid MATLAB identifiers. 

Invoke mapreduce

mr = mapreducer(0); % Serial execution
% mr = mapreducer(); % Parallel execution

result = mapreduce(ds, @mapper, @reducer, mr);

out = readall(result);
fullResults = table(out.Value{1},out.Value{2},out.Value{3},...
    'VariableNames',{out.Key{1},out.Key{2},out.Key{3}})
********************************
*      MAPREDUCE PROGRESS      *
********************************
Map   0% Reduce   0%
Map  40% Reduce   0%
Map  60% Reduce   0%
Map  80% Reduce   0%
Map 100% Reduce  33%
Map 100% Reduce  67%
Map 100% Reduce 100%

fullResults = 

     timeOfDeceleration                                      fileName                                      decelerationAmount
    ____________________    ___________________________________________________________________________    __________________

    15-Jan-2015 15:46:22    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-15_15-3...'    -5.59234142       
    15-Jan-2015 17:09:48    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-15_16-5...'    -8.69919682       
    16-Jan-2015 06:16:29    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -6.21371079       
    16-Jan-2015 06:20:22    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -9.32056809       
    16-Jan-2015 06:23:22    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -6.21371174       
    16-Jan-2015 06:23:37    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -6.21371174       
    16-Jan-2015 06:39:52    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -6.21371174       
    16-Jan-2015 06:40:22    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -6.21371078       
    16-Jan-2015 06:40:23    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -6.83508301       
    16-Jan-2015 06:40:24    'C:\Documents\AnalyzingFleetData\DeerIncident\trackLog-2015-Jan-16_06-1...'    -7.45645333       

Next step - point to HDFS

% ds = datastore('hdfs://ec2-54-69-190-244.us-west-2.compute.amazonaws.com/user/fleetdata/*.csv');