feature classification tasks, responses must be categorical

I am doing a bearing fault classification project. I have segmented the vibration signals into small portion with a window. Did one hot encoding. When i try to train neural network it says feature classification tasks, responses must be categorical. this is the portion from the project

Combined_data = readtable('Data_file.csv');
win_len = 1000;
stride = 200;
X_Segmented = [];
Y_Segmented = {};
unique_faults = unique(Combined_data.Fault); % Uncomment this line if you want to use unique_faults
for i = 1:numel(unique_faults) % Loop over unique_faults
    desired_fault = unique_faults{i}; % Get the desired fault label
    df_temp_2 = Combined_data(strcmp(Combined_data.Fault, desired_fault), :);
    for j = 1:stride:(height(df_temp_2) - win_len)
        temp = df_temp_2{j:j + win_len - 1, 1};
        temp = temp(:)';
        X_Segmented = [X_Segmented; temp];
        Y_Segmented = [Y_Segmented; {desired_fault}]; % Store the desired fault label
    end
end
encoded_Y = grp2idx(Y_Segmented);
encoded_Y_row = encoded_Y';
OHE_Y = full(ind2vec(encoded_Y_row ))';
size(X_Segmented)
disp OHE_Y
%%%%%%%%%%%%%%%%%%%%%%test train validation split
% Extract the data and labels
%X = Combined_data{:, 1:end-1}; % Features (data)
%Y = Combined_data.Fault;       % Labels
% Convert labels to categorical
Y = categorical(Y_Segmented);
% Specify the proportion for the test set
testProportion = 0.3;
% Split the data into training and testing sets
cv = cvpartition(Y, 'HoldOut', testProportion);
idxTrain = training(cv);
idxTest = test(cv);
X_train = X_Segmented(idxTrain, :);
X_test = X_Segmented(idxTest, :);
y_train = OHE_Y(idxTrain, :); % Use OHE_Y for train and test
y_test = OHE_Y(idxTest, :);

%%% neural network modelling
numFeatures = size(X_train, 2);
numClasses = 14;
layers = [
    featureInputLayer(numFeatures, 'Normalization', 'zscore')
    fullyConnectedLayer(1024)
    reluLayer
    fullyConnectedLayer(512)
    reluLayer
    fullyConnectedLayer(256)
    reluLayer
    fullyConnectedLayer(128)
    reluLayer
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer
    ];
% Train options
options = trainingOptions('adam', ...
    'MaxEpochs', 50, ...
    'MiniBatchSize', 500, ...
    'Shuffle', 'every-epoch', ...
    'ValidationData', {X_test, y_test}, ...
    'ValidationFrequency', 10, ...
    'Verbose', true, ...
    'Plots', 'training-progress', ...
    'ExecutionEnvironment', 'auto'); % or 'gpu' if available
% Train the network
net = trainNetwork(X_train,y_train, layers, options);

The size of the X_Segmented is 15048*1000 , X_train 10534*1000 double , y_train 10534*14 double. I have 14 labels . Y_train is a subset of OHE_Y , so it should be already in a categorical form, if I am right. When I do small corrrections, like changing the response in trainNetwork, and then it says responses must be a vector of categorical responses. What might the issue actually?? Why is OHE_Y not a categorical form?

[Image Analyst]

Too bad you forgot to attach 'Data_file.csv'. I'll check back later for it.

What does this show in the command window

whos OHE_Y
class(OHE_Y)

Don't use semicolons.

If you have any more questions, then attach your data and code to read it in with the paperclip icon after you read this:

TUTORIAL: How to ask a question (on Answers) and get a fast answer

[Hrishikesh Kesavan Nair]

Thanks for the reply. In the command wondow it shows as follows

ans =

15048 1000

Name Size Bytes Class Attributes

OHE_Y 15048x14 1685376 double

ans =

'double'

[Image Analyst]

Too bad you forgot to attach 'Data_file.csv' AGAIN! We'll check back later for it.

Looks like it's double when it wants categorical. We'll check the code later you've uploaded the data and we can actually run it.

[Hrishikesh Kesavan Nair]

Hi. I have attached the 'Data_file.csv' . Unfortunately it is larger in size than allowed here to upload. i have added a google drive link. its 10 mb. please have a look.

https://drive.google.com/file/d/1HXr3irpBRWmFOYCsw-Ovi0V1Bn4oAhb6/view?usp=share_link

if it is not preffered to access a file from G Drive, please tell if a subset of the same would work.

Answer 1

[Sebastian Gross]

Hi Hrishikesh,

I had a quick look at your code. Your problem is actually the validation aka test data. y_test is not categorical.

% Train options
options = trainingOptions('adam', ...
    'MaxEpochs', 50, ...
    'MiniBatchSize', 500, ...
    'Shuffle', 'every-epoch', ...
    'ValidationData', {X_test, Y_test}, ...
    'ValidationFrequency', 10, ...
    'Verbose', true, ...
    'Plots', 'training-progress', ...
    'ExecutionEnvironment', 'auto'); % or 'gpu' if available
% Train the network
Y_train = Y(idxTrain);
Y_test = Y(idxTest);
net = trainNetwork(X_train,Y_train, layers, options);

I changed your last lines to this. Please note that

a) I do not know why your Y data look is so complex. Should it not be just a vector of your responses?

I created new response vectors from your categorical Y.

b) I changed y_test to the Y_test I created in the validation data options and in the train network line.

and ...

Training on single CPU.

Initializing input data normalization.

|======================================================================================================================|

| Epoch | Iteration | Time Elapsed | Mini-batch | Validation | Mini-batch | Validation | Base Learning |

| | | (hh:mm:ss) | Accuracy | Accuracy | Loss | Loss | Rate |

|======================================================================================================================|

| 1 | 1 | 00:00:05 | 10.80% | 19.03% | 2.7485 | 2.3343 | 0.0010 |

| 1 | 10 | 00:00:06 | 55.00% | 56.54% | 1.3041 | 1.1979 | 0.0010 |

| 1 | 20 | 00:00:06 | 75.80% | 78.07% | 0.6762 | 0.6176 | 0.0010 |

[...]

| 50 | 1050 | 00:01:32 | 100.00% | 95.77% | 0.0003 | 0.1664 | 0.0010 |

|======================================================================================================================|

Training finished: Max epochs completed.

Best

Sebastian