LSTM prediction deviations in same test data despite very good RMSE

Hello everyone!

I am currently investigating different machine learning/deep learning methods to determine the speed of the gearbox based on vibration data from the gearbox.

I use the following network with input_size = 20000 (1 second of vibration data sampled with 20kHz) and number_of_neurons = 200:

layers = [sequenceInputLayer(input_size,"Name","sequence")
       lstmLayer(number_of_neurons,'OutputMode','sequence')
       fullyConnectedLayer(1,"Name","fc_o")
       regressionLayer("Name","regressionoutput")];

options = trainingOptions('adam',  ...
                'ExecutionEnvironment','gpu', ...
                'MiniBatchSize',128, ...
                'MaxEpochs',2000,...
                'Verbose',false, ...
                'InitialLearnRate',0.01,  ...
                'GradientThreshold',2, ...
                'Shuffle','every-epoch', ...
                'OutputFcn',@(info)stopIfAccuracyNotImproving(info,30), ...
                'Plots','training-progress');

[net,info] = trainNetwork(datagps.data_vib_split',datagps.data_rpm_split',layers,options);

I enter 20000 values of vibration data (1s, at 20kHz sampling) as input and 1 value of speed (the average speed of the gearbox over one second) as output. Altogether my trainingsset has a size of 3266x20000 samples (= 3266s measurement). The training results in an RMSE of 0.7 after about 370 epochs as the following image shows.

If I then transfer my training data set to the trained net I get good results (as you would expect with an RMSE of less than 1).

But if I just use a smaller part of the training data set as input with the same network (for example the part about the peak at 2300s) the results looks like the following

For this smaller part of the same data set the result of the network is very poor. When i exclude one dateset from the training and use it as testdata afterwards the result looks similiar (bad, compared to the rmse). Now I ask myself what could be the reason for this? Is this some kind of overfitting?

Have I possibly done something wrong in my construction/training? Or are there possibilities to make the net more robust for such cases?

Another thing i am wondering about is the number of iterations per epoch. Since i choose the minibatchsize as 128 and i have round about 3000 samples, shouldnt then the iteration per epoch be greater than 1? During the training progress window the iterations per epoch is 1 the whole time.

If there are any open questions, I will be happy to answer them. I would be pleased about an assessment or ideas.

Best regards,

Andreas