How to customize kernel functions in SVR

Previously, the answer to this question and the official documentation on how to construct an external kernel function were vague, and we hope to obtain a complete version of the example.

Due to my inability to obtain the kernel function matrix within fitrsvm, I want to know how to embed a Gaussian kernel function externally. I followed the instructions（https://ww2.mathworks.cn/matlabcentral/answers/688594-matern-kernel-functions-for-support-vector-regression?s_tid=sug_su） for this issue and wrote a string of code.

load carsmall.mat

X = [Horsepower,Weight]';

Y = MPG';

% This is how you pass custom KernelFunction

mdl = fitrsvm(X, Y, 'KernelFunction', char(myGaussianKernel(X,1,Y)));

function omega = myGaussianKernel(Xtrain, kernel_pars,Xt)

nb_data = size(Xtrain,1);

if kernel_pars(1)==0

kernel_pars=10^-4;

end

% if nargin<4

% XXh = sum(Xtrain.^2,2)*ones(1,nb_data);

% omega1 = XXh+XXh'-2*(Xtrain*Xtrain');

% omega = exp(-omega1./kernel_pars(1));

% omega = exp(-omega1./kernel_pars(1)^2);

% else

XXh1 = sum(Xtrain.^2,2)*ones(1,size(Xt,1));

XXh2 = sum(Xt.^2,2)*ones(1,nb_data);

omega1 = XXh1+XXh2' - 2*Xtrain*Xt';

omega = exp(-omega1./kernel_pars(1)^2);

% end

end

However, the following error occurred:

Incorrect use of strncmpi

The number of rows in the character matrix must match the number of elements in another array.

出错classreg.learning.modelparams.SVMParams.make (第 228 行)

tf = strncmpi(kernelfun,allowedVals,length(kernelfun));

出错classreg.learning.FitTemplate/fillIfNeeded (第 712 行)

this.MakeModelParams(this.Type,this.MakeModelInputArgs{:});

出错 classreg.learning.FitTemplate.make (第 140 行)

temp = fillIfNeeded(temp,type);

出错 RegressionSVM.template (第 363 行)

temp = classreg.learning.FitTemplate.make('SVM','type','regression',varargin{:});

出错 RegressionSVM.fit (第 357 行)

temp = RegressionSVM.template(varargin{:});

出错 fitrsvm (第 280 行)

obj = RegressionSVM.fit(X,Y,RemainingArgs{:});

出错 untitled3 (第 5 行)

mdl = fitrsvm(X, Y, 'KernelFunction', char(myGaussianKernel(X,1,Y)));

Another issue is that I don't know how a custom kernel function distinguishes between training and testing. I believe that the kernel function is obtained by multiplying itself during training. You can see this case. But even if I input X and X, the above error message still appears.

load carsmall.mat

X = [Horsepower,Weight];

Y = MPG;

% This is how you pass custom KernelFunction

mdl = fitrsvm(X, Y, 'KernelFunction', char(mykernel(X,X)));

function KMN = mykernel(XM,XN)

%mykernel - Compute sum of squared exponential and squared exponential ARD.

% KMN = mykernel(XM,XN,theta) takes a M-by-D matrix XM, a N-by-D matrix

% XN and computes a M-by-N matrix KMN of kernel products such that

% KMN(i,j) is the kernel product between XM(i,:) and XN(j,:). theta is

% the R-by-1 unconstrained parameter vector for the kernel.

%

% Let theta = [log(sigmaL1);log(sigmaL2);log(sigmaF1);log(sigmaF2)]

%

% where

%

% sigmaL1 = D-by-1 vector of length scales for squared exponential ARD.

% sigmaL2 = scalar length scale for squared exponential.

% sigmaF1 = scalar signal standard deviation for squared exponential ARD.

% sigmaF2 = scalar signal standard deviation for squared exponential.

D = size(XM,2);

sigmaL10 = 0.1*ones(D,1);

sigmaL20 = 0.1;

sigmaF10 = 1;

sigmaF20 = 1;

theta = [log(sigmaL10);log(sigmaL20);log(sigmaF10);log(sigmaF20)];

% 1. Get D. Assume XN, XM have the same number of columns.

D = size(XM,2);

% 2. Convert theta into sigmaL1, sigmaL2, sigmaF1 and sigmaF2.

params = exp(theta);

sigmaL1 = params(1:D,1);

sigmaL2 = params(D+1,1);

sigmaF1 = params(D+2,1);

sigmaF2 = params(D+3,1);

% 3. Create the contribution due to squared exponential ARD.

KMN = pdist2(XM(:,1)/sigmaL1(1), XN(:,1)/sigmaL1(1)).^2;

for r = 2:D

KMN = KMN + pdist2(XM(:,r)/sigmaL1(r), XN(:,r)/sigmaL1(r)).^2;

end

KMN = (sigmaF1^2)*exp(-0.5*KMN);

% 4. Add the contribution due to squared exponential.

KMN = KMN + (sigmaF2^2)*exp(-0.5*(pdist2(XM/sigmaL2, XN/sigmaL2).^2));

end

Answer 1

[Garmit Pant]

Hello

It is my understanding that you are trying to use a custom kernel function in SVM regression training.

The ‘fitsvmr’ function provides the functionality to use custom kernel functions as is mentioned in the documentation. You need to use Name-Value Argument ‘KernelFunction’ to specify your kernel function.

The custom kernel function should be of the form:

function G = kernel(U,V)

where:

• U is an m-by-p matrix.
• V is an n-by-p matrix.
• G is an m-by-n Gram matrix of the rows of U and V.

And 'kernel.m' must be on the MATLAB path or in the same directory as the main file.

You can use the following code snippet as example to how to define and use custom kernel functions.

load carsmall
rng 'default'  % For reproducibility
% Specify Horsepower and Weight as the predictor variables (X) and MPG as the response variable (Y). 
X = [Horsepower Weight];
Y = MPG;
% Train the SVM with the Gaussian kernel function. 
MdlGau = fitrsvm(X,Y,'Standardize',true,'KFold',5,'KernelFunction','gaussian')
mseGau = kfoldLoss(MdlGau)
% Train the SVM with the custom Gaussian kernel function defined as a separate function file named ‘gaussianKernelMatrix.m’. 
MdlGau2 = fitrsvm(X,Y,'Standardize',true,'KFold',5,'KernelFunction','gaussianKernelMatrix')
mseGau2 = kfoldLoss(MdlGau2)

In the above code snippet, ‘gaussianKernelMatrix.m’ is defined as follows:

function K = gaussianKernelMatrix(X, Y)
    % Compute the Gaussian kernel matrix between two sets of data points X and Y.

    % X and Y should be matrices where each row represents a data point.
    % gamma is taken as 1.

    % Calculate the squared Euclidean distances
    distSquared = sum(X.^2, 2) - 2 * (X * Y') + sum(Y.^2, 2)';

    % Compute the Gaussian kernel matrix
    K = exp(-distSquared / (2 * 1^2));
end

You can refer to the following documentation for further understanding:

https://in.mathworks.com/help/stats/fitrsvm.html