replicating logistic regression in patternnet, probabilities match but weights don't

4 views (last 30 days)
I'm attempting to replicate a linear regression using a patternnet. The trained network gives back the probabilities I expect, but the weights are off by a factor of 2. This is true even if I initialize weights to where I expect them to end up. Am I interpreting the weights incorrectly? Thanks.
clear; rng(0);
% create data with logistic regression model
n = 1e6;
p = 2;
X = [ones(n,1) rand(n,p)];
beta = [1 2 -3]';
prob = logsig(X*beta);
y = binornd(1,prob);
net = patternnet([]); % zero hidden layers
net = train(net,X(:,2:end)',y');
getwb(net) % outputs beta/2 instead of beta
% probabilities are fine
x0 = [.9 .5]';
logsig([1 x0'] * beta)
net(x0)

Answers (1)

Debraj Bhattacharjee
Debraj Bhattacharjee on 11 Jan 2019
The reason you are getting such an output is because when a “net” is created, it automatically scales the input data. This scaling is being done internally by the “mapminmax” function which scales the input data between [-1 1]. This can be checked by entering “net.inputs{1}.processFcns(2)” in the MATLAB Command Window. You can find more information about this function in the link given below.
In the original code, the target probabilities passed on to the network do not reflect this scaling on the input data and hence the output does not align with the expected values. The following two code snippets can help you understand what the network does under the hood. The additions to the code can be found in the block of comments containing asterisks.
1. Using the “mapminmax” function to scale the input data before computing the probabilities ensures that the target probabilities are computed using the scaled data and, hence, the output is same as the expected values. The corresponding MATLAB code is given below.
rng(0);
% create data with logistic regression model
n = 1e6;
p = 2;
X = [ones(n,1) rand(n,p)];
% ********************************
% Scale X using mapminmax
[X,ps]= mapminmax(X);
% ********************************
% Same code as before
beta = [1 2 -3]';
% Probabilities are now computed based on
% scaled data
prob = logsig(X*beta);
y = binornd(1,prob);
net = patternnet([]); % zero hidden layers
net = train(net,X(:,2:end)',y');
getwb(net) % Output beta is now correct
% probabilities are fine
x0 = [.9 .5]';
logsig([1 x0'] * beta)
net(x0)
2. Another way of taking care of this issue would be to actively change the settings of the network so that the network does not scale the input data implicitly. The corresponding MATLAB code is given below.
rng(0);
% create data with logistic regression model
n = 1e6;
p = 2;
X = [ones(n,1) rand(n,p)];
beta = [1 2 -3]';
prob = logsig(X*beta);
y = binornd(1,prob);
net = patternnet([]); % zero hidden layers
% ********************************
% Remove mapminmax from automatic input processing
net.inputs{1}.processFcns(2) = [];
% ********************************
% Same code as before
net = train(net,X(:,2:end)',y');
getwb(net) % Output beta is now correct
% probabilities are fine
x0 = [0.9 .5]';
logsig([1 x0'] * beta)
net(x0)
In both cases, the output “beta” aligns with the expected values. Another useful article on this topic can be found in the link below.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!