Content

Scalability test

Reference to code Determinant model

For scalability test we need large dataset, large list of features and accuracy challenge. It has been found that determinants of random matrices answer this need. In case of $3 * 3$ matrix, the model is simple and accurate and training needs only a few seconds, so we show here more challenging example with $4 * 4$ matrix. Training dataset is 100,000 records. Validation dataset is 10,000 to 20,000 records.

Experiment 1

1.1 MATLAB

We use MATLAB for comparison of accuracy and performance. To achieve reasonable accuracy with these data in MATLAB we need multiple layers. We used network of 5 layers with 10 neurons in each. The accuracy metric is Pearson correlation coefficient for computed and actual targets of validation set. The MATLAB script is below:
clear variables;
close all;

%. number of training records
Ntrain = 100000;

%. number of validation records
Nval = 20000;

%. dimensionality of matrix
dim = 4;

%% generate data
N = Ntrain + Nval;
Nid = Ntrain + 1;

%. inputs
m = dim * dim;
x = rand(N, m);

%. outputs
y = zeros(N, 1);
for ii = 1:N
    xx = reshape(x(ii, :).', dim, dim);
    y(ii) = det(xx);
end

%. labels
lab = ones(N, 1);
lab(Nid:end) = 2;
identID = 1;
verifID = 2;

%%
Xtrain = x(1:(Nid-1),:);
Ytrain = y(1:(Nid-1));
Xtest = x(Nid:end,:);
Ytest = y(Nid:end);

tic;

Mdl = fitrnet(Xtrain, Ytrain, ...
    'Standardize', 1, 'Activations', 'tanh', 'LayerSizes', ...
    [10 10 10 10 10], 'IterationLimit', 1000, 'Verbose', 1);
Ypred = predict(Mdl, Xtest);

%. correlation coefficient
cc = corrcoef(Ytest, Ypred); 
COR = cc(1,2);
disp(['Correlation coefficient: ', num2str(COR)]);

toc;

%%
plot(Ytest, Ypred, 's');
The results are below in table.

MATLAB
index accuracy training time (sec)
1 0.94 104
2 0.94 111
3 0.94 110


MATLAB generated image showing correlation for computed and actual targets:


1.2 Classic Kolmogorov-Arnold representation (KAR)

The C++ implementation of Kolmogorov-Arnold is downloadable from the link on the top.

C++ KAR classic
index accuracy training time (sec)
1 0.94 35
2 0.94 33
3 0.94 33

Experiment 2

2.1 MATLAB

Another comparison is between larger neural network and 3 layer KAN. The training is longer in both cases and accuracy is higher in both cases. Below is MATLAB script: 
% Parameters
num_train = 100000;
num_val = 2000;
input_size = 16; % 4x4 matrices flattened to vectors

% Generate random data
train_features = rand(num_train, 4, 4);
train_targets = zeros(num_train, 1);

for i = 1:num_train
    train_targets(i) = det(squeeze(train_features(i, :, :)));
end
train_features = reshape(train_features, [num_train, input_size]);

val_features = rand(num_val, 4, 4);
val_targets = zeros(num_val, 1);

for i = 1:num_val
    val_targets(i) = det(squeeze(val_features(i, :, :)));
end
val_features = reshape(val_features, [num_val, input_size]);

% Define the neural network
layers = [ 
    featureInputLayer(input_size, 'Name', 'input')
    fullyConnectedLayer(64, 'Name', 'fc1')
    reluLayer('Name', 'relu1')
    fullyConnectedLayer(128, 'Name', 'fc2')
    reluLayer('Name', 'relu2')
    fullyConnectedLayer(64, 'Name', 'fc3')
    reluLayer('Name', 'relu3')
    fullyConnectedLayer(32, 'Name', 'fc4')
    reluLayer('Name', 'relu4')
    fullyConnectedLayer(1, 'Name', 'output')
    regressionLayer('Name', 'regression')
];

% Training options
options = trainingOptions('adam', ...
    'MaxEpochs', 50, ...
    'MiniBatchSize', 256, ...
    'InitialLearnRate', 0.001, ...
    'Shuffle', 'every-epoch', ...
    'ValidationData', {val_features, val_targets}, ...
    'ValidationFrequency', 30, ...
    'Plots', 'training-progress', ...
    'Verbose', false);

% Train the neural network
net = trainNetwork(train_features, train_targets, layers, options);

% Evaluate performance on validation data
predictions = predict(net, val_features);
corr_coeff = corr(predictions, val_targets);

fprintf('Correlation Coefficient on Validation Data: %.4f\n', corr_coeff);
It runs much longer (near 250 seconds) and builds a little bit more accurate model (correlation coefficient is near 97%).

2.2 Three layer Kolmogorov-Arnold network

The reference to the code is on the top of the page. The results are below in table.

C++ 3 layer KAN
index accuracy training time (sec)
1 0.98 64
2 0.98 62
3 0.98 66

Experiment 3

This test is modeling of determinants of $5 * 5$ random matrices.

3.1 MATLAB

Neural network has 7 layers, each layer has 60 neurons. Total number of parameters 23 581. Number of training records 10 000 000. Number of validation records 2 000 000. Training time is 8 hours and 5 minutes. The accuracy is measured by Pearson correlation coefficient which is near 82%.

3.2 C++ KAN

Same number of training and validation records. Training time 2 hours and 57 minutes. Accuracy is 85%. Number of parameters in a model 26 440.

Conclusion

Obviously, MATLAB code can also be elaborated for higher accuracy by making neuron network larger and more complex. In all cases when we tried it, we obtained similar accuracy for much longer training time. So, the conclusion for these quick experiments is that KAN and NN are close in accuracy, but KAN needs less time for training when Kaczmarz method is used.