Scalability test
For scalability test we need large dataset, large list of features and accuracy challenge. It has been
found that determinants of random matrices answer this need. In case of $3 * 3$ matrix, the model is simple and accurate and training needs only a few seconds,
so we show here more challenging example with $4 * 4$ matrix.
Training dataset is 100,000 records. Validation dataset is 10,000 to 20,000 records.
Experiment 1
1.1 MATLAB
We use MATLAB for comparison of accuracy and performance.
To achieve reasonable accuracy with these data in MATLAB we need multiple layers. We used network
of 5 layers with 10 neurons in each. The accuracy metric is Pearson correlation coefficient for computed
and actual targets of validation set. The MATLAB script is below:
clear variables;
close all;
%. number of training records
Ntrain = 100000;
%. number of validation records
Nval = 20000;
%. dimensionality of matrix
dim = 4;
%% generate data
N = Ntrain + Nval;
Nid = Ntrain + 1;
%. inputs
m = dim * dim;
x = rand(N, m);
%. outputs
y = zeros(N, 1);
for ii = 1:N
xx = reshape(x(ii, :).', dim, dim);
y(ii) = det(xx);
end
%. labels
lab = ones(N, 1);
lab(Nid:end) = 2;
identID = 1;
verifID = 2;
%%
Xtrain = x(1:(Nid-1),:);
Ytrain = y(1:(Nid-1));
Xtest = x(Nid:end,:);
Ytest = y(Nid:end);
tic;
Mdl = fitrnet(Xtrain, Ytrain, ...
'Standardize', 1, 'Activations', 'tanh', 'LayerSizes', ...
[10 10 10 10 10], 'IterationLimit', 1000, 'Verbose', 1);
Ypred = predict(Mdl, Xtest);
%. correlation coefficient
cc = corrcoef(Ytest, Ypred);
COR = cc(1,2);
disp(['Correlation coefficient: ', num2str(COR)]);
toc;
%%
plot(Ytest, Ypred, 's');
The results are below in table.
MATLAB
index |
accuracy |
training time (sec) |
1 |
0.94 |
104 |
2 |
0.94 |
111 |
3 |
0.94 |
110 |
MATLAB generated image showing correlation for computed and actual targets:
1.2 Classic Kolmogorov-Arnold representation (KAR)
The C++ implementation of Kolmogorov-Arnold is downloadable from the link on the top.
C++ KAR classic
index |
accuracy |
training time (sec) |
1 |
0.94 |
35 |
2 |
0.94 |
33 |
3 |
0.94 |
33 |
Experiment 2
2.1 MATLAB
Another comparison is between larger neural network and 3 layer KAN. The training is longer in both cases and accuracy is higher in both cases.
Below is MATLAB script:
% Parameters
num_train = 100000;
num_val = 2000;
input_size = 16; % 4x4 matrices flattened to vectors
% Generate random data
train_features = rand(num_train, 4, 4);
train_targets = zeros(num_train, 1);
for i = 1:num_train
train_targets(i) = det(squeeze(train_features(i, :, :)));
end
train_features = reshape(train_features, [num_train, input_size]);
val_features = rand(num_val, 4, 4);
val_targets = zeros(num_val, 1);
for i = 1:num_val
val_targets(i) = det(squeeze(val_features(i, :, :)));
end
val_features = reshape(val_features, [num_val, input_size]);
% Define the neural network
layers = [
featureInputLayer(input_size, 'Name', 'input')
fullyConnectedLayer(64, 'Name', 'fc1')
reluLayer('Name', 'relu1')
fullyConnectedLayer(128, 'Name', 'fc2')
reluLayer('Name', 'relu2')
fullyConnectedLayer(64, 'Name', 'fc3')
reluLayer('Name', 'relu3')
fullyConnectedLayer(32, 'Name', 'fc4')
reluLayer('Name', 'relu4')
fullyConnectedLayer(1, 'Name', 'output')
regressionLayer('Name', 'regression')
];
% Training options
options = trainingOptions('adam', ...
'MaxEpochs', 50, ...
'MiniBatchSize', 256, ...
'InitialLearnRate', 0.001, ...
'Shuffle', 'every-epoch', ...
'ValidationData', {val_features, val_targets}, ...
'ValidationFrequency', 30, ...
'Plots', 'training-progress', ...
'Verbose', false);
% Train the neural network
net = trainNetwork(train_features, train_targets, layers, options);
% Evaluate performance on validation data
predictions = predict(net, val_features);
corr_coeff = corr(predictions, val_targets);
fprintf('Correlation Coefficient on Validation Data: %.4f\n', corr_coeff);
It runs much longer (near 250 seconds) and builds a little bit more accurate model (correlation coefficient is near 97%).
2.2 Three layer Kolmogorov-Arnold network
The reference to the code is on the top of the page. The results are below in table.
C++ 3 layer KAN
index |
accuracy |
training time (sec) |
1 |
0.98 |
64 |
2 |
0.98 |
62 |
3 |
0.98 |
66 |
Experiment 3
This test is modeling of determinants of $5 * 5$ random matrices.
3.1 MATLAB
Neural network has 7 layers, each layer has 60 neurons. Total number of parameters 23 581. Number of training records 10 000 000.
Number of validation records 2 000 000. Training time is 8 hours and 5 minutes. The accuracy is measured by Pearson correlation coefficient
which is near 82%.
3.2 C++ KAN
Same number of training and validation records. Training time 2 hours and 57 minutes. Accuracy is 85%. Number of parameters in a model
26 440.
Conclusion
Obviously, MATLAB code can also be elaborated for higher accuracy by making neuron network larger and more complex. In all
cases when we tried it, we obtained similar accuracy for much longer training time. So, the conclusion for these quick
experiments is that KAN and NN are close in accuracy, but KAN needs less time for training when Kaczmarz method is used.
|
|