Content

Piecewise linear KAN

Reference to piecewise linear KAN

The code for piecewise linear version is written in C#. The logic is explained in the previous section.

The accuracy for piecewise linear version is only slightly lower than for spline version. The code has a self test on three different formulas:

$$ z = exp(sin(\pi x) + y^2), $$ $$ z = exp((sin(\pi (x_1^2 + x_2^2) ) + sin(\pi (x_3^2 + x_4^2) ))/2), $$ $$ y = \frac{2 + 2 x_3}{3\pi} \left[ {arctan}\left( 20( x_1 - \frac{1}{2} + \frac{x_2}{6})\exp\left(x_5\right) \right) + \frac{\pi}{2} \right] + \frac{2 + 2 x_4}{3\pi} \left[ {arctan}\left( 20( x_1 - \frac{1}{2} - \frac{x_2}{6})\exp\left(x_5\right) \right) + \frac{\pi}{2} \right], $$
and the relative errors are the fraction of percent. But piecewise linear version is designed mostly for modeling of physical systems. They always have some unobserved factors, they are not perfectly stationary, that means the systems are changing their properties during collecting of the data, they have at least small stochastic factors and recordings errors. On all these reasons accurate deterministic model simply not exists and 5 to 10 percent training errors are considered normal.

However, when testing the training algorithm, using exact formulas is justified, because it shows the potential accuracy. The choice of the formula can be identified in the code:

            //Formula1 f = new Formula1();
            //(List inputs, List target) = f.GenerateData(1000);

            //Formula2 f = new Formula2();
            //(List inputs, List target) = f.GenerateData(3000);

            Formula3 f = new Formula3();
            (List inputs, List target) = f.GenerateData(10000);

Each choice needs corresponing model options:

            //This is for Formula1
            //int nEpochs = 101;
            //int nModels = 5;
            //double zmin = targetMin / nModels;
            //double zmax = targetMax / nModels;
            //KANAddendPL[] addends = new KANAddendPL[nModels];
            //for (int i = 0; i < nModels; ++i)
            //{
            //    addends[i] = new KANAddendPL(xmin, xmax, zmin, zmax, 6, 8, 0.1, 0.1);
            //}

            //This is for Formula2
            //int nEpochs = 401;
            //int nModels = 9;
            //double zmin = targetMin / nModels;
            //double zmax = targetMax / nModels;
            //KANAddendPL[] addends = new KANAddendPL[nModels];
            //for (int i = 0; i < nModels; ++i)
            //{
            //    addends[i] = new KANAddendPL(xmin, xmax, zmin, zmax, 8, 8, 0.1, 0.1);
            //}

            //This is for Formula3
            int nEpochs = 101;
            int nModels = 11;
            double zmin = targetMin / nModels;
            double zmax = targetMax / nModels;
            KANAddendPL[] addends = new KANAddendPL[nModels];
            for (int i = 0; i < nModels; ++i)
            {
                addends[i] = new KANAddendPL(xmin, xmax, zmin, zmax, 5, 12, 0.01, 0.01);
            }

KANAddendPL is this term $$ \Phi\left(\sum_{p=1}^{n} \phi_{p}(x_{p})\right). $$ The integers passed to KANAddendPL constructor are numbers of points in piecewise linear functions of inner and outer operators. They may be different for each addend. The code also draws the identified functions. Two of them are shown below as example:

This code is not optimized for performance, but it is quick enough. For example, the first function is taken from MIT Python code example. It takes 23 seconds for training on Python. In this C# code the time is 0.06 of a second.