Content
- About this project
Stable code and as generic as possible for the moment
This code is recommended as end user friendly and something that can be easily customized. The example is vector target. The features are coordinates of vertices of random triangles, the targets are lengths of the medians.  In this code, I reduced everything for two classes $U$ and $Layer$, about 200 lines total without usage of 3rd party library. Training set is 10,000 records, processing near 0.7 seconds, it is about 3 times faster than MATLAB CPU script, but comparing performances when it is running faster than a second is not relevant. There are some other performance tests, where training lasts for hours. For this one we can say, it is about as quick as MATLAB. The reason for two classes is an attempt to approach classical theoretical concept. KAN is explained differently by different people, although programmatic implementation may be similar. Our explanation used Urysohn integral equation, which is only our preference. There is nothing wrong with other explanations, it is a matter of choice. In our vision KAN is a sequence of layers, where each of them is defined by Urysohn equation with 3D kernel $$ u(t) = \int_{s_1}^{s_2} F[z(s),s,t]ds, \quad t \in [t_1, t_2]. $$ Pavel Urysohn, in 1924, considered a problem of its solution relative to $z(s)$ provided $u(t)$ and $F[z,s,t]$. KAN training is the different task of building kernel provided set of $u$ and $z$. The network is not reduced to a single layer, it is a chain of layers, where each output $u$ is an input in another equation, i.e. sitting under kernel. In the code we are using quadrature of above expression $$ u_i = \sum_{j = 1}^{n} f_{i,j}(z_j), \quad i=1,2,3...m. $$ One particular scalar $u$ depends on a set of functions $$ u = \sum_{j = 1}^{n} f_{j}(z_j), $$ which is simplified urysohn or matrix (each function numerically is a vector), set of such matrices is a tensor or layer object in a program. I hope everything else is clear from the code. |