Content
Download KAN Professional
This code shows several computational improvements.
Replacement of basis functionsThe training time depends on the choice of basis functions. For example, using of piecewise linear functions is about 100 times faster than splines. This code shows how to do main training by using piecewise linear model and replace it by splines at the end of training.Gradual upgrade of the modelWhen using piecewise linear model, its accuracy depends on the number of linear segments. The code shows how to change number of linear segments and tune the accuracy while prevent overfitting.Here is one example of code execution. Dataset is 10 000 records, 5 features, training set is 6200 records and others are validation set. The target is generated by complex algebraic formula, so simple polynomial models are failing and only deep machine learning can achieve good accuracy.  The training starts from piecewise linear model and performs 40 epochs for less than 0.2 seconds, then the model is replaced by splines, where each piecewise linear function is adequately replaced by linear combination of basis spline functions. Then some tuning of spline model completes training. The relative root mean square error is near half percent. Total time of training is less than a second. The code optimization resources are not yet exhausted. I can make all running faster, but the code will not be user friendly. |