Benchmarks

This is collection of projects and datasets for benchmarking. It usually reproduces previously published as benchmark result for comparison.

This is not a competition. The goal is to find if KAN is capable to produce comparable results to those which are published as benchmarks. The comparison makes sense but competition does not. The competitor who challenges already published benchmark is in a big advantage. He has unlimited time and can tune code as long as necessary. Also some datasets may have certain hidden properties that can be detected and used, but that dataset specific logic will not help with another data. However, the comparison is necessary, we need to have a big picture, and difference in a few percent does not matter.

I already found several discussions in internet forums saying that KAN is significantly slower, which is very far from the truth. All my experimenting and comparisons from the start of this research show that performance is the same or better than MLP.

To backup this statement I show one example right here. The dataset has 5 features and 10 000 records, the target is computed by formula: $$ y = \frac{2 + 2 x_3}{3\pi} \left[ {arctan}\left( 20( x_1 - \frac{1}{2} + \frac{x_2}{6})\exp\left(x_5\right) \right) + \frac{\pi}{2} \right] + \frac{2 + 2 x_4}{3\pi} \left[ {arctan}\left( 20( x_1 - \frac{1}{2} - \frac{x_2}{6})\exp\left(x_5\right) \right) + \frac{\pi}{2} \right]. $$
This algebraic expression is chosen to provide certain complexity, the target can't be easily modeled by linear or simple polynomial models, we need some form of AI. Below is console output for C++ version of KAN. The code does not have any specific C accelerators, such as embedded assembly blocks or concurrent training of the different pieces of model and merging. It is straight forward single threaded C++ implementation of the concept. The training time is below 0.1 second, the accuracy is above 99%. Any experienced MLP software user can confirm that this performance is at least somewhere near MLP. This version can be found and tested in Download section.