Content
|
Benchmarks
This is collection of projects and datasets for benchmarking. It usually reproduces previously published as benchmark result for
comparison.
This is not a competition.
The goal is to find if KAN is capable to produce comparable results to those which are published as benchmarks. The comparison makes sense
but competition does not. The competitor who challenges already published benchmark is in a big advantage. He has unlimited time and
can tune code as long as necessary. Also some datasets may have certain hidden properties that can be detected and used, but that
dataset specific logic will not help with another data. However, the comparison is necessary, we need to have a big picture, and
difference in a few percent does not matter.
I already found several discussions in internet forums saying that KAN is significantly slower, which is very far from the truth. All
my experimenting and comparisons from the start of this research show that performance is the same or better than MLP.
To backup this statement I show one example right here. The dataset has 5 features and 10 000 records, the target is computed
by formula:
$$
y = \frac{2 + 2 x_3}{3\pi} \left[ {arctan}\left( 20( x_1 - \frac{1}{2} + \frac{x_2}{6})\exp\left(x_5\right) \right) + \frac{\pi}{2} \right] + \frac{2 + 2 x_4}{3\pi} \left[ {arctan}\left( 20( x_1 - \frac{1}{2} - \frac{x_2}{6})\exp\left(x_5\right) \right) + \frac{\pi}{2} \right].
$$
This algebraic expression is chosen to provide certain complexity, the target can't be easily modeled by linear or simple polynomial models, we need some form of AI.
Below is console output for C++ version of KAN. The code does not have any specific C accelerators, such as embedded assembly blocks or concurrent training of the different
pieces of model and merging. It is straight forward single threaded C++ implementation of the concept. The training time is below 0.1 second,
the accuracy is above 99%. Any experienced MLP software user can confirm that this performance is at least somewhere near MLP.
This version can be found and tested in Download section.
|
|
|