Content


Stochastic models

The reasonable expectation from the stochastic model $M$ $$ M(x_1, x_2, x_3, ... , x_n)$$ is return of 1D distribution, which depends on features $x_1, x_2, x_3, ... , x_n$. It is not multivariate distribution by its nature. All features are always given, they can't be variables (they are provided) and result or target is random, but fit some distribution.

Many applications claim the capability of building stochastic models, but in reality they provide fake result. After finding that deterministic modeling is not possible and there is always a random error, they offer some very approximate and frequently not accurate estimation of either variance or confidence interval, assuming normal distribution of targets. And the logic sounds usually as follows: "we don't know anything, but we can see random errors, so we assume they are normal and try to estimate confidence interval at least for entire dataset or, if possible, find how it depends on features". That is kind of trivial task and not what we try to solve on this site.

The goal is to provide the code which supports recognition of input dependent multimodality, i.e. the probability densities with multiple peaks like it is shown below
The validation of modeled probability density is called calibration. We provide calibrating tests estimating the accuracy of probabilistic modeling.

Types of uncertainty

1. Aleatoric
Example: an income of working for paycheck individual depending on demographic factors, such as education, profession, age, sex, marital status and so on.

2. Epistemic
Example:


The distance between two circle centers when circles are defined by three points (not on the straight line). The features are points coordinates, the target is distance. Each three points (not on the straight line) define circle and its center point in a unique way, so 12 features uniquely define distance, but modeling it as a black box is a problem. Anyone can try it using freely available MLPs and see the spectacular and miserable failure. The errors are huge. Such demos are very disappointing for regular users and raise questions.

I think it is clear on intuitive level. Aleatoric means that target is random, but has stable distribution. Epistemic means target is exact, but either dataset is too short or we need more information about the nature of the object and some feature restructuring.

Probabilistic classification

The probabilistic classification model simply provides probability for each class, while deterministic classification returns the class label.