These types of models have a series of challenges; they need to have business sanity, need to operate in a low data environment where the number of data points may be less than the number of input variables, be able to handle nonlinear dynamics, be transparent in the model design i.e. not a “black box”.
Standard tools in data science and machine learning are unsuitable for this, as they tend to be data-hungry, prone to overfitting, and unable to express business dynamics. So to overcome this we use Bayesian inference methodology, which allows us to express business knowledge as priors and thus reduces the need for data to train the model without overfitting.
Some of the key benefits of this approach are:
- Accounts for business dynamics
- Automatically estimates proper uncertainty
- Robust decision support
- Faster generalization
- Works well with high dimensional data with few data points
- Pool information across variables in a hierarchical fashion, reducing further the need for data
- Avoids overfitting