keras-adf provides implementations of probabilistic assumed density filtering
(ADF) based version of layers commonly used in neural networks. They are to be
used within the Tensorflow/Keras
framework. Unlike the standard (deterministic) Keras layers that propagate
point estimates, ADF layers propagate a probability distribution parametrized
by its mean and (co-)variance.
We think it is best to show the core concepts of the package by a simple exemplary demonstration.
For this let us define a simple feed-forward model with fully-connected ADF layers.
We begin by importing the relevant Keras and
>>> from tensorflow.keras.layers import Input >>> from tensorflow.keras.models import Model >>> from kerasadf.layers import Dense
Every model begins with its input. Since we propagate the mean and (co-)variance through the ADF layers we need two inputs instead of just one.
>>> mean_in = Input((32,)) >>> variance_in = Input((32,))
Next, we define a fully-connected hidden layer with rectified linear unit (ReLU) activations and a fully-connected output layer with no activation.
>>> mean_hidden, variance_hidden = Dense(64, activation="relu")([mean_in, variance_in]) >>> mean_out, variance_out = Dense(1)([mean_hidden, variance_hidden])
The complete model can now be assembled exactly like any other Keras model.
>>> model = Model([mean_in, variance_in], [mean_out, variance_out])
We have defined a dense feed-forward neural network with 32-dimensional input space, a 64-dimensional hidden representation space, and a 1-dimensional output space. The model summary looks like this:
>>> model.summary() Model: "model" __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== input_1 (InputLayer) [(None, 32)] 0 __________________________________________________________________________________________________ input_2 (InputLayer) [(None, 32)] 0 __________________________________________________________________________________________________ dense (Dense) [(None, 64), (None, 2112 input_1 input_2 __________________________________________________________________________________________________ dense_1 (Dense) [(None, 1), (None, 1 65 dense dense ================================================================================================== Total params: 2,177 Trainable params: 2,177 Non-trainable params: 0 __________________________________________________________________________________________________
This model can now be used like any other Keras model: It can be trained after providing a loss function and an optimizer, it can be saved and restored, it can be used to make predictions, …