MLP

class MLP : public madspace::FunctionGenerator

Fully connected neural network as a compute-graph function.

A stack of layers dense layers of width hidden_dim with activation between them and a linear output layer. Used as the subnetwork of the Flow and DiscreteFlow adaptive samplers [1]. The weights and biases are compute-graph globals named with prefix. Call initialize_globals(context) once before use.

batch is the leading batch dimension.

Arguments

  • input – float, shape (batch, input_dim) – the network input.

Returns

  • output – float, shape (batch, output_dim) – the network output.

References

Public Types

enum Activation

Activation applied between the hidden layers.

Values:

enumerator relu
enumerator leaky_relu
enumerator elu
enumerator gelu
enumerator sigmoid
enumerator softplus
enumerator linear

Public Functions

MLP(std::size_t input_dim, std::size_t output_dim, std::size_t hidden_dim = 32, std::size_t layers = 3, Activation activation = leaky_relu, const std::string &prefix = "")
Parameters:

  • input_dim – Input width.

  • output_dim – Output width.

  • hidden_dim – Hidden-layer width.

  • layers – Number of layers.

  • activation – Activation between the hidden layers.

  • prefix – Prefix for the trainable global names.

inline std::size_t input_dim() const

Input width.

inline std::size_t output_dim() const

Output width.

void initialize_globals(ContextPtr context, std::optional<std::uint64_t> seed = std::nullopt) const

Register the network’s trainable parameters on context.

Parameters:

  • context – Context receiving the globals.

  • seed – Seed for the random initialization. If unset (the default), the initialization is non-deterministic.

inline std::string last_layer_bias_name() const

Global name of the final layer’s bias vector.

std::vector<std::string> global_names() const

Global names of all trainable parameters.