Multilayer networks: a forward pass that computes XOR
A multilayer perceptron (MLP) arranges neurons in layers. Each layer computes new values from the previous layer's outputs with a weight for every connection, a bias for every neuron, and an activation function. The forward pass computes the output by passing the inputs through the layers in order. The hidden layers, those between the input and the output, let the network build intermediate features.
Python
def dense(inputs, weights, biases, activation):
return [
activation(sum(w * x for w, x in zip(row, inputs)) + bias)
for row, bias in zip(weights, biases)
]
def step(z):
return 1 if z > 0 else 0
hidden_weights = [[1, 1], [1, 1]]
hidden_biases = [-0.5, -1.5]
output_weights = [[1, -1]]
output_biases = [-0.5]
for x in ((0, 0), (0, 1), (1, 0), (1, 1)):
hidden = dense(x, hidden_weights, hidden_biases, step)
output = dense(hidden, output_weights, output_biases, step)[0]
print(x, hidden, output)