Java: a two-layer network that computes XOR

Combine two hidden neurons so that a network solves the problem a single perceptron cannot.

Java

import java.util.Locale;

class Main {
  static double sigmoid(double z) {
    return 1.0 / (1.0 + Math.exp(-z));
  }

  public static void main(String[] args) {
    int[][] inputs = {{0, 0}, {0, 1}, {1, 0}, {1, 1}};
    for (int[] in : inputs) {
      double or = sigmoid(20 * in[0] + 20 * in[1] - 10);
      double and = sigmoid(20 * in[0] + 20 * in[1] - 30);
      double out = sigmoid(20 * or - 20 * and - 10);
      System.out.printf(Locale.ROOT, "%d XOR %d = %.3f%n", in[0], in[1], out);
    }
  }
}

Output

0 XOR 0 = 0.000
0 XOR 1 = 1.000
1 XOR 0 = 1.000
1 XOR 1 = 0.000

Explanation

A hidden layer lets a network build new features. One hidden neuron can learn OR and another can learn AND. The output neuron then fires when OR is on but AND is off, which is exactly XOR.

The program uses the sigmoid function, which maps any number smoothly into the range 0 to 1. Large weights make the neurons behave almost like on/off switches. Computing the outputs from given weights is called a forward pass. Training would find such weights automatically by backpropagation.

Explore the dry run

Guided walkthroughReady

1import java.util.Locale;

3class Main {

4 static double sigmoid(double z) {

5 return 1.0 / (1.0 + Math.exp(-z));

6 }

8 public static void main(String[] args) {

9 int[][] inputs = {{0, 0}, {0, 1}, {1, 0}, {1, 1}};

10 for (int[] in : inputs) {

11 double or = sigmoid(20 * in[0] + 20 * in[1] - 10);

12 double and = sigmoid(20 * in[0] + 20 * in[1] - 30);

13 double out = sigmoid(20 * or - 20 * and - 10);

14 System.out.printf(Locale.ROOT, "%d XOR %d = %.3f%n", in[0], in[1], out);

15 }

16 }

17}

Follow the example

Start to inspect the source, values, and output. These prepared steps explain the published sample; grouped loop steps describe multiple iterations.

View the complete dry-run table

Prepared trace of the published program. Some rows group repeated iterations.
What happens	State after step	Output so far
This hidden neuron is on when at least one input is 1.	`OR feature`	—
This one is on only when both inputs are 1.	`AND feature`	—
The output fires for OR but not AND.	`out ≈ OR and not AND`	—
The four outputs round to 0, 1, 1, 0.	`out ≈ OR and not AND`	0 XOR 0 = 0.000 0 XOR 1 = 1.000 1 XOR 0 = 1.000 1 XOR 1 = 0.000

Cost and common mistakes

O(1) per input for this fixed network; in general a forward pass costs about the number of weights.

Without a non-linear activation, stacking layers collapses into a single linear function and cannot compute XOR.
A sigmoid with very large inputs saturates, so its gradient becomes tiny and learning stalls.

Try it yourself

Why are two hidden neurons needed?

Reveal answer and reasoning

One neuron can draw only one line. Two hidden neurons give two features, OR and AND, and the output combines them with a third line.

Java: a two-layer network that computes XOR

Explanation

Follow the example

Cost and common mistakes

Try it yourself

Further reading