TypeScript: composing neural-network layers behind an interface

Build a small network from Dense and Sigmoid layers that share one Layer interface.

interface Layer {
  forward(input: number[]): number[];
}

class Dense implements Layer {
  private weights: number[][];
  private biases: number[];

  constructor(weights: number[][], biases: number[]) {
    this.weights = weights;
    this.biases = biases;
  }

  forward(input: number[]): number[] {
    return this.weights.map((row, j) => row.reduce((sum, w, i) => sum + w * input[i], this.biases[j]));
  }
}

class Sigmoid implements Layer {
  forward(input: number[]): number[] {
    return input.map((z) => 1 / (1 + Math.exp(-z)));
  }
}

class Sequential implements Layer {
  private layers: Layer[];

  constructor(layers: Layer[]) {
    this.layers = layers;
  }

  forward(input: number[]): number[] {
    return this.layers.reduce((value, layer) => layer.forward(value), input);
  }
}

const network = new Sequential([
  new Dense([[20, 20], [20, 20]], [-10, -30]),
  new Sigmoid(),
  new Dense([[20, -20]], [-10]),
  new Sigmoid(),
]);

for (const input of [[0, 0], [0, 1], [1, 0], [1, 1]]) {
  console.log(input[0] + " XOR " + input[1] + " = " + network.forward(input)[0].toFixed(3));
}

0 XOR 0 = 0.000
0 XOR 1 = 1.000
1 XOR 0 = 1.000
1 XOR 1 = 0.000

Explanation

Deep-learning libraries describe a network as a stack of layers. Each layer takes a vector and returns a vector, so they can all implement one interface with a forward method. A Sequential network then just feeds the output of each layer to the next.

A Dense layer computes a weighted sum plus a bias for each output. A Sigmoid layer applies the sigmoid function to each element. With the weights below, the first Dense layer builds OR-like and AND-like features, and the second combines them into XOR. The structure is the same as in frameworks such as PyTorch or TensorFlow.js, with the weights fixed instead of learned.

—

—

—

—

0 XOR 0 = 0.000
0 XOR 1 = 1.000
1 XOR 0 = 1.000
1 XOR 1 = 0.000

Cost and common mistakes

O(weights) per forward pass.

• Dense expects input vectors of the right length. Frameworks check shapes and report errors early; this sketch does not.
• Class fields are written out explicitly here instead of as constructor parameter properties, so the program also runs under type-stripping runtimes.
• A Sequential network is itself a Layer, so networks can be nested.

Try it yourself

Why does the second Dense layer use the weights 20 and −20?

Further reading

• TypeScript Handbook (external)
• Node.js API documentation (external)
• Deep Learning (Goodfellow, Bengio, Courville) (external)