What a Neuron Draws: Lines, Planes and Linear Separability

A threshold neuron splits its input space in two. Seeing that split explains what it can compute and where it will fail.

Which Boolean functions can one MP neuron implement? To answer that we stop doing arithmetic and look at geometry.

Two inputs: a line

Take OR with two inputs. The neuron fires when $x_1 + x_2 \ge 1$ . Replace the inequality with an equality and you get $x_1 + x_2 = 1$ , which is a straight line. It passes through $(1, 0)$ and $(0, 1)$ .

Only four input points exist, since the inputs are binary: $(0,0)$ , $(0,1)$ , $(1,0)$ and $(1,1)$ . Put them on a plane.

Two squares with four binary input points. The OR line leaves only the origin on one side. The AND line leaves only the point (1,1) on the firing side. — Changing the threshold slides the line. OR puts only (0,0) on the silent side. AND puts only (1,1) on the firing side.

The line divides the plane into two half-spaces:

where $x_1 + x_2 \ge 1$ , the positive half-space, the neuron fires (output 1),
where $x_1 + x_2 < 1$ , the negative half-space, it stays silent (output 0).

For OR, three points fall in the positive half-space and only $(0, 0)$ in the negative. That is exactly OR's truth table.

For AND the inequality is $x_1 + x_2 \ge 2$ . Only $(1, 1)$ satisfies it. For the always-1 function, $\theta = 0$ and all four points are on the firing side.

Do not say above and below

It is tempting to say "the neuron fires for points above the line". That depends on how the line is written and which way it leans. The reliable definition is the inequality: a point is in the positive half-space if the expression is $\ge 0$ and in the negative half-space if it is $< 0$ .

Three inputs: a plane

With three inputs the same idea gives a plane. For 3-input OR, the plane $x_1 + x_2 + x_3 = 1$ separates the origin, which is the only point below it, from the other seven corners of the cube, which are on or above it.

For $n$ inputs you get a hyperplane. You cannot draw it, but the statement is the same: points on one side give 1 and points on the other give 0.

Try it yourself

Thresholding simulator →

Move the threshold and watch the dividing line and the classification change.

Linear separability

A Boolean function is linearly separable if some line (or plane, or hyperplane) puts every input with output 1 on one side and every input with output 0 on the other.

A single MP neuron can implement exactly the linearly separable Boolean functions, and no others. AND, OR, NOT and the always-1 function all qualify.

That raises two questions. Are there Boolean functions that are not linearly separable? And what do we do about them? "Where One Perceptron Fails" answers the first, and "A Network of Perceptrons" the second.

EasyGeometry

For a 2-input MP neuron with threshold 2, which inputs are in the positive half-space?

MediumGeometry

Two inputs: a line

Three inputs: a plane

Linear separability

For a 2-input MP neuron with threshold 2, which inputs are in the positive half-space?

Why is a point above the line not always in the positive half-space?