The McCulloch-Pitts Neuron

The first computational neuron: add up binary inputs, compare with a threshold, and build logic gates.

Deep Learning- Fundamentals to Advanced Concepts

In 1943 Warren McCulloch (a neuroscientist) and Walter Pitts (a logician) proposed a deliberately simple model of a neuron. It is the starting point for everything in this module.

The model

Every input is binary: 0 (off) or 1 (on). The output is binary too. The neuron has two stages.

McCulloch-Pitts neuron: binary inputs are summed by g, then f compares the sum with a threshold theta
g aggregates, f decides.
  1. Aggregation, gg. Add up the inputs: g(x)=x1+x2+⋯+xng(x) = x_1 + x_2 + \dots + x_n. Since each input is 0 or 1, this simply counts how many inputs are on.
  2. Decision, ff. Fire if the count reaches a threshold θ\theta:
y=f(g(x))={1if g(x)≥θ0otherwisey = f(g(x)) = \begin{cases} 1 & \text{if } g(x) \ge \theta \\ 0 & \text{otherwise} \end{cases}

The threshold is chosen by hand. A lazy person who goes out only when many things line up has a high θ\theta. Someone who is always looking for an excuse to go out has a low one.

Excitatory and inhibitory inputs

Inputs come in two kinds.

  • Excitatory inputs push toward firing. They are the ones that get summed.
  • Inhibitory inputs are a veto. If any inhibitory input is 1, the neuron outputs 0 no matter what the others say.

Think of deciding whether to go to a film. "Good film is showing" and "it's not raining" are excitatory. "I have a high temperature" is inhibitory: if it is on, you stay home whatever else is true.

Building logic gates

A Boolean function takes binary inputs and gives a binary output. "Implementing" one means the neuron produces exactly the function's truth table.

FunctionSetupWhy
AND (n inputs)threshold θ=n\theta = nevery input must be on
OR (n inputs)threshold θ=1\theta = 1one input is enough
NOTone inhibitory input, θ=0\theta = 0input 1 vetoes the output to 0. Input 0 leaves a sum of 0, which meets θ=0\theta = 0, so it fires
Always 1 (tautology)threshold θ=0\theta = 0the sum is never below 0

With three inputs and θ=3\theta = 3, the input (1,1,0)(1, 1, 0) sums to 2, which is below 3, so AND correctly outputs 0.

Try it yourself
Thresholding simulator →

Change the threshold and watch which inputs make the neuron fire.

In code

def mp_neuron(excitatory_inputs, inhibitory_inputs, threshold):
    """Simulates a McCulloch-Pitts neuron."""
    # 1. An active inhibitory input vetoes everything
    if sum(inhibitory_inputs) > 0:
        return 0

    # 2. Aggregation g(x): count the active excitatory inputs
    aggregation = sum(excitatory_inputs)

    # 3. Decision f(g(x)): compare with the threshold
    return 1 if aggregation >= threshold else 0


def AND_gate(x1, x2):
    return mp_neuron([x1, x2], [], threshold=2)

def OR_gate(x1, x2):
    return mp_neuron([x1, x2], [], threshold=1)

def NOT_gate(x1):
    return mp_neuron([], [x1], threshold=0)


print(AND_gate(1, 1), AND_gate(1, 0))   # 1 0
print(OR_gate(0, 0), OR_gate(0, 1))     # 0 1
print(NOT_gate(1), NOT_gate(0))         # 0 1

What to notice

  • The model can only take binary inputs.
  • The threshold has to be worked out by a person for each function.
  • Every input counts equally.

The next lessons fix all three, and ask a geometric question about what this neuron can and cannot do.

EasyMP neuron

What threshold makes a 4-input MP neuron compute AND? What about OR?

MediumMP neuron

Build NOR of two inputs with one MP neuron.

MediumMP neuronTheory

Why can a neuron with only excitatory inputs never implement NOT?