In 1943 Warren McCulloch (a neuroscientist) and Walter Pitts (a logician) proposed a deliberately simple model of a neuron. It is the starting point for everything in this module.
The model
Every input is binary: 0 (off) or 1 (on). The output is binary too. The neuron has two stages.
- Aggregation, . Add up the inputs: . Since each input is 0 or 1, this simply counts how many inputs are on.
- Decision, . Fire if the count reaches a threshold :
The threshold is chosen by hand. A lazy person who goes out only when many things line up has a high . Someone who is always looking for an excuse to go out has a low one.
Excitatory and inhibitory inputs
Inputs come in two kinds.
- Excitatory inputs push toward firing. They are the ones that get summed.
- Inhibitory inputs are a veto. If any inhibitory input is 1, the neuron outputs 0 no matter what the others say.
Think of deciding whether to go to a film. "Good film is showing" and "it's not raining" are excitatory. "I have a high temperature" is inhibitory: if it is on, you stay home whatever else is true.
Building logic gates
A Boolean function takes binary inputs and gives a binary output. "Implementing" one means the neuron produces exactly the function's truth table.
| Function | Setup | Why |
|---|---|---|
| AND (n inputs) | threshold | every input must be on |
| OR (n inputs) | threshold | one input is enough |
| NOT | one inhibitory input, | input 1 vetoes the output to 0. Input 0 leaves a sum of 0, which meets , so it fires |
| Always 1 (tautology) | threshold | the sum is never below 0 |
With three inputs and , the input sums to 2, which is below 3, so AND correctly outputs 0.
Change the threshold and watch which inputs make the neuron fire.
In code
def mp_neuron(excitatory_inputs, inhibitory_inputs, threshold):
"""Simulates a McCulloch-Pitts neuron."""
# 1. An active inhibitory input vetoes everything
if sum(inhibitory_inputs) > 0:
return 0
# 2. Aggregation g(x): count the active excitatory inputs
aggregation = sum(excitatory_inputs)
# 3. Decision f(g(x)): compare with the threshold
return 1 if aggregation >= threshold else 0
def AND_gate(x1, x2):
return mp_neuron([x1, x2], [], threshold=2)
def OR_gate(x1, x2):
return mp_neuron([x1, x2], [], threshold=1)
def NOT_gate(x1):
return mp_neuron([], [x1], threshold=0)
print(AND_gate(1, 1), AND_gate(1, 0)) # 1 0
print(OR_gate(0, 0), OR_gate(0, 1)) # 0 1
print(NOT_gate(1), NOT_gate(0)) # 0 1
What to notice
- The model can only take binary inputs.
- The threshold has to be worked out by a person for each function.
- Every input counts equally.
The next lessons fix all three, and ask a geometric question about what this neuron can and cannot do.