Neurons, Weights and Bias Explained

Neurons, Weights and Bias: The Smallest Piece of AI

A note on tone

Like my other notes, this is my own understanding written in plain words. For the full picture, take a proper deep learning course. Think of this as a friendly map, not the textbook.

This is part 1 of a series: Neurons (this note) → Neural Networks Explained → Activation Functions and GELU → Transformers Explained → Multi-Head Attention Explained.

Every giant AI model you have heard of is built from one tiny repeated piece. If you understand that piece, the rest is just how many and how they are wired. That piece is the neuron.

Where the idea came from

Your brain has neurons. Each one receives signals from others, adds them up, and if the total is strong enough, it fires a signal onward.

In 1943, McCulloch and Pitts turned that into math: a unit that takes inputs, adds them, and outputs on or off. Sound familiar? It is a close cousin of the machines in my note on State Machines Explained. In 1958, Frank Rosenblatt added the idea that the strengths could be learned, and called it the perceptron.

An honest caveat

An artificial neuron is a very loose sketch of a real one. Real neurons are far more complicated. The name stuck, but think of it as a small function with adjustable numbers, not a brain cell.

The anatomy of one neuron

A neuron does exactly four things:

Takes inputs $x_{1}, x_{2}, x_{3}$ (numbers).
Multiplies each input by its own weight $w_{1}, w_{2}, w_{3}$ .
Adds everything up, plus one extra number called the bias $b$ .
Bends the result with an activation function $f$ and outputs it.

z = w_{1} x_{1} + w_{2} x_{2} + w_{3} x_{3} + b a = f (z)

In shorthand, with vectors, that first part is a dot product: $z = w \cdot x + b$ . (If you did Matrix Multiplication Explained, this is a row times a column.)

Weights: how much each input matters

A weight is a dial that says how much do I care about this input?

Big positive weight: this input pushes the neuron strongly toward "yes".
Weight near zero: this input is basically ignored.
Negative weight: this input pushes toward "no".

Say a neuron decides whether I should go out today. Its inputs are nice weather, is it raining, and is a friend coming. It would learn a positive weight for the friend, a negative one for the rain.

Bias: how easily it says yes

The bias is a number added no matter what the inputs are. It shifts the whole decision.

A negative bias makes the neuron picky: it needs strong evidence before saying yes.
A positive bias makes it eager: it says yes even with weak evidence.

A word collision

This "bias" is just an offset, like the constant in $y = m x + c$ . It has nothing to do with unfairness or social bias, which is a totally different topic that shares the name.

Real numbers, one neuron

Let me plug in numbers so it stops being abstract.

Inputs: nice weather = 1, raining = 0, friend coming = 1
Weights: 0.5, −0.6, 0.8
Bias: −1

z = (1) (0.5) + (0) (- 0.6) + (1) (0.8) - 1 = 0.3

Then a sigmoid squeezes 0.3 into a value between 0 and 1: $σ (0.3) \approx 0.57$ . Above 0.5, so the neuron says go!

Notice what happened. If there were no friend coming (input 0), then $z = 0.5 - 1 = - 0.5$ , and the sigmoid gives about 0.38. The neuron says stay in. The weights and bias are the entire "opinion" of the neuron.

import math

def neuron(x, w, b):
    z = sum(xi * wi for xi, wi in zip(x, w)) + b
    return 1 / (1 + math.exp(-z))     # sigmoid

print(neuron([1, 0, 1], [0.5, -0.6, 0.8], -1))   # 0.574...
print(neuron([1, 0, 0], [0.5, -0.6, 0.8], -1))   # 0.377...

What a neuron is really doing: drawing a line

Here is the picture that made it click for me. With two inputs, plot every possible input as a point on a page. The equation $w \cdot x + b = 0$ is a straight line. Points on one side make the neuron say yes, points on the other side say no.

The weights turn the line. They decide its direction (the arrow $w$ points toward the "yes" side).
The bias slides the line without turning it.

That is why a single neuron can only separate things with a straight cut. Set the weights and bias right, and it sorts cats from dogs, spam from not-spam, if a straight cut is enough. When it is not, we need many neurons and bends. That is the next note: Neural Networks Explained.

Where do the weights come from?

Nobody sets them by hand. They start as small random numbers, and the network learns them. The loop is simple: guess, measure how wrong it was, and nudge every weight slightly in the direction that makes it less wrong. Repeat millions of times.

So when you hear "a model with 70 billion parameters", the parameters are just weights and biases. That is the whole model: a very long list of dial settings.

Key takeaway

A neuron is a weighted vote plus a bias, then a bend. Weights say who matters. Bias says how hard it is to convince. Everything else in deep learning is built by wiring many of these together.

Try it yourself

Change the weights in the code above and watch the output change
Find a bias that makes the neuron say yes even when all inputs are 0
With two inputs, sketch the line $x_{1} + x_{2} - 1 = 0$ and test a few points
Set every weight to 0. What does the neuron output? What does that say about bias?

Next: Neural Networks Explained