The maximum likelihood estimator of normal distribution is a cornerstone of statistical inference, offering a principled way to estimate the mean and variance of a Gaussian population from observed data. Whether you are analyzing test scores, measuring physical constants, or building machine learning models, understanding how maximum likelihood estimation works for the normal distribution equips you with a tool that appears everywhere from introductory statistics courses to advanced data science pipelines. At its core, this method asks a simple but powerful question: given a sample of data, which values of the population parameters make the observed data most probable?
What Is Maximum Likelihood Estimation?
Maximum likelihood estimation, often abbreviated as MLE, is a method used to estimate the parameters of a statistical model. On top of that, instead of relying on intuition or arbitrary rules, MLE selects parameter values that maximize the likelihood function. The likelihood function measures how probable the observed sample is, given specific parameter values Not complicated — just consistent. Nothing fancy..
In plain terms, if you flip a coin ten times and observe seven heads, maximum likelihood intuition suggests that the probability of heads is likely close to 0.7 rather than 0.Day to day, 5, because 0. 7 makes your observed outcome more probable than 0.5 does. When applied to the normal distribution, the same logic helps us identify the population mean and variance that best explain the data we have collected.
Understanding the Normal Distribution
The normal distribution, also called the Gaussian distribution, is one of the most widely used probability distributions in science and engineering. It is fully described by two parameters:
- μ (mu): the population mean, which determines the center of the bell curve.
- σ² (sigma squared): the population variance, which determines the spread or width of the curve.
The probability density function (PDF) of a normal distribution is written as:
$f(x \mid \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right)$
This formula tells us the relative likelihood of observing a particular value x, assuming the data comes from a normal distribution with mean μ and variance σ². Here's the thing — in real-world applications, we rarely know these parameters in advance. Instead, we collect a random sample and use MLE to estimate them.
The Likelihood Function for a Normal Distribution
Suppose we have a sample of n independent observations: $x_1, x_2, ..., x_n$. Because the observations are assumed to be independent, the joint likelihood of the entire sample is the product of the individual probability densities:
$L(\mu, \sigma^2) = \prod_{i=1}^{n} \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left(-\frac{(x_i - \mu)^2}{2\sigma^2}\right)$
This product can be cumbersome to work with directly. Multiplying many small probability values often leads to numerical underflow, and differentiating a product is algebraically difficult. To simplify the problem, statisticians take the natural logarithm of the likelihood function.
You'll probably want to bookmark this section That's the part that actually makes a difference..
that maximizes the likelihood also maximizes the log-likelihood function. Substituting the normal probability density into this transformed expression gives:
[ \ell(\mu, \sigma^2) = -\frac{n}{2}\ln(2\pi\sigma^2) - \frac{1}{2\sigma^2}\sum_{i=1}^{n}(x_i - \mu)^2 ]
Because the term (-\frac{n}{2}\ln(2\pi)) does not depend on the parameters, maximizing the log-likelihood is equivalent to maximizing:
[ \ell(\mu, \sigma^2) = -\frac{n}{2}\ln(\sigma^2) - \frac{1}{2\sigma^2}\sum_{i=1}^{n}(x_i - \mu)^2 ]
To find the maximum, we take partial derivatives with respect to both (\mu) and (\sigma^2) and set them equal to zero.
First, differentiating with respect to (\mu):
[ \frac{\partial \ell}{\partial \mu} = \frac{1}{\sigma^2}\sum_{i=1}^{n}(x_i - \mu) = 0 ]
Solving this gives:
[ \sum_{i=1}^{n}(x_i - \mu) = 0 ]
[ \mu = \bar{x} ]
So the maximum likelihood estimate for the mean is simply the sample mean.
Next, differentiating with respect to (\