When classifying data with logistic classification using the maximum likelihood method, the likelihood function represents the probability of observing the given data under the model parameters.
Upper Bound of the Likelihood
- The likelihood function for logistic regression is bounded between 0 and 1 because it is a product of probabilities, each of which lies in
- The theoretical upper bound of the likelihood is 1, which would correspond to the model perfectly predicting all observed data points with probability 1
Attainability of the Upper Bound
- Achieving a likelihood of exactly 1 is generally not attainable in practice. This would require the model to perfectly classify every data point without error, which is rare and often unrealistic due to noise, overlapping classes, or model limitations.
- In practice, the maximum likelihood estimate (MLE) finds parameter values that maximize the likelihood as much as possible, but the likelihood typically remains less than 1
- The likelihood function may increase during optimization but may never reach the supremum value of 1, especially with finite and noisy data.
Summary
- Upper bound of likelihood: 1 (perfect fit).
- Is it attainable? Usually no; it is a theoretical maximum rarely achieved in real data scenarios.
Thus, logistic regression aims to find parameters that maximize the likelihood, but the maximum likelihood value is typically less than 1 and depends on the data and model fit quality
