When classifying data with logistic classification using the maximum likelihood method, the likelihood function represents the probability of observing the given data under the model parameters.
Upper Bound of the Likelihood
- The likelihood is a probability value, so its upper bound is 1. This means the likelihood function can never exceed 1 because it is a probability measure over the observed data
Is This Upper Bound Attainable?
- Achieving a likelihood of exactly 1 would imply the model perfectly predicts all observed data points with certainty, which is generally not attainable in practice. Real-world data usually contain noise or variability that prevents perfect prediction.
- Instead, the maximum likelihood estimate (MLE) finds parameter values that maximize the likelihood, often resulting in a value close to but less than 1
Additional Context
- Logistic regression models the probability of class membership using a sigmoid function, and the MLE approach optimizes parameters to maximize the product of these probabilities across all data points (or equivalently, the sum of log-likelihoods).
- The likelihood function for logistic regression is bounded between 0 and 1 because it is a product of probabilities for each data point belonging to its observed class.
- The MLE is consistent and asymptotically efficient, meaning with enough data it converges to the true parameter values, but it does not guarantee a likelihood of exactly 1 for finite samples
In summary, the upper bound of the likelihood in logistic classification under the maximum likelihood method is 1, but this value is typically not attainable in practice due to data noise and model limitations. Instead, the goal is to find parameter values that maximize the likelihood as much as possible.
