What is multi-factor scoring?
Multi-factor scoring is a way to compare things based on aspects that are not directly comparable.
Search engines use this technique to present the most relevant search results. At Rangle, we built a learning plan generator using multi-factor scoring to rank study topics. And today, we will apply this method to find the cutest elephant!
Using multi-factor scoring to find the cutest elephant
Scoring
Can your items be sorted at all? Start by rejecting the null hypothesis. For example, in the case of elephants, we know that some elephants are cuter than others.
After you know sorting is possible, figure out what factors you would use to evaluate the items. For an elephant, it might be tusk size, ear size, weight, height, etc.
Simplistically, assign a weight to each factor, sum them, and get a score. Now you can sort items based on this score.
Factors are numeric representations of item metrics. Make sure you align factors in the same direction — either all metrics growing is good, or all metrics growing is bad. For an elephant, a bigger tusk might mean less cute, but bigger ears might mean more cute. You can multiply by -1 when calculating the factor value if necessary.
When mapping from metric to a factor, you can use any function to produce a number, even combining multiple metrics. For example, we don't want to consider overweight elephants cute, so we could discount high-BMI elephants when considering a weight factor.
When computing a sum, it is important that all factor values and weights belong on the same scale. Elephant weight is measured in thousands of kilograms, and tusks are measured in single-digit meters. Without normalization, small changes to the weight will always have a bigger impact than a big change to a tusk length.
When taking into account value transformations and normalization, you end up with something like this:
Optimizing weights
Factor weight determines the relative importance of a factor. The weights you pick directly determine the score, so paying attention to how you select them is crucial.
You can optimize the weights by comparing the scoring results to external data, such as a person's ranking of the cutest elephant and measuring the error between the two. Gradient descent is a mathematical optimization technique you can use to fine-tune the weights to produce the lowest error and the most accurate results.
Overall, multi-factor scoring is a powerful tool used in various industries to evaluate and rank elements based on a set of factors. If you're interested in learning more about this process and how you can apply it to your organization’s unique context, we invite you to watch our video on the topic for a more in-depth explanation.