No announcement yet.

Why All Rating & Polling Methods Are Wrong

  • Filter
  • Time
  • Show
Clear All
new posts

  • Why All Rating & Polling Methods Are Wrong

    Date: Thu, 30 Nov 2000 04:34:55 -0600
    From: Mark Hopkins <>
    Subject: Why All Rating & Polling Methods Are Wrong

    A little incisive commentary for your enjoyment. And a challenge (which
    I believe is an impossible challenge).

    All rating & polling methods, no matter how conceived (be they defined
    by human
    voting, checking animal entrails or star charts, coin tosses,
    mathematical formula,
    computational algorithm or otherwise) will ALL fail in one of the
    following three ways:

    (a) The "Mount Union" effect. Exaggerated ranking of undefeated/untied

    Particularly, Division III Mount Union being placed in the midsts of
    Division I teams in recent years; but also cases like 1956 Oklahoma
    being ranked #1 even though it played only 1 (nowhere-near-contender)
    winning team in 1956!

    (b) The "Tennessee" effect. Exaggeration of schedule strength.

    Particular, in 1997, the SEC was so completely dominant over everyone
    else, as to be virtually an entire division onto itself. A rating
    method which puts too much emphasis on schedule strength will place
    Tennessee at #1 even over Nebraska. Nebraska won its bowl game against

    A similar thing happened in 1919 with the Western Conference.


    (c) It will be at odds with the Won-Loss percentage, even in
    round-robins where the "Pct." is the nearly universal standard
    definition of "Best" and (more importantly) in tight-enough schedules
    where its use is equally universal (like conferences).

    More generally and more to the point:

    (c') It will be at odds with the Won-Loss Percentage (even taking into
    account opposition strength) in not-so-loose schdules (like Division I-A

    There is simply no rating method which can avoid all 3 failures.