E Pluribus Hugo, and other proposals (long post)
One of the reactions to this year's Hugo crisis has been the proposal of a tweak to the system for selecting Hugo Award finalists, with the goal of preventing slates from dominating as they did this year - as if you needed reminding, around 15% of voters got a clean sweep of the ballot in half a dozen categories, and would have done better if some of their own candidates had not withdrawn or declined nomination. There are a couple of other minor proposals as well, which I'll comment on at the end, but the big one goes by the name of E Pluribus Hugo (a pun on the Latin phrase e pluribus unum, "out of many, one", found on the Great Seat of the United States). All of the proposed changes to the system can be found on the Sasquan Business Meeting page. The E Pluribus Hugo proposal has been developed and discussed at the Making Light blog, and queries about its functioning should be addressed there rather than to me; that page also links to the extensive discussion from which it emerged.
The process as it is now: At present, the nominating pool of voters for the Hugo Awards consists of paid up members of this year's Worldcon, last year's and next year's, as of the nominations deadline which is usually in March. Each voter can nominate up to five candidates for each of the Hugo categories (and the John W. Campbell Award). Each of those votes counts the same. The votes are tallied, and in principle the top five, together with the No Award option, form the actual ballot (with a couple of wrinkles: if the fourth or fifth-placed works get less than 5% of total nominations, they are omitted; if there's a tie for fifth place, all the tied candidates get on).
The winner of each category is then chosen by transferable vote, the system variously known as instant-runoff voting, Alternative Vote, Australian ballot, or STV for a single seat. Voters list the options in order of preference; first-preference votes are tallied; if none of the candidates has more than half of the first-preference votes, votes are transfferred from the least popular candidates until there is one which does have more than half of the remaining votes. The winner of that count is then checked against No Award, to see which was preferred by more voters, ignoring all other finalists. There is no proposal to change that part of the process, and I personally like it as it is.
What's been proposed: The E Pluribus Hugo proposal retains the current system of inputs at the nominations stage from the voter's perspective. You still get to choose up to five candidates for each category. The difference is that there is a two-stage process. In the first stage, your vote gets split fractionally between your nominees. If you nominate five candidates in a particular category, your have cast 0.2 of a vote for each. If you nominate four, your vote counts for 0.25. If you nominate only one, your vote counts in full.
That is, at the first stage. The proposal then eliminates seriatim all those works which have not got enough support to get on the final ballot, and reallocates their votes as if they had not been nominated. That process continues until there are only enough left to fill the ballot. So if you nominate five books, and three of them are unpopular with other voters, your vote ends up being split with a value of 0.5 each for the other two. What usually happens to me is that I nominate five books, four of which nobody else has heard of, and my vote would then end up completely concentrated on the remaining one. This is obviously too complex to run by hand, so it will need to be done by computer (Hugo voting has anyway been administered by computer for many years).
The envisaged consequence is that if 15% of voters blindly follow the dictates of a slate, and the other 85% smear their votes around a number of other works, the votes of the 85% will end up being concentrated in such a way as to ensure that the slates do not get all five slots. On the other hand, a slate which is supported by 15-20% of voters probably would end up with one or two candidates on the ballot, which is fair enough if they can muster that level of support. They then, as now, have to face the verdict of all other voters, who will also have No Award as an option if they are unimpressed by the ballot as a whole or by individual entries.
So, what do I, as an elections nerd, think?
The proponents of E Pluribus Hugo say that their fundamental principle is that "No group - whoever that group may be - should be able to absolutely prevent nominees from having the chance to be considered for the Hugo Award." Behind this, I think, lies an assumption with which I agree: that the Hugo ballots should resemble a recognisable reflection of the state of the genre in each year. (This was one of the questions asked searchingly of previous years by Jo Walton in her Revisiting the Hugos series of posts on Tor.com. Most years pass the test for her, though some don't.) This year's ballots for most of the fiction categories and several others, notably Best Related Work, are not a recognisable reflection of the state of the genre, and fail that test. The slates have filled them with finalists which are based only in one corner of the field (and also, for the most part, aren't very good, though I'm reluctant to countenance any change that aims to prevent people from nominating rubbish if they want to; I have faith that the voters will sort it out in the end). A wider selection of finalists - including the slate candidates, if they have the support - must be a good thing.
To be honest, I'm not sure that recent slate-free Best Short Story ballots, with fewer than five finalists in three of the last four years, have really managed that reflection of the state of the genre either. What happened in those cases was that the nominations for Best Short Story were so smeared out among a large number of candidates that not enough of them crossed the 5% threshold to produce a full ballot. My instinctive reaction at the time was that the 5% threshold itself should be abolished (which has also been proposed this year as a standalone change), but I can see that this is not satisfactory: it leaves you with nominees on the final ballot who got 20% of the available slots (one of the final five) with only 4.5% of the vote, or less. It's not quite in the territory of the Australian Sports Party, who managed to elect one of the six senators from Western Australia in 2014 with 0.23% of first preference votes, but it's not brilliant either. (The Sports Party's senator was later unseated due to irregularities in the election count.) E Pluribus Hugo will produce a more consensual ballot, comprising finalists which have broader support than simply being the fifth-least obscure of a number of obscure works.
The downside of E Pluribus Hugo is that it does reduce the value of the votes of those who nominate more than one work. At present, if you nominate five candidates, each of them gets a full vote. Under the proposed system, your vote gets full value only when it is cumulated onto the most popular of those candidates. It means that the choice of the candidates who get through to the final ballot will be largely determined by those who nominate a single candidate, and give it enough votes to last to the end of the counting of nominations. I'm a bit sad about this. In previous years, one of the values of the Hugo process for me was that I was able to take recommendations from people better-read than me, and I did feel that it was fair enough that someone with enough knowledge to cast five nominating votes effectively had five times the fire-power of someone who cast only one. But the slates broke that system this year, and I guess we must discard it. Edited to add: I had missed the important point that the elimination process proceeds by pairwise comparison of the works with fewest points, eliminating the work with fewest nominations, which pretty much addresses my concerns.
Why not the Single Transferable Vote? In general, I'm a fan of and evangelist for the Single Transferable Vote in multi-member constituencies, for all public elections where more than one person is to be elected. It seems to me to get the best balance possible between voter impact on the result, proportionality of outcome between different political groups, and encouraging accountability from elected representatives. However, I can see that two of those three considerations don't matter much in this case.
Until recently, there were no slates for Hugo nominations, and the designers of E Pluribus Hugo specifically don't want to encourage their formation for future ballots. So it is not advantageous to have a system like STV that rewards political party-like behaviour, if that behaviour is something you want to discourage. Some of the slate-mongers actually do seem to desire a situation where politically oriented slates duke it out through the awards process, to the extent that they claim that this has already been happening for years (though when challenged they can produce no evidence whatsoever for this). I don't want this to happen, and I think that E Pluribus Hugo is better than the present system for discouraging it.
As for accountability, while I wish that some of this year's finalists were behaving in a more dignified or indeed decent way, the fact is that this is a yearly vote for an award, not a choice of public representatives who will carry out the work of one or other branch of government. So there is no point in designing a system that rewards responsiveness from the nominees to the voters as STV does (though equally good behaviour during the process doesn't do them any harm at all).
There's a process issue as well which makes STV unsuitable for this purpose. By definition, it gives your first preference candidate 100% of your vote, unless and until it is transferred. The ranking of candidates is hugely important. For the final stage of the Hugos, that matters; for the nominations stage, which is about assembling a decent ballot rather than electing representatives, it isn't. One would also have to do imaginative things with quotas and the values of surplus votes; while I understand that similar tweaks have been implemented elsewhere without complaint, I think that E Pluribus Hugo scores on transparency and ease of implementation.
What does still matter is maximising voter impact on the result. As I said above, I regret that the voters whose impact is reduced by E Pluribus Hugo are those who nominate several candidates, and those whose impact is increased are those who nominate only one. But I think that the system does mitigate that a bit through the serial cumulation process. (Edited to add: Actually quite a lot, it seems.) Those who have run simulations report that the difference to actual ballots is not very large. What I fear is that the quirky candidate that got a lot of people's fourth or fifth nominating votes will lose out to the popular candidate which the fourth or fifth largest number of voters have heard of. But in fairness, the quirky candidate was unlikely to win the award anyway, and the Hugos are a popularity contest rather than a quirkiness competition.
My conclusions on the various proposals: So with a slightly heavy heart - I regret that small-minded slate-mongers have killed off a large part of the wisdom-of-crowds aspect of the Hugo nominations process - I endorse E Pluribus Hugo as the best fix to prevent slates from dominating the process in future without irreparable damage to the credibility of the awards. Edited to add: I no longer think that a "large" part of the wisdom-of-crowds aspect has been killed off.
Three other proposals for reforming the Hugo process have been submitted to Sasquan. One is to abolish the 5% threshold; as I mentioned above, I agree with this faute de mieux, but E Pluribus Hugo removes the threshold requirement anyway, so I would only support it if E Pluribus Hugo is rejected.
I don't support the proposal to merge two of the short fiction categories and create a "Best Saga" category. The multiple short fiction awards at present reward writers who express their ideas succinctly rather than at big commercial length, and I’m in favour of that. The “Best Saga” proposal doesn’t fix any existing problem but does create new ones - not least of which, who is going to have time to read all the finalists between close of nominations and close of voting?
I do support the "4 and 6" proposal, to restrict voters to a maximum of four nominations rather than five as at present, but to extend the final ballot to include six rather than five finalists. If E Pluribus Hugo is not adopted, the "4 and 6" proposal is a lesser safeguard against slates, in that it becomes much more difficult to marshall your minions to support six slated works if they have only four votes each. And if E Pluribus Hugo is adopted, voters who nominate five candidates will get less value for their nomination than those who nominate four, and so on; the first part of the "4 and 6" proposal seems to me a decent indication to voters that a slightly different nominating strategy is now necessary (even though it's not actually part of E Pluribus Hugo). As for the second part, I do feel that good work is left off the Hugo ballot every year, and while Mike Scott's proposal from April ( 1, 2, 3) would have designed a certain responsiveness in the system specifically in reaction to the slates, I'd prefer a broader, simpler and less slate-dependent change, and I think that expanding the final ballot to six rather than five does that.
2,000 words later, I'm sorry to say that this is all a bit theoretical, in that I won't be voting on the proposed changes myself. They must be approved by those physically attending the WSFS Business Meeting in Sasquan this summer, and I won't be there; and then they must be ratified by those physically attending the WSFS Business Meeting in MidAmeriCon II next year as well, and it's unlikely that I will be there either. But I hope these observations will be useful to those who are attending and voting at either or both. This is important: a major flaw in the Hugo process (which was known, but rarely exploited) has been used to further a political agenda and to lock 85% of voters out of significant parts of the ballot paper, and it's absolutely appropriate to try and prevent this from happening again.
Good luck, everyone.
Edited to add I had missed the important point that the elimination phase in principle between the two works with fewest points, but is itself decided by the number of nominations. This is fairly clearly explained in the FAQ, but less clearly in the body of the proposal which is what I was concentrating on.
Kyra over at File770 has outlined a case where my scenario could still happen, but I have to admit that it's pretty improbable.
So my major objection to the EPH proposal is substantially mitigated by new information, and I can endorse it more whole-heartedly than before.
The process as it is now: At present, the nominating pool of voters for the Hugo Awards consists of paid up members of this year's Worldcon, last year's and next year's, as of the nominations deadline which is usually in March. Each voter can nominate up to five candidates for each of the Hugo categories (and the John W. Campbell Award). Each of those votes counts the same. The votes are tallied, and in principle the top five, together with the No Award option, form the actual ballot (with a couple of wrinkles: if the fourth or fifth-placed works get less than 5% of total nominations, they are omitted; if there's a tie for fifth place, all the tied candidates get on).
The winner of each category is then chosen by transferable vote, the system variously known as instant-runoff voting, Alternative Vote, Australian ballot, or STV for a single seat. Voters list the options in order of preference; first-preference votes are tallied; if none of the candidates has more than half of the first-preference votes, votes are transfferred from the least popular candidates until there is one which does have more than half of the remaining votes. The winner of that count is then checked against No Award, to see which was preferred by more voters, ignoring all other finalists. There is no proposal to change that part of the process, and I personally like it as it is.
What's been proposed: The E Pluribus Hugo proposal retains the current system of inputs at the nominations stage from the voter's perspective. You still get to choose up to five candidates for each category. The difference is that there is a two-stage process. In the first stage, your vote gets split fractionally between your nominees. If you nominate five candidates in a particular category, your have cast 0.2 of a vote for each. If you nominate four, your vote counts for 0.25. If you nominate only one, your vote counts in full.
That is, at the first stage. The proposal then eliminates seriatim all those works which have not got enough support to get on the final ballot, and reallocates their votes as if they had not been nominated. That process continues until there are only enough left to fill the ballot. So if you nominate five books, and three of them are unpopular with other voters, your vote ends up being split with a value of 0.5 each for the other two. What usually happens to me is that I nominate five books, four of which nobody else has heard of, and my vote would then end up completely concentrated on the remaining one. This is obviously too complex to run by hand, so it will need to be done by computer (Hugo voting has anyway been administered by computer for many years).
The envisaged consequence is that if 15% of voters blindly follow the dictates of a slate, and the other 85% smear their votes around a number of other works, the votes of the 85% will end up being concentrated in such a way as to ensure that the slates do not get all five slots. On the other hand, a slate which is supported by 15-20% of voters probably would end up with one or two candidates on the ballot, which is fair enough if they can muster that level of support. They then, as now, have to face the verdict of all other voters, who will also have No Award as an option if they are unimpressed by the ballot as a whole or by individual entries.
So, what do I, as an elections nerd, think?
The proponents of E Pluribus Hugo say that their fundamental principle is that "No group - whoever that group may be - should be able to absolutely prevent nominees from having the chance to be considered for the Hugo Award." Behind this, I think, lies an assumption with which I agree: that the Hugo ballots should resemble a recognisable reflection of the state of the genre in each year. (This was one of the questions asked searchingly of previous years by Jo Walton in her Revisiting the Hugos series of posts on Tor.com. Most years pass the test for her, though some don't.) This year's ballots for most of the fiction categories and several others, notably Best Related Work, are not a recognisable reflection of the state of the genre, and fail that test. The slates have filled them with finalists which are based only in one corner of the field (and also, for the most part, aren't very good, though I'm reluctant to countenance any change that aims to prevent people from nominating rubbish if they want to; I have faith that the voters will sort it out in the end). A wider selection of finalists - including the slate candidates, if they have the support - must be a good thing.
To be honest, I'm not sure that recent slate-free Best Short Story ballots, with fewer than five finalists in three of the last four years, have really managed that reflection of the state of the genre either. What happened in those cases was that the nominations for Best Short Story were so smeared out among a large number of candidates that not enough of them crossed the 5% threshold to produce a full ballot. My instinctive reaction at the time was that the 5% threshold itself should be abolished (which has also been proposed this year as a standalone change), but I can see that this is not satisfactory: it leaves you with nominees on the final ballot who got 20% of the available slots (one of the final five) with only 4.5% of the vote, or less. It's not quite in the territory of the Australian Sports Party, who managed to elect one of the six senators from Western Australia in 2014 with 0.23% of first preference votes, but it's not brilliant either. (The Sports Party's senator was later unseated due to irregularities in the election count.) E Pluribus Hugo will produce a more consensual ballot, comprising finalists which have broader support than simply being the fifth-least obscure of a number of obscure works.
The downside of E Pluribus Hugo is that it does reduce the value of the votes of those who nominate more than one work. At present, if you nominate five candidates, each of them gets a full vote. Under the proposed system, your vote gets full value only when it is cumulated onto the most popular of those candidates. It means that the choice of the candidates who get through to the final ballot will be largely determined by those who nominate a single candidate, and give it enough votes to last to the end of the counting of nominations. I'm a bit sad about this. In previous years, one of the values of the Hugo process for me was that I was able to take recommendations from people better-read than me, and I did feel that it was fair enough that someone with enough knowledge to cast five nominating votes effectively had five times the fire-power of someone who cast only one. But the slates broke that system this year, and I guess we must discard it. Edited to add: I had missed the important point that the elimination process proceeds by pairwise comparison of the works with fewest points, eliminating the work with fewest nominations, which pretty much addresses my concerns.
Why not the Single Transferable Vote? In general, I'm a fan of and evangelist for the Single Transferable Vote in multi-member constituencies, for all public elections where more than one person is to be elected. It seems to me to get the best balance possible between voter impact on the result, proportionality of outcome between different political groups, and encouraging accountability from elected representatives. However, I can see that two of those three considerations don't matter much in this case.
Until recently, there were no slates for Hugo nominations, and the designers of E Pluribus Hugo specifically don't want to encourage their formation for future ballots. So it is not advantageous to have a system like STV that rewards political party-like behaviour, if that behaviour is something you want to discourage. Some of the slate-mongers actually do seem to desire a situation where politically oriented slates duke it out through the awards process, to the extent that they claim that this has already been happening for years (though when challenged they can produce no evidence whatsoever for this). I don't want this to happen, and I think that E Pluribus Hugo is better than the present system for discouraging it.
As for accountability, while I wish that some of this year's finalists were behaving in a more dignified or indeed decent way, the fact is that this is a yearly vote for an award, not a choice of public representatives who will carry out the work of one or other branch of government. So there is no point in designing a system that rewards responsiveness from the nominees to the voters as STV does (though equally good behaviour during the process doesn't do them any harm at all).
There's a process issue as well which makes STV unsuitable for this purpose. By definition, it gives your first preference candidate 100% of your vote, unless and until it is transferred. The ranking of candidates is hugely important. For the final stage of the Hugos, that matters; for the nominations stage, which is about assembling a decent ballot rather than electing representatives, it isn't. One would also have to do imaginative things with quotas and the values of surplus votes; while I understand that similar tweaks have been implemented elsewhere without complaint, I think that E Pluribus Hugo scores on transparency and ease of implementation.
What does still matter is maximising voter impact on the result. As I said above, I regret that the voters whose impact is reduced by E Pluribus Hugo are those who nominate several candidates, and those whose impact is increased are those who nominate only one. But I think that the system does mitigate that a bit through the serial cumulation process. (Edited to add: Actually quite a lot, it seems.) Those who have run simulations report that the difference to actual ballots is not very large. What I fear is that the quirky candidate that got a lot of people's fourth or fifth nominating votes will lose out to the popular candidate which the fourth or fifth largest number of voters have heard of. But in fairness, the quirky candidate was unlikely to win the award anyway, and the Hugos are a popularity contest rather than a quirkiness competition.
My conclusions on the various proposals: So with a slightly heavy heart - I regret that small-minded slate-mongers have killed off a large part of the wisdom-of-crowds aspect of the Hugo nominations process - I endorse E Pluribus Hugo as the best fix to prevent slates from dominating the process in future without irreparable damage to the credibility of the awards. Edited to add: I no longer think that a "large" part of the wisdom-of-crowds aspect has been killed off.
Three other proposals for reforming the Hugo process have been submitted to Sasquan. One is to abolish the 5% threshold; as I mentioned above, I agree with this faute de mieux, but E Pluribus Hugo removes the threshold requirement anyway, so I would only support it if E Pluribus Hugo is rejected.
I don't support the proposal to merge two of the short fiction categories and create a "Best Saga" category. The multiple short fiction awards at present reward writers who express their ideas succinctly rather than at big commercial length, and I’m in favour of that. The “Best Saga” proposal doesn’t fix any existing problem but does create new ones - not least of which, who is going to have time to read all the finalists between close of nominations and close of voting?
I do support the "4 and 6" proposal, to restrict voters to a maximum of four nominations rather than five as at present, but to extend the final ballot to include six rather than five finalists. If E Pluribus Hugo is not adopted, the "4 and 6" proposal is a lesser safeguard against slates, in that it becomes much more difficult to marshall your minions to support six slated works if they have only four votes each. And if E Pluribus Hugo is adopted, voters who nominate five candidates will get less value for their nomination than those who nominate four, and so on; the first part of the "4 and 6" proposal seems to me a decent indication to voters that a slightly different nominating strategy is now necessary (even though it's not actually part of E Pluribus Hugo). As for the second part, I do feel that good work is left off the Hugo ballot every year, and while Mike Scott's proposal from April ( 1, 2, 3) would have designed a certain responsiveness in the system specifically in reaction to the slates, I'd prefer a broader, simpler and less slate-dependent change, and I think that expanding the final ballot to six rather than five does that.
2,000 words later, I'm sorry to say that this is all a bit theoretical, in that I won't be voting on the proposed changes myself. They must be approved by those physically attending the WSFS Business Meeting in Sasquan this summer, and I won't be there; and then they must be ratified by those physically attending the WSFS Business Meeting in MidAmeriCon II next year as well, and it's unlikely that I will be there either. But I hope these observations will be useful to those who are attending and voting at either or both. This is important: a major flaw in the Hugo process (which was known, but rarely exploited) has been used to further a political agenda and to lock 85% of voters out of significant parts of the ballot paper, and it's absolutely appropriate to try and prevent this from happening again.
Good luck, everyone.
Edited to add I had missed the important point that the elimination phase in principle between the two works with fewest points, but is itself decided by the number of nominations. This is fairly clearly explained in the FAQ, but less clearly in the body of the proposal which is what I was concentrating on.
Kyra over at File770 has outlined a case where my scenario could still happen, but I have to admit that it's pretty improbable.
So my major objection to the EPH proposal is substantially mitigated by new information, and I can endorse it more whole-heartedly than before.