Monday, September 18, 2006

TIFF 2006 #13: Insane TIFF Math – The Fest’s Awards Baffle Me Yet Again

I don’t get it.

I don’t understand how the People’s Choice award is decided. Does anyone really know? Because the rules and processes aren’t outlined anywhere, and I’m really curious.

In years past, audience members would fill out a ballot in a quasi-lottery set-up where you were not only casting a vote for the PC award, but also entering yourself into a prize lottery where you could win things like an Air Canada vacation (years ago) or mutual funds (more recently). So, you took ballots, wrote in the name of the movie(s) you liked best along with your contact info, and dropped it/them into a box. Simple. But not totally fair (see math below).

This year, going into every screening, we were handed a ballot with a line on which to write that film’s title (sometimes the title had already been pre-written for you), and then the numbers one to five printed in a large font below. All we had to do was circle the number we felt the film deserved…though, no explanation was ever given that a number “5” meant you thought the film was excellent, and a “1” meant you didn’t like it at all, not the other way around.

This seems like it would be fairly obvious, but director Lucy Walker (Blindsight) told my audience at the second screening of her film that she, herself, voted incorrectly for her own movie at its previous showing…having thought that by voting “1” she was, in fact, giving it the highest rating and voting it the “#1 movie.”

I wonder how many other filmgoers made that same mistake?

Aside from numerical confusion and the lack of a prize incentive
for bothering to vote, I’d like to know: how were the votes tallied? There’s no way this could be at all fair, especially to the smaller films screening in smaller venues. Here’s why…

Given the wide range in venue capacity, where some theaters hold only a couple hundred people while others boast more than 1200 seats, it seems hugely unfair if the award is decided based on the total sum of the numeric votes (i.e., adding up all the numeric votes to reach a total).

Ex. 1

* Movie A screens twice at the Cumberland, and gets 400 votes of 5 (excellent) for a total point value of 2000.
* Movie B screens three times (let’s say, twice at the Paramount and once at the Varsity) and gets 900 votes of 3 (meh) for a total point value of 2700.

Does this mean that movie B wins?

Let’s say for argument’s sake that the People’s Choice winner goes to the film that scored the most 5s. That doesn’t work either.

Ex. 2

* Movie A screens twice at the Cumberland and gets 400 votes of 5 (excellent), from a total audience of, say, 600 people…meaning 2/3 of the people who saw it thought it was fantastic.
* Movie B screens once at Roy Thompson Hall and once at the Ryerson, and gets 500 votes of 5, from a total audience of more than 2000…meaning, only ¼ of the people who saw it thought it was great.

Again, does this mean movie B, which technically had less success, wins simply because its sample audience was much, much bigger?

No matter how you slice it, the system is wildly flawed, and that’s without getting into the issue of ballot stuffing or the fact that the People’s Choice prize is being decided before some films have had their final screenings on day 10 of the festival (where do those votes go???). Heck, by the time I got home from my final film on Saturday, TIFF brass had already handed out the awards and held an afternoon press conference to say so!

I’m going to say it once, and I’m going to hope for the love of fairness that the folks at TIFF hear me…

The only fair way (or, perhaps, the least unfair way) for the People’s Choice Award to be decided is to allow each filmgoer to vote only ONCE. Period. Save the voting for the day after the fest is over, and allow each person to vote ONLY for a single, absolute-favourite film. Then count ‘em all up and hand out a trophy. The end.

This concludes the PC Rant of 2006.

Having said all that, here’s a list of the official fest winners:

THE PEOPLE'S CHOICE AWARD went to Bella. What the hell? Bella??? Really?? (A quick glance at the schedule reveals that Bella screened three times – 2 Paramounts + a Varsity. Did that help? Probably.) Honourable mentions went to Mon Meilluer Ami and Dixie Chicks: Shut Up and Sing.

SWAROVSKI’s CULTURAL INNOVATION AWARD went to Özer Kýzýltan’s Takva: A Man’s Fear of God. Honourable mention went to Khadak. I can honestly say I have no recollection of either film in the program book, and didn’t hear anyone in any line or theater talking about either.

THE DIESEL DISCOVERY AWARD went to Joachim Trier’s Reprise, a comedy from Norway. Didn’t see it.

THE FIPRESCI PRIZE, awarded by a group of international press, went to the much-ballyhooed Death of a President (D.O.A.P.). All the buzz I heard about this one was that it was highly overrated and unworthy of all the attention, so go figure.

CITYTV AWARD FOR BEST CANADIAN FIRST FEATURE FILM went to Sur La Trace de’Igor Rizzi. Never heard of it.

TORONTO-CITY AWARD FOR BEST CANADIAN FEATURE FILM went to Manufactured Landscapes, the doc about photographer Peter Mettler. Honourable mention went to Monkey Warfare, which was mightily crapped upon by many. Again, go figure.

SHORT CUTS CANADA AWARD for best Canadian short film went to Les Jours.

I dunno…what do those of you who attended TIFF think about the prizes, either in terms of how they’re handed out or who actually won this year?

As for me, I’m going to take a short breather and return with my take a little later.


Linda said...

At SIFF there is almost always a volunteer introducing the movie and the voting system, before the lights go down, as in "Tear through the 1 if you didn't like the movie, and 5 if you thought it was the greatest thing since sliced bread!" The drill this over and over at every screening, so that everyone is rolling their eyeballs, but the audience at least is not confused.

As for the number of screenings/sizes of audience question, I know that SIFF has some very careful formula to accomodate that issue. Otherwise, every locally-made so-so production would win the top prize, because the entire cast, crew, and their friends and family would totally stuff the ballot box with 5s for a sweep. Luckily that technique never seems to work.

Usually the Golden Space Needle (Audience Award) goes to a true crowd pleaser (yes, often where the director and/or actors are there to hype up the audience).

That said, I think it is only twice in the last 10 or so SIFFs that I've attended that I managed to see the winner at the fest, and those were Whale Rider (where the audience was so electrified by the experience, you KNEW it would win), and Facing Windows (which had the director there, and the audience reduced to a sobbing mess at the end of the movie).

Matt said...

None of the theatre staff or volunteers seem to know how the ballots are tabulated. It happens later, at an "undisclosed location". The fine print at the bottom of the ballot does say that "5 is most positive", but it's too hard to read (especially in dim theatres). I've heard many people asking how it all works.

More factors to consider: theatres aren't always full (a few daytime screenings at the 1200 seat Ryerson get Varsity-sized crowds). And this year's write-in-the-title system allowed people to vote for a different movie than the one they were attending, though I don't know if and how those ballots would be counted. At single-screen venues, ballots with blank titles were kept together per film, though again I can't say for sure they did get counted.

It seems to me an average rating system (total points divided by total ballots for that film; probably need over 4.9 to win) would be the fairest way to count these ballots, though tricky to get right for films with very small audiences. The old system was prone to ballot box stuffing, so it wasn't perfect either.

Last year, with the same system except no place to write in the title, the pick was Tsotsi. I'd never heard of it then, but it did go on to win the Oscar for Best Foreign Language Film, so the system wasn't completely broken...

Vickie said...

The thing about calculating an average score (out of 5) would be that you'd likely wind up with multiple films scoring, say, 4.8 or higher. So, they'd either have to award numerous films a People's Choice plaque (and cheque) or find someway of then picking one from the top scorers. TIFF loves lotteries, so I wouldn't be surprised if a random draw to determine the People's Choice winner becomes next year's brilliant idea.

Again, it's all rather flawed and, given what you've just posted matt, somewhat suspect.

Vickie said...

Geez. Please insert a space between the "some" and the "way" of "someway." Damn the lack of a comment-editing feature! ;-)

vickie testing the comment function said...

This is a test to see if non-bloggers can comment.

Greg said...


I love that you're so angry about the TIFF awards. It makes for great reading.

I'm sad the Festival is gone for another year.

Seriously though, you need to eat better food. Perhaps your readers should pool our cash and buy you gift cards to Pusateri's or Hemmingway's or something.

Vickie said...

Yeah, my fest diet is always horrible. I feel like grease actually oozes out of my pores for a good few days after my last film. (Pretty, no?)

Thanks for posting, and for reading the blog!

Matt said...

With enough ballots an exact tie is unlikely. Maybe Bella beat Summercamp! 4.8147 to 4.8133 (that'd be a one-vote difference in a sample of 750). Or, you could throw in some advanced statistics (which I'm glad to have avoided taking the courses for) to determine which vote pattern has more "significance".

I really like the idea of one person, one vote. But while it'd be easy to make happen for pass & coupon book people, and might be doable for people who pay for screening tickets with credit cards, it'd be nearly impossible to handle cash sales.

Linda said...

Re: Voting in general...

I noticed that at SIFF, the Full Pass holders, or "Fool Serious" folks, or "passholes" ;) or whatever you want to call them, tend to be much more critical when rating a movie at the fest. Do you think the same is true at TIFF?

You'll have the folks seeing three movies in a row at a theater, who have made a nest of their stuff in their favorite spot, and they'll be the first to rip ANYTHING to shreds as soon as the credits roll (if they haven't darted out the door at that point)... VS. three grandmas from the suburbs who will coo at the in-person directors and actors and murmur to each other, "Well, I'd give that a 5! It was wonderful, don't you think?" (while the Full Series folks roll their eyes in the row right behind them).

Luckily, for all involved I think, this guaranteed mix will just about prevent any movie from getting perfect 5's. But it also means that often crowd-pleasing "pop" movies like Pump Up the Volume (a SIFF winner way back when) will occasionally win the Audience Award.

Needless to say, I am often surprised by festival winners!

Vickie said...

Yep, I totally agree. The hardcore festgoers are less easily swayed and are often much tougher to please.

Then again, there are the people like me, who see lots of movies but still fall head over heels for feel-good movies almost every time. ;-)

The_Voice said...

The best way to do it would be to do a root square mean voting system, with a weighting to push it to either higher or lower sections based on the number of ballots submitted or seats in the theatre.

Think about: say 1 person voted for a film out of 750 people who saw it... and that person gave it a 5... it shouldn't win on one vote. For every uncounted ballot for a certain film, you count a vote of 3 and count those votes equal to other votes. That way a 1 person voted film would be scaled to a more avg rating. It would theoretically be fair towards small films, large films, and films people vote on or not vote on.

The root square mean part of it just evens out extremes a bit more.