Saturday, 27 April 2013

Slightly better than chimpanzee...

There was a popular joke that made the rounds in the late 1990's about the different levels of efficiency of the election commissions in various Asian countries. At the time, many countries in the region were experiencing a surge of interest in elections and politics. The joke goes something like this:

A journalist wants to know which country in Asia is fastest in announcing the results of their national elections. First, he went to Indonesia and met with the Election Commission (EC) there. The Indonesian EC said, "we are a big country and our system isn't really good, so we can only announce the official results three months after the election." The journalist next went to India and asked the EC there the same question. The Indian EC said, "we have a huge population but our system is good, we can announce the results three days after the election". Next, to Singapore, and the journalist was told, "we are very efficient, we announce the results in three hours". The journalist finally went to Malaysia and told the Malaysian EC how efficient Singapore was, to which the Malaysian EC chief retorted, "we can do even better, we can announce the results three months before the election!"   

This was of course during the time when elections in Malaysia were more predictable and if someone did really want to fix the election, he could have done so quite easily. Civil society was not strong, election monitoring bodies almost non-existent and opposition parties most often did not have enough counting and polling agents. Elections can still be fixed today, of course, but to do so without anyone noticing it would be extremely difficult.  

Fixed or not, to predict (correctly) election results is not easy. We often rely on experts and pundits for their views but how valid really are their views? Well, according to Professor Philip Tetlock, in his book Expert Political Judgment: How Good Is It? How Can We Know? (2005), an 'expert' is only slightly more accurate than a dart-throwing chimpanzee! A rather sad statement, isn't it? But the evidence is overwhelming.

Here, in Malaysia, we only need to go back to the 2008 General Election to find examples of how inaccurate many expert predictions were. If you browse through all mainstream newspapers the editions just prior to the election, you will find one of the most talked about issues was 'who from Gerakan will replace Koh Tsu Koon as Penang Chief Minister?'; a question totally oblivious of even the possibility of Gerakan losing in Penang. And among pro-Barisan Nasional experts, a view often drummed-up was the 'death' of Parti Keadilan Rakyat after the election and the end of PAS' rule in Kelantan.

What happened then was really quite amusing. I can still recall watching at around 7 pm on election day how excited an RTM presenter was about wanting to know the latest tally in Permatang Pauh. She was so excited because she was expecting a result that signals the end of PKR. PKR, of course not only retained Permatang Pauh but won 30 other parliamentary seats in 2008.      

Local political experts and politicians are very much more cautious this time around. Pro-government experts have lately become more philosophical. Those with statistics and numbers are gaining more prominence.

A few days ago, Professor Redzuan Othman from the University of Malaya presented 'selected findings' from his survey on the election. He reported that 43 percent of his respondents preferred Anwar Ibrahim as Prime Minister, and 37 percent preferred the incumbent Najib Abdul Razak. That is indeed good news for the opposition. Pakatan Rakyat supporters are ecstatic and many are now predicting a resounding victory.

I am not a member of any political party but for this election, I have openly expressed my wish to see PR win. I honestly believe a PR win would be good for the country, and good for Umno and BN too;) But a PR victory, with one week to go, is far from certain. Seven days is a long time in politics. Many things can still happen, many 'videos' can still be produced, and many mistakes can still be made. If everything falls into place, PR may not even need Sabah and Sarawak to win federal. But make just one mistake and PR may end up with only Penang and Kelantan to govern.        

Professor Redzuan's findings would be more meaningful if this was a direct presidential election. But that's not the case here. We have a really unfair system where even if a coalition garners more than 50 percent of the popular votes, it may only get 25 percent seats in parliament. The magic number is 112. Whoever gets at least 112 parliamentary seats will form the next federal government. And at this point, nobody really knows who will that be.

The only way any pundit or expert can validly predict the outcome is to do what Nate Silver did for the US Presidential Election last year. He correctly predicted the outcome in all 50 states. More remarkably, in 48 of those, the actual votes received by each candidate fell within his margin of error.

Nate Silver did not collect any data on his own. He uses data from various polling and media agencies, averages them and run a series of regression analyses to cater for various adjustments. The method isn't really that complicated. But you need a diverse range of present and historical data to get it right, and we just don't have that here in Malaysia.

That, however should not stop us from making our own predictions. Unlike 2008, I haven't been able to observe how things are on the ground but I still would like to offer my own humble slightly-better-than-chimpanzee's prediction. Later...


YKC said...

Hi zaki, a couple things i would like to say..

you seem to share the confidence of pakatan leaders. i can see why especially when we all see the big crowds anwar ibrahim and other pr attracts in their ceramah. but you know and i know., big crowds do not necessarily mean votes. the biggest advantage umno and bn has that pr parties do not is a solid bulk of loyal votes. these are your everyday pakcik and makcik in the kampungs, people in government flats, policemen, soldiers and civil servants. that's a solid 4 or 5 million votes. and you dare imagine pr winning without sabah sarawak?! you must be dreaming! and i mean that in the nicest possible way because you are a nice guy:) umno youth could do with someone savvy like u but i know, we've talked about this once and you said you are a muslim first, malay fourth (or was it fifth?) so i guess if u jump into politics you'll be joining pas, or maybe berjasa? :))

second thing i want to say is and i think you with your own contacts may know already, if pr wins anwar may not actually be PM. it will from PAS and my guess is it is husam. the guy didnt just leap to putrajaya for nothing. it was strategic. the interesting part is whether DAP knows about this and will they accept a PAS PM candidate? your thoughts on this...

interesting times ahead indeed. i know things have been rough on you lately but i sure hope your dream for political office is still alive. in 5 years time if not less, i hope to see you as a candidate. people in kelana jaya can rally behind u. many of us your schoolmates are still registered here. we can mobilise them to support a true local boy ;))

Zaki Samsudin said...

Dear YKC, thanks for dropping by.

Pakatan leaders and hardcore supporters are indeed extremely confident. I am not that confident. I would like to see how things are in the next few days before making a final prediction.

The point you made about BN's block of guaranteed votes is spot-on, and this is something many PR people are taking for granted. But in the past few months, with a few retired senior police officers and army personnel now in PR's ranks, perhaps they have conjured up a strategy to tap into this block of votes. How successful they are remains to be seen.

Me stepping into the fray is a distant possibility. I certainly find public service very appealing but the vision I have is always it is something to think about in later age, when the kids (at least the elder ones) are safely thru school and already have a clear path in life. So, that is at least another 10 years from now. That's the plan but of course, only God knows, some things may change.

Thanks again.

YKC said...

i understand where you are coming from. yes, i remember clearly your point about philosopher king that politics should be for someone who has made a life outside politics. but then, dont u think when you see people like nik nazmi, hannah yeoh, yusmadi and few others that perhaps u can do it too? i dont think you realise that you have this gift with people. my mother still remembers you even though the last time she met you was like 15 years ago. it's a gift that you have, a superpower :) and you know as uncle ben said "with great powers comes with great responsibility" :))

my own prediction is PR will only slightly improve what they got in 2008 somewhere between 90 to 100. tell me yours and see who wins. siapa kalah belanja kuey teow ok? ;)

you didnt respond to what i said about Husam Musa. tell me what you know.

Zaki Samsudin said...

What I have heard is that some time last month, Ustaz Haron Din went alone to see Anwar for the sole purpose of informing him that in the event PR wins, PAS does not want him to be PM. I don't know who PAS' nominee is. It could very well be Husam. Certainly there are untold reasons why he is contesting Putrajaya. It could well be a direct challenge Anwar gave to PAS, "put your PM nominee in Putrajaya... if he wins, he'll be PM!" ;)

Personally, I prefer to see Anwar as PM. For a man who has gone thru so much and victimised by so many, it would be nice, almost fairytale like to see him as PM. The man is far from perfect but his 'superpower' exceeds all else.

Let me come back to the point I made about the possibility PR winning federal without Sabah and Sarawak. Granted, this is a bit farfetched but think about this... back in 2008, 80 out of 82 of PR seats came from the peninsular. You win some and lose some this time around but supposed PR gains 10 more seats from all the states excluding Terengganu and Johor... PR now has 90, and needs 22 more to win. That can come from both Terengganu and Johor if PAS Terengganu can sweep 7 (out of 8) like they did in 1999, and PR takes 15 (out of 26) in Johor! What do you think?

Appreciate your klnd appreciation of my 'superpowers':) Still, I remain thus far that a career in politics is something to think about later.

My regards to your mom :)

YKC said...

i,ve heard the same thing too so husam looks like the real deal. only thing i'll add is this depends on who wins more seats pas or pkr. it's a given pm will not be from dap so pas and pkr will slug it out.

pakatan winning 15 in johor is a bit too much lah bro..! come on punya kampung kan johor. mana boleh org2 melayu kat johor tu bagi dap menang. mixed seats yes dap and pkr banyak boleh menang. salehudin ayub not very known there so pas difficult to penetrate malay area.

i hope our kuey teow bet is still on. will wait your final prediction :) i will give mine too maybe ;))

Anonymous said...

Greetings Sir.

I am a student of political science. I followed the American presidential election last year and came across Nate Silver's blog. Jon Stewart calls him Mr.Arithmetic because of his abilities to use maths for predictions. Malaysia needs our own Mr.Arithmetic. Why not you,sir? You can be great in this:)

p/s. I support BN but I feel more and more BN will lose:(

Zaki Samsudin said...

Thanks for your kind suggestion. I'm not so much a numbers guy so I don't think I'm the right person to be Malaysia's Nate Silver. I can perhaps be part of a team that does this, so, we'll see... :)

Don't believe too much to rumours. I honestly still think BN will (barely) scrap thru. Wanita Umno in fact today said based on their own survey BN will win 2/3, which is rather outrageous. Still four days to go...:)

YKC said...

Zak, wanna make sure i have this engraved here... I WON!

you owe me kuey teow n two teh tarik. the teh tarik is for your wildly off the mark prediction for Johor.

see ya! peace!

Zaki Samsudin said...

My old friend YKC has every right to engrave his victory call here. Congrats bro! Kuey teow & teh tarik next time will be on me.

I, however, would like to engrave it here too that about a week before polling day, I had used the voting trend application and got a predicted result of 131 to BN and 91 to PR. Unfortunately I did not save a screen shot of that so I have no evidence to back that claim. That was the result I thought realistically we would get although being a PR supporter, I was hoping for a slight miracle tipping the result in PR's favour. Obviously that didnt happen. Disappointed yes, but I am greatly optimistic about a great many things.

Afz said...

As Salaam Dr.Zaki. I'm a statistician. We met few years ago at UPM at the M'sian Social Science Association conference.I've been looking at the many predictions for the last elections. I'm curious to know how our very own Majlis Professors got it so very wrong with their predictions. Do you know why? There are many learned people in that council. I'm intringued to know what happened.


Zaki Samsudin said...

wa'alaykum salam warahmatAllah

Afz, thanks for dropping by.

Honestly, I am as perplexed as you are as to how Majlis Professor Negara got their predictions horribly wrong. I don't really know anyone who was part of that UUM-MPN survey so I don't have any info on the details. It was stated though that the survey was done by phone. That is potentially one of the main reasons why the data collected skewed so much towards PM Najib and BN. Most Malaysians are still not comfortable with answering questions on the phone. You don't know who you are talking to. So, to be safe, just respond in support of the status quo even though in reality, you don't intend to vote accordingly.

Just my two-cents... Drop me a line thru email, fb or twitter. Let's meet up one of these days. Wassalam.