the urtak blog

Thinking about Gen. McChrystal

Yesterday, Barack Obama fired Gen. Stanley McChrystal for his bizarre indiscretion.

FOX News ran the following web poll:

205,283 responses and 4346 comments later, we learn that 67% of the respondents believe Obama should have forgiven McChrystal. And nothing else.

One question garnered huge response, but we know nothing about who they are. That massive amount of engagement was channeled into a dead end.

Our friends at The Toilet Paper made McChrystal the subject of the Urtak of the day.

They collected over 1300 responses to ten questions. When we add the x-tabs, that means that they generated 100 opinion facts about McChrystal, every one of which is publicly available on urtak.com. Incidentally, only 54% of their readers thought the general should be forgiven. Still very high, I think!

With far less traffic, The Toilet Paper generated more information than the mighty FOX News organization. Once again, the value of the collaborative approach is demonstrated.

100. 200. 400?

We are pleased to announce that today we have reached the milestone of 200 Urtaks in the public directory. That means 200 collaborative polls that anyone can study to gain new insights into how people think.

It took us more than a year to get to 100 Urtaks in the directory. The second 100 has taken three months.

Our sincerest thanks to everyone who has participated. Onward and upward!

Do you have a favorite Urtak? Let us know in the comments!

A response from The Toronto Star

Last week, I wrote an article for The Faster Times challenging the Toronto Star’s reporting of a Toronto mayoral election poll.

Here is the response of The Star’s Public Editor to my article:

Dear Marc:

I have now had opportunity to speak to David Rider and City Editor Graham Parley about your concerns.

As you may know, David has also blogged about some of the issues you have raised http://thestar.blogs.com/thegoods/2010/06/qas-about-the-forum-poll-on-ford-smitherman-and-tory.html and we have ascertained that the entire poll is now available online. [link]

To be honest, I don’t know enough about the art and science of polling to judge whether this poll is problematic. You certainly know more than me about polling but so too does the firm that conducted this poll.

The newsroom is of the view that Lorne Bozinoff is a reputable pollster whose firm has been in operation since 1993 —though associated more with market research than political polling. Mr. Bozinoff, who has no connection to any of the candidates, offered this poll to the Star. That should have been made clear to readers.

We do understand that a high number of undecideds can skew a poll. But, Mr. Parley, who directed the Star’s coverage of the last federal election campaign, tells me that: “But I also know you don’t simply subtract the undecideds and only report the decideds. Virtually every political poll I’ve ever seen is reported with the undecideds proportionately divided so that the preference answer always adds up to 100. (For example, PC 42, Libs 38, NDP 15, Bloc, 5).”

I do recall that some readers had some concerns about federal election polls not including information about undecided voters. I think this is an interesting issue for journalists in reporting on polls and I would like to learn more about it, as we move more fully into the municipal election campaign.

In this instance however, the bottom line for the Star is that Forum Research’s name is behind this poll and Lorne Bozinoff appears amply qualified to design and analyze it.

This wasn’t a poll commissioned by the Star and the article never suggested it was. It was a piece of reporting on an independent poll done by a reputable pollster and I believe Dave Rider reported it accurately.

Please give me a call if you wish to discuss further.

Best Regards,

Kathy English

I am not satisfied by this reply, especially since it doesn’t address the error made by misreporting the poll’s margin of error, and especially that of reading far too much into statistically meaningless figures. It is disappointing that the Star does not seem to have a terribly profound grasp of polling, yet gives poll results so much weight in its pages.

I would be very interested to hear other points of view. Please share your thoughts in the comments.

Public Opinion - From xkcd

http://xkcd.com/756/

No one has ever called us gamines

Some Urtaks are very large. Others are smaller. But what makes the collaborative poll more interesting than any other is always the quality of the questions.

The Gamine is a blog that defies categorization. Its content is a confluence of art, fashion, and lifestyle, with a focus on how these are experienced in New York. I always find it to be attractive, insightful, and unexpected.

But of course, we really love them for their Urtak, the gamine wants to know…. They have asked some very, very important questions.

As a man once berated for wearing black with navy, I was pleased to find that the readers unanimously agree that this is no crime.

They ask the factual and the transgressive together.

And of course, The Gamine believes as we do, that rules are meant to be broken.

Take a look at The Gamine, and then examine their Urtak. As always, the results and x-tabs are fascinating. Submit an interesting discovery in the comments!

the gamine wants to know…

Made in America

While watching yesterday’s World Cup disaster, Aaron and I had the good fortune to catch this Dodge Challenger ad, starring Robin Williams as George Washington.

After teaching the limeys another lesson (when will they learn?), the narrator reminds us that, “Here’s a couple of things America got right: Cars, and freedom.” This is the typical American modesty.

The U.S.A. has given the world so much more than cars and freedom. Among others, the swivel chair, potato chips, lasers, and volleyball (indoor and beach).

Next time, we hope the good people at Dodge will show their country more respect.

BONUS: Check out what people have said about CARS and FREEDOM in the general interest Urtak.

UPDATE: The Dodge Challenger is not an American car at all. It is produced in Brampton, Ontario, Canada. For shame.

UPDATE #2: Of course, a Canadian car is still “American.” What it isn’t is estadounidense, as our Latin brothers and sisters say.

Live from New York!

On Saturday, I had the good fortune to be invited to give a short talk at NYC Startup Weekend. NYCSW is a very cool event where over a hundred people come together to create and launch new ideas over the course of just one weekend.

I talked about how Aaron and I took Urtak from an idea in our minds to the dynamic product that it is. I also shared some of our next steps for the future. Unfortunately, only the last eight minutes of it were captured on video. Though to be fair, that should be more than enough for any civilized person anyway!

Here is the video, from NYC Startup Weekend at NYU’s Courant Insitute:

At the end of the video, I also took a quick look at the results of the Urtak set up for the event. If you want to take a look at it yourself, here it is, below:

Startup Weekend survey

6,000,000

Six million responses is a big deal.

It took us less than seven weeks to gain our last million responses, our fastest ever.

Hundreds of thousands of people have participated in our project to create both a resource and a technology for collaborative public opinion research.

With a small group of dedicated individuals, and a much larger group of friends and allies, we have shown that gathering millions of responses is something that is within reach of many, many organizations.

Collaborative public opinion is real. Now our task is to make it hegemonic. More than two years after the first response came in to our system, we are determined to grow faster. Expect exciting new releases in the next 30 days, 12 months, and years to come.

Next stop is 60 million responses. Any guesses as to when that will be?

Less than a year, say Aaron and I.

Bad Polling in Toronto

From The Faster Times:

This morning, readers of the Toronto Star woke up to a new fact. On page GT1, Urban Affairs Bureau Chief David Rider writes that Rob Ford, the right-wing populist candidate for the mayoralty of Canada’s largest city has caught up to George Smitherman, the centrist candidate of Toronto’s liberal establishment.

How does The Star know? It’s simple. Lorne Bozinoff, president of the little-known polling firm Forum Research, Inc. told them. Forum Research conducted a telephone poll of 405 Torontonians over the weekend and supplied the results exclusively to the Star.

The Star gives the results as a percentage of decided voters.

George Smitherman: 29%
Rob Ford: 26%
Sarah Thomson: 17%
Joe Pantalone: 12%
Rocco Rossi: 10%
Giorgio Mammoliti: 4%.

The Star then informs that since the poll’s margin of error is 4.9%, 19 times out of 20, Ford and Smitherman are statistically tied. This is problematic. Since Bozinoff informs that 44% of voters are undecided, that means that only about 227 of the 405 respondents actually expressed a preference, and that the margin of error of the results of this subsample is considerably higher.

For a margin of error of 4.9%, all opinions should be represented, which would look like this:

George Smitherman: 16%
Rob Ford: 15%
Sarah Thomson: 10%
Joe Pantalone: 7%
Rocco Rossi: 6%
Giorgio Mammoliti: 2%
Undecided: 44%

Bozinoff not only supplies this article’s facts but also the bulk of its analysis. He claims that it shows, “It’s a dogfight between Ford and Smitherman — they are well ahead of the others.”

This is demonstrably false. Since a poll’s swing can be either plus or minus, it becomes clear that the real story is that no candidate has any significant public support, and that five months away from election day, all the candidates (even Mammoliti), are within striking distance of one another.

Bozinoff also makes a comment about Ford’s support in the area of Etobicoke. Given that Etobicoke represents 13% of Toronto’s population, we can assume that it represents 13% of a diligent pollster’s sample. In this case, that would be about 53 people. That is a very small number of opinions from which to be drawing conclusions so confidently.

All of this makes the critical reader want to take a closer look at the poll’s results. What were the questions asked? In what order? And what is the demographic makeup of the sample? The Star has not made this information available to its readers, and a visit to forumresearch.com turns up no data either. Readers of the Star (including the author) are left having to take these opinion facts on faith.

In the Star’s article, David Rider proves himself an uncritical reporter, and Lorne Bozinoff a pollster with a poor command of his own numbers. They do not inspire trust.

This inaccurate and incomplete piece of journalism is a disservice to the city of Toronto. It can be argued that it does more to shape public opinion than it does to reflect it. To defend its credibility, the Star must make the full results of its polls available to the public and explain why it has printed and distributed these errors to hundreds of thousands of people.

Quick Hit

From the Meta-Urtak, the Urtak about the Urtak Project (it’s in the right sidebar).

Do you personally know a member of the Urtak team?

Right now, it’s 61% yes. I wonder where it will be in 30 days. Remind me if I forget.