A.I. accurately predicted the full baseball post-season back in July
marketwired.comThis reminds me of one of the chapters from "How Not to Be Wrong: The Power of Mathematical Thinking" by Jordan Ellenberg (highly recommended). He describes how "stock brokers" would send out a "free stock prediction" to thousands of email addresses. The prediction would be a simple up/down prediction for a specific stock. The prediction was randomly chosen. But these "brokers" would send an equal number of up and down predictions, ensuring that they got a correct prediction for half of their recipients. They would then throw away half of the emails (the wrong half), and repeat with the remaining half. After ten predictions, there would still be a small number of people remaining for whom they'd sent only correct predictions to (10 in a row, which seems really impressive if you can't see the full picture). They would then contact these few people and offer to keep selling them predictions for a fee.
Stories like this (And Paul the Octopus, who I see was mentioned already) are exactly the same thing. Thousands of people are trying to using deep learning (i.e. stats), or other crazy methods as in this article, to make predictions. Of course every now and then one of them is going to work better than expected. This would be the case even if people were simply using random numbers. But we ignore all the ones that fail and give heaps of attention to the Pauls.
If anyone is interested, this is known as p-hacking in statistics (https://en.wikipedia.org/wiki/Data_dredging), and works in a similar way.
For instance, you have a statistical population of one hundred men and one hundred women: you collect as much data as possible about them - as many features as possible, actually - until you find something which happens to be statistically significant for your group (eg. salt consumption). Then, you publish your results, pretending that the feature you found was the original hypothesis for the study ("Our study confirms that salt consumption is higher in males.")
It would be far more specific - you'd collect all their medical details, their ethnicity, age, etc., and then you end up with:
'Salt consumption can increase the risk of liver consumption for middle-aged males of African descent'
... liver consumption ...
I meant liver disease. But I'll leave it this way because it's funnier. And pretty tasty.
"Consumption" is an old-fashioned word for classes of tuberculosis, which can affect the liver. So you could still be right :)
its filled with vitamin a.
I think I see these types of click-bait headlines all the time... and come to think of it, they have very small sample sizes.
Here is a modern version of the same scam [0], using social media accounts and deleting the wrong predictions while the account is set to private.
[0] https://medium.com/message/how-to-always-be-right-on-the-int...
Fantastic comment! In fact, it seems that sports games, or at least NBA games, can be described accurately and consistently using (slightly modified) random walks. Put differently: Outcomes are indeed random and there's not much machine learning you can do here.
Source: https://arxiv.org/abs/1109.2825
And here's a slightly more exciting description of a talk one of the authors gave on that topic at UMass Amherst last year:
https://www.physics.umass.edu/seminars/statistics-of-basketb...
EDIT: I was too stupid to realize that the paper linked above actually supports the parent's opinion, i.e. the idea that successful predictions are statistical artifacts, contrary to what I was thinking earlier.
"The general root of superstition is that men observe when things hit, and not when they miss, and commit to memory the one, and pass over the other." — Sir Francis Bacon
Derren Brown did the same thing with horse racing
But this isn't that at all.
1. They made the predictions well before hand and released them to the public.
2. As the article stated, they also did the same thing with Hockey, Derby, and Academy Awards.
If there were an extremely large number of AI's making all those predictions publicly in advance, so many that one might randomly do that well, then the comment would be accurate. But that does not appear to be the case.
There was absolutely SOME luck involved, however, because I don't believe that, for instance, there is zero randomness in the World Series, which would have to be the case if one could absolutely predict it accurately.
[UPDATE: to be clear, I'm assuming that Unanimous didn't make thousands of similarly high-level predictions, and then only report the ones that did well. I think that's a reasonable assumption, because there aren't thousands of high-level predictions on the level of the Oscars and World Series.]
[UPDATE 2: I just registered at the site. It appears that many people can ask the same question, many times. The same question looks like it can be asked, in fact, many thousands of times. If they were simply cherry-picking the one answer out of thousands that was correct, then this is p-hacking. However, the press release is listing questions asked by prominent entities such as Newsweek and TechRepublic. There aren't all that many of such entities asking such questions of UNU. So the water is a little murky, but it still looks like UNU is doing something impressive.]
This technique was also described on The Simpsons: http://simpsons.wikia.com/wiki/Professor_Pigskin
How dare you blaspheme against our prophet, Paul?
(no seriously, great comment)
Except this was a prediction that was done formally for the Boston Globe, at their request. You can see their article about it here:
https://www.bostonglobe.com/sports/redsox/2016/10/04/group-g...
That's pretty different than sending out thousands of random predictions. This was ONE prediction about MLB.
But we don't know how many other predictions were also formally done, by other entities. We're only hearing about this one because it was right.
They predicted the Kentucky Derby (Superfecta) using this same A.I., based on a challenge from another reporter:
http://www.newsweek.com/artificial-intelligence-turns-20-110...
It probably would be more useful if you disclose your connection to the company, and then gave us some technical arguments.
At the moment your comment history doesn't make a great argument, eg: https://news.ycombinator.com/item?id=11663155
Nothing about this seems to add up.
They claim they made the prediction in early July, but link to a newspaper article dated 4 August that indicates the predictions were made just one day earlier.
They picked the team with the best record all season long to win the championship. They got one of the division winners wrong.
Just publishing the current favorites from MLB.com's probability page [0] as of 3 August would have also gotten 9 of 10 postseason teams correct, including going 6/6 on division winners. So the 'knowledge' of fans voting actually did worse than a monte carlo simulation.
I'm not impressed.
There's no way this should be considered predicting the "full baseball post-season," and I am not seeing any evidence that it happened in July. Wish they'd have shared it.
[0] http://mlb.com/mlb/standings/probability.jsp?ymd=20161002
They tend to publish academic papers about the predictions. This one is obviously too recent to review, but here is an academic paper (IEEE) about their SUPERBOWL PREDICTIONS, complete with formal statistics:
http://unu.ai/wp-content/uploads/2016/10/Crowds-Vs-Swarms-SH...
By "They tend to publish academic papers" you mean you used an "academic paper" template and uploaded it to your website.
According to the article:
"A group of Boston Globe readers accurately predicted nine of baseball’s 10 playoff teams after participating in a 30-minute online experiment using Unanimous A.I.’s Swarm Intelligence on Aug. 3."
So they don't credit the AI as much as the readers. I agree it is all fishy. Someone trying to pump up the value of their company.
Actually, that's how a Swarm Intelligence works - it's a real-time system that connects LIVE PEOPLE using swarming algorithms.
So, the Boston Globe provided the people and provided the questions... they formed a Swarm Intelligence, and made the predictions.
The Boston Globe did this to see if the swarm intelligence could make strong picks. It did.
I get it, but there is a little bit of dishonesty in saying it was the system that did the work. It was the system that automated discovery of a solution, but it was the people that did the work.
As I pointed out in a different post, this is an update of the established technique of delta polling. Delta polling is useful, and an automated way of doing it can help if find even more uses for a lower cost. I see the value here. But, it isn't AI, and the system isn't doing the assessment. It is not intelligence.
There's also the issue of the full suite of predictions, if these were the only predictions made then it's impressive, but if they made lots of predictions then some of their predictions coming true may be no better than chance.
They also predicted which managers would win the MVP awards, and which players would win the CY YOUNG awards but those don't get announced for 2 weeks.
Managers don't win MVP awards.
Well, The Boston Globe only made the one set of predictions.
UNU seems to get their press releases on here a lot. As far as I can see there's not much "AI" involved, just a UI over the "wisdom of crowds" method of making predictions. In this case, the Cubs were heavily favored all season to win the World Series, had arguably one of the best GMs and managers in baseball, and a raft of all-star players. Goat aside, it was fairly smart money to lean towards them from mid-season on.
Same thing with their Kentucky Derby prediction this year. The swarm literally decided the horses in the exact odds they were going off at (which makes sense since gambling odds by their very nature are "the wisdom of the crowd") and that's how they finished.
Agreed - predicting that the Cubs would win the world series isn't impressive - the majority of SI writers [4/7] did that at the beginning of the season: http://www.si.com/mlb/2016/03/31/playoff-picks-awards-picks-...
Correctly predicting who would advance in the post season is mostly luck.
Not to be overly critical but:
It does not match my definition of A.I:
"UNU enables groups of online users to think together as a unified emergent intelligence -- a "brain of brains" that can express itself as a singular entity. Touted to as the world's first "hive mind," the UNU platform has had over 60,000 human participants in swarming sessions this year, together answering over 250,000 questions."
Also I would reasonably expect some of those 250.000 questions to beat the odds and get answered right.
Except this was a prediction that was done formally for the Boston Globe, at their request. You can see their article about it here:
https://www.bostonglobe.com/sports/redsox/2016/10/04/group-g...
Still, this is not "AI" in the traditional sense of the phrase. Asking a bunch of humans and then deciding an outcome is not AI.
Well, I think it's pretty clearly an "emergent intelligence" that is distinct from any of the individuals' unique intelligence.
In other words, whose intelligence is being represented by the swarm?
Corporations have been collectively intelligent for centuries, but we don't call that AI.
"Wisdom of crowds" might be the correct term.
1) The AI was just sythesizing answers given by human readers. It didn't do any of its own analysis of the data set.
2) The experiment was published in August, when the regular season was already two thirds completed. The cubs were well ahead of everybody at that point and were favourites to win (although in baseball that doesn't necessarily mean you are going to win in the postseason). Here are the standings at that Date: http://www.baseball-reference.com/games/standings.cgi?year=2...
You can see that the 10 playoff teams were ranked 1-5 in each league at that point. So predicting the playoff teams was just "Which 10 teams are leading right now", which they asked humans about.
The AI didn't predict the full post-season, just which two teams would be in the World Series, which happened to be the team everybody thought it would be from one league and the second placed team from the other.
This reminds me very much of delta polling, where you survey experts in a field with a complex and unsolvable question, tally the results, send that information back to the experts, and then ask them again. After a few rounds this tends to arrive at what is usually a pretty solid answer.
It is used sometimes in scientific and medical research. An automated tool is pretty neat, but like others said, it doesn't really classify as AI. I'm not sure how much money I would really put down on the bets the site makes, but it is similar in some ways to the scandal that rocked Draft Kings/Fan Duel, where admins were using high-level data to make bets on opposing systems. They did in fact make money.
It irritates me that this is called "A.I".
Anyone remember Tamara Rand. [0]
Well, one of the greatest Tamara Rand jokes was from CNN sports tonight: "The Cubs are predicted to win the World Series. Only thing is it was predicted by Tamara Rand."
Quite cool at a time when tv commentary was never light hearted.
What UNU does is more like "An live online poll of a group of people picked the post-season in July".
How many AIs screwed it up? Remember the hits, forget the misses.
I'd be curious to know what else they predicted that turned out to be wrong. This could be an impressive run, or it could be that the company's press release highlights several victories and omits several (or more) failures.
I have no evidence one way or the other but would be interested to see more context.
Has Unu made any incorrect predictions? Their blog only seems to cover the big, successful ones.
In response to such skepticism, reporters come up with their own questions and ask UNU to make predictions. And the reporters monitor the process. That's what this set of picks is - it was done for the BOSTON GLOBE, at their request, with their own participants:
https://www.bostonglobe.com/sports/redsox/2016/10/04/group-g...
That doesn't actually answer the question.
Survivorship bias.
Also why the stated historical performance for your 401k funds are probably tricking you.
Oh cool. How many AIs did they have doing the predictions? Survivorship bias.
The article mentions "swarm intelligence" that essentially forms a hive-mind. Where is the AI/ML when it seems like it just picks the most popular responses from its many respondents?
Here is the latest UNU election pick: http://unu.ai/election-fatigue/
What are the chances? Probably not that slim considering how many people are trying to make predictions using methods like this.
What a game. What a series.
Hi and welcome to Hacker News! Please only post comments that add something meaningful to the topic of discussion (a proclaimed artificial intelligence that claims to have predicted this result much earlier).
Found this forward-looking post on which states will pass marijuana legalization ballot issues. http://unu.ai/legalization/
Who does the A.I. say will win the election next week?
I'd prefer not to have to go to click a link, seeing that you were replying and could've just included the info.
So could you. Would have made others like you happier.
wow, I sure do lack self reflection
Unsurprisingly it's Clinton.
I'd love to see an update; 538 has cut a third off of Trump's chance.
This is a more recent post http://unu.ai/election-fatigue/
Now what does Paul the octopus say?
Nothing. He's dead.
Hillary Clinton
