The Justice Department has started a preliminary investigation into whether I.B.M. has abused its monopoly position in the market for mainframe computers, which remain vital to many of the world’s largest businesses. This month, antitrust regulators at the Justice Department began seeking information about I.B.M.’s business practices from companies that compete with I.B.M. in the market for large computer hardware and software, people who had been contacted in the inquiry said.
The requests for information followed a complaint filed by the Computer and Communications Industry Association, a trade group with a history of involvement in antitrust disputes. The organization, which is backed by I.B.M. competitors like Microsoft and Oracle, contends that I.B.M. stymied competition in the mainframe market and blocked efforts by competitors and potential partners to license I.B.M.’s software.
The complaint follows similar legal action taken by T3 Technologies against I.B.M.
T3, a small company that resold mainframelike computers, filed an antitrust complaint against I.B.M. in January in Europe. T3 also filed a civil suit against I.B.M. in the United States. Last week a federal district judge in New York dismissed that case. T3 said it planned to appeal.
Steven Friedman, the president of T3, said he had received a formal request for information from the Justice Department about I.B.M.’s actions in the mainframe market. “They asked for a very broad set of documents and information,” he said.
Edward J. Black, the chief executive of the Computer and Communications Industry Association, confirmed that it had filed a complaint against I.B.M. with the Justice Department and that investigators had contacted some members.
A Justice Department spokeswoman declined to comment.
The inquiry is the early stages and may not result in charges against I.B.M. The last time federal regulators pursued an antitrust suit against I.B.M. in the mainframe market, the result was a humbling setback for the department when the Reagan administration dropped the case in 1982, after 13 years.
In a statement on Wednesday, I.B.M. cited the judge’s ruling last week against T3 and said, “We continue to believe there is no merit to T3’s claims.”
“We understand the Department of Justice has asked T3 for documents from the litigation,” I.B.M. said. “I.B.M. intends to cooperate with any inquiries from the Department of Justice.”
While sometimes called the dinosaurs of computing, mainframes continue to play a vital role in business. The systems are estimated to handle 50 billion transactions a day in such areas as automated teller machines, health records and accounting.
Mainframes are also important to I.B.M. About 25 percent of its $104 billion in annual revenue comes from the sale of mainframes and associated products like storage systems, software and services, said A. M. Sacconaghi, a securities analyst with Sanford C. Bernstein.
Historically, I.B.M. has faced off against competitors in the mainframe market. Past rivals, including Amdahl, Hitachi and Fujitsu, built computers that could run I.B.M. software, which was a standard for mainframes.
Most rivals abandoned their mainframe systems when I.B.M. developed more advanced chips for its machines, in part because of the cost of moving to the new chips.
More recently, smaller companies and even individuals have worked to create software to mimic mainframe functions on lower-cost, mainstream computer servers.
One start-up called Platform Solutions had modest success with this approach earlier this decade and drew the interest of Hewlett-Packard, which held discussions about buying the company.
I.B.M., however, declined to license its mainframe software to Platform and sued the company. Last year, I.B.M. acquired Platform for $150 million and discontinued the company’s computer systems.
I.B.M.’s decision not to license its mainframe software has been at the heart of the antitrust complaints against it. In addition, competitors have argued that I.B.M. has stepped up aggressive tactics to block them from selling products that can lower the cost of mainframe technology.
But I.B.M.’s opposition to licensing its technology to outsiders is not enough to build a successful government antitrust case, said Andrew I. Gavil, a law professor at Howard University. More likely, Professor Gavil said, the Justice Department is investigating to see if I.B.M. is engaged in other tactics that might be anticompetitive.
In the ruling in the private case last week, Judge Lewis A. Kaplan of Federal District Court in Manhattan found that I.B.M. had invested heavily in its modern mainframe technology and its decision not to license it “does not constitute anticompetitive conduct.”
The technology industry has undergone an accelerating wave of consolidation as companies like Hewlett-Packard, Oracle, EMC and Dell have made large acquisitions. The deals reflect a desire by large technology companies to offer a wider range of hardware, software and services so that customers see them as one-stop shops.
Analysts contend that I.B.M.’s dominance in the mainframe market gives it a major advantage because it deals with its customers’ most confidential information. They also say that I.B.M.’s practices have resulted in higher costs for customers.
By ASHLEE VANCE and STEVE LOHR
Source: New York Times
Showing posts with label Technology News. Show all posts
Showing posts with label Technology News. Show all posts
Thursday, October 8, 2009
Tuesday, October 6, 2009
Soon, Bloggers Must Give Full Disclosure
FOR nearly three decades, the Federal Trade Commission’s rules regarding the relationships between advertisers and product reviewers and endorsers were deemed adequate. Then came the age of blogging and social media.
On Monday, the F.T.C. said it would revise rules about endorsements and testimonials in advertising that had been in place since 1980. The new regulations are aimed at the rapidly shifting new-media world and how advertisers are using bloggers and social media sites like Facebook and Twitter to pitch their wares.
The F.T.C. said that beginning on Dec. 1, bloggers who review products must disclose any connection with advertisers, including, in most cases, the receipt of free products and whether or not they were paid in any way by advertisers, as occurs frequently. The new rules also take aim at celebrities, who will now need to disclose any ties to companies, should they promote products on a talk show or on Twitter. A second major change, which was not aimed specifically at bloggers or social media, was to eliminate the ability of advertisers to gush about results that differ from what is typical — for instance, from a weight loss supplement.
For bloggers who review products, this means that the days of an unimpeded flow of giveaways may be over. More broadly, the move suggests that the government is intent on bringing to bear on the Internet the same sorts of regulations that have governed other forms of media, like television or print.
“It crushes the idea that the Internet is separate from the kinds of concerns that have been attached to previous media,” said Clay Shirky, a professor at New York University.
Richard Cleland, assistant director of the division of advertising practices at the F.T.C., said: “We were looking and seeing the significance of social media marketing in the 21st century and we thought it was time to explain the principles of transparency and truth in advertising and apply them to social media marketing. Which isn’t to say that we saw a huge problem out there that was imperative to address.”
Still, sites like Twitter and Facebook, as well as blogs, have offered companies new opportunities to pitch products with endorsements that carry a veneer of authenticity because they seem to be straight from the mouth — or keyboard — of an individual consumer. In some cases, companies have set up product review blogs that appear to be independent. One such case involved Urban Nutrition, a seller of supplements, that ran Web sites like WeKnowDiets.com and GoogleDiets.com. The National Advertising Review Council, which governs the industry’s self-regulatory programs, said the sites were “formatted as independent product-review blogs.”
Jonathan Zittrain, a professor at Harvard Law School and co-founder of the Berkman Center for Internet and Society, said, “the rules are looking ahead to a quite possible future when there is a market to buy ‘authentic’ public endorsements.”
Some marketing groups fought the changes. “If a product is provided to bloggers, the F.T.C. will consider that, in most cases, to be a material connection even if the advertiser has no control over the content of the blogs,” said Linda Goldstein, a partner at Manatt Phelps & Phillips, a law firm that represents three marketing groups, the Electronic Retailing Association, the Promotion Marketing Association and the Word of Mouth Marketing Association. “In terms of the real world blogging community, that’s a seismic shift.”
Ms. Goldstein added, “We would have preferred the F.T.C. to work closer with the industry to learn how viral marketing works.”
The new guidelines were not unexpected — the commission gave notice last November that it would take up the matter. They will affect scores of bloggers who began as hobbyists only to find that companies flocked to them in search of a new way to reach consumers.
About three-and-a-half years ago Christine Young, of Lincoln, Calif., began blogging about her adventures in home schooling. It led to her current blog, FromDatesToDiapers.com, about mothers and families. The free products soon started arriving, and now hardly a day goes by without a package from Federal Express or DHL arriving at her door, she said. Mostly they are children’s products, like Nintendo Wii games, but sometimes not. She said she recently received a free pair of women’s shoes from Timberland.
Ms. Young said she had always disclosed whether or not she received a free product when writing her reviews. But companies have nothing to lose when sending off goodies: if she doesn’t like a product, she simply won’t write about it.
“I think that bloggers definitely need to be held accountable,” said Ms. Young. “I think there is a certain level of trust that bloggers have with readers, and readers deserve to know the whole truth.”
Source: New York Times
On Monday, the F.T.C. said it would revise rules about endorsements and testimonials in advertising that had been in place since 1980. The new regulations are aimed at the rapidly shifting new-media world and how advertisers are using bloggers and social media sites like Facebook and Twitter to pitch their wares.
The F.T.C. said that beginning on Dec. 1, bloggers who review products must disclose any connection with advertisers, including, in most cases, the receipt of free products and whether or not they were paid in any way by advertisers, as occurs frequently. The new rules also take aim at celebrities, who will now need to disclose any ties to companies, should they promote products on a talk show or on Twitter. A second major change, which was not aimed specifically at bloggers or social media, was to eliminate the ability of advertisers to gush about results that differ from what is typical — for instance, from a weight loss supplement.
For bloggers who review products, this means that the days of an unimpeded flow of giveaways may be over. More broadly, the move suggests that the government is intent on bringing to bear on the Internet the same sorts of regulations that have governed other forms of media, like television or print.
“It crushes the idea that the Internet is separate from the kinds of concerns that have been attached to previous media,” said Clay Shirky, a professor at New York University.
Richard Cleland, assistant director of the division of advertising practices at the F.T.C., said: “We were looking and seeing the significance of social media marketing in the 21st century and we thought it was time to explain the principles of transparency and truth in advertising and apply them to social media marketing. Which isn’t to say that we saw a huge problem out there that was imperative to address.”
Still, sites like Twitter and Facebook, as well as blogs, have offered companies new opportunities to pitch products with endorsements that carry a veneer of authenticity because they seem to be straight from the mouth — or keyboard — of an individual consumer. In some cases, companies have set up product review blogs that appear to be independent. One such case involved Urban Nutrition, a seller of supplements, that ran Web sites like WeKnowDiets.com and GoogleDiets.com. The National Advertising Review Council, which governs the industry’s self-regulatory programs, said the sites were “formatted as independent product-review blogs.”
Jonathan Zittrain, a professor at Harvard Law School and co-founder of the Berkman Center for Internet and Society, said, “the rules are looking ahead to a quite possible future when there is a market to buy ‘authentic’ public endorsements.”
Some marketing groups fought the changes. “If a product is provided to bloggers, the F.T.C. will consider that, in most cases, to be a material connection even if the advertiser has no control over the content of the blogs,” said Linda Goldstein, a partner at Manatt Phelps & Phillips, a law firm that represents three marketing groups, the Electronic Retailing Association, the Promotion Marketing Association and the Word of Mouth Marketing Association. “In terms of the real world blogging community, that’s a seismic shift.”
Ms. Goldstein added, “We would have preferred the F.T.C. to work closer with the industry to learn how viral marketing works.”
The new guidelines were not unexpected — the commission gave notice last November that it would take up the matter. They will affect scores of bloggers who began as hobbyists only to find that companies flocked to them in search of a new way to reach consumers.
About three-and-a-half years ago Christine Young, of Lincoln, Calif., began blogging about her adventures in home schooling. It led to her current blog, FromDatesToDiapers.com, about mothers and families. The free products soon started arriving, and now hardly a day goes by without a package from Federal Express or DHL arriving at her door, she said. Mostly they are children’s products, like Nintendo Wii games, but sometimes not. She said she recently received a free pair of women’s shoes from Timberland.
Ms. Young said she had always disclosed whether or not she received a free product when writing her reviews. But companies have nothing to lose when sending off goodies: if she doesn’t like a product, she simply won’t write about it.
“I think that bloggers definitely need to be held accountable,” said Ms. Young. “I think there is a certain level of trust that bloggers have with readers, and readers deserve to know the whole truth.”
Source: New York Times
Friday, September 25, 2009
Twitter Appears Set to Raise $100 Million, Valuing It at $1 Billion
SAN FRANCISCO — Twitter has trained people to compress their thoughts into 140 characters and given a public stage to both dissidents in Iran and voluble stars like Shaquille O’Neal.
Now the start-up appears to have chalked up another achievement. Twitter, which has no discernible revenue, is set to raise about $100 million of new funding that would value the company at around $1 billion, a person briefed on the company’s plans said Thursday.
For context, that is almost double the market capitalization of Domino’s Pizza, which has 10,500 employees and had $1.4 billion in sales last year. Twitter has some 60 employees, and although it is experimenting with running advertisements on its Web site, Biz Stone, a Twitter founder, said this week at an industry conference that the company had no plans to begin widely running ads until 2010.
But Twitter’s cash infusion and exospheric valuation are not easily reduced to the level of the blind bets of past dot-com bubbles. In its three and a half years, Twitter has become a magnet for media attention, and its Web site now attracts 54 million visitors a month, according to comScore, the tracking firm. Along with Facebook, it is helping to remake the Web as a forum for the perpetual sharing of even the most trivial bits of information about people’s lives.
“There have probably been less than five examples of companies that have grown like Twitter has,” said John Borthwick, the chief executive of Betaworks, which created the link-shortening service Bit.ly. (Betaworks also invested in Summize, a Twitter search engine that Twitter acquired last year, and it now owns a small stake in the company.)
Mr. Borthwick lists Google and Facebook as other examples. Twitter “represents a next layer of innovation on the Internet,” he said. “This investment is happening because it represents a shift.”
The new investors include Insight Venture Partners, a venture capital firm based in New York; T. Rowe Price, the mutual fund company, which is not normally known for placing such bets; and the current Twitter backers Spark Capital and Institutional Venture Partners.
The investment is likely to kick off more discussion about the heady valuations investors are assigning to some Internet start-ups, even as the United States economy struggles to emerge from a deep recession and the window for initial public offerings remains weak.
Twitter is what insiders charitably describe as pre-revenue, and the service has become known for going down periodically, although its reliability record has been improving lately.
To some, Twitter’s new valuation makes sense. Facebook, Google and Microsoft have all reportedly made entreaties to acquire the company, and its desirability to the Internet giants elevates its value.
Then there is the nonstop media attention, with everyone from Oprah Winfrey to local radio stations increasingly using the service to communicate with fans. “There is so much media hype around them, it was probably easy to go to mainstream investors and find someone who would be interested,” said Jeremiah Owyang, a social media analyst at the Altimeter Group.
Twitter has not yet commented on the investment, so it is not clear how it will use the new cash. The company does not appear to need the capital. It previously raised $55 million and has said it still has $25 million of that in the bank. But it is known to have wide aspirations to ultimately reach one billion users and become “the pulse of the planet,” according to internal documents that were illicitly obtained by a hacker and published on the blog TechCrunch earlier this year.
Twitter could use the investment to build the technology infrastructure required to grow to that scale. It also might use the cash to acquire one or more of the companies that are writing Twitter programs for mobile phones and computer desktops.
Twitter could even find a business model for itself if it were to buy one of the several start-ups devoted to helping companies manage their Twitter presence and monitor how their brands are being discussed.
But close followers of Twitter do not sense that the company is in any great rush to prove itself as a profitable venture.
“It would be trivially easy for them to turn on a revenue source today,” said Steve Broback, founder of the Parnassus Group, which runs conferences on Twitter and other business topics. “I don’t see that they are in a big hurry to start generating revenues, mostly because they want to minimize any sort of negative effect on their community.”
Twitter’s newly lined pockets may have the biggest impact on its chief rival, Facebook. Executives from the two companies often claim that their services do not quite overlap and can peacefully coexist. But both firms are essentially on the same mission: to allow people to share with friends and fans what they are doing now, in real life and on the Web.
Despite their protestations, the companies appear, especially recently, to be taking swipes at each other. Last week, when Facebook announced that it had signed up its 300 millionth member and that its finances were strengthening, executives used the occasion to play down any threat posed by Twitter. Chamath Palihapitiya, a Facebook vice president, told the technology blog VentureBeat that Twitter was now in “the rearview mirror.”
Even more pointedly, earlier this year Facebook redesigned the stream of updates from friends that each user sees to be more of a constant flow of information, similar to Twitter. And earlier this month, Facebook began allowing users to “tag” messages about particular friends with the @ symbol, a familiar convention on Twitter.
For its part, Twitter has previewed a new way to allow people to see when other Twitter users have “retweeted” or relayed their messages. The feature looks eerily similar to the display of friends who have commented on, or indicated that they liked, an update on Facebook.
Twitter’s ascension has clearly clouded Facebook’s aspirations to dominate the market for sharing over the Web, said Keith Rabois, an Internet entrepreneur and vice president of strategy at Slide, a Web social entertainment firm.
“Twitter is so likely to be successful at this point, it is almost impossible to envision a way in which Facebook can truly monopolize online content-sharing,” he said.
Now the start-up appears to have chalked up another achievement. Twitter, which has no discernible revenue, is set to raise about $100 million of new funding that would value the company at around $1 billion, a person briefed on the company’s plans said Thursday.
For context, that is almost double the market capitalization of Domino’s Pizza, which has 10,500 employees and had $1.4 billion in sales last year. Twitter has some 60 employees, and although it is experimenting with running advertisements on its Web site, Biz Stone, a Twitter founder, said this week at an industry conference that the company had no plans to begin widely running ads until 2010.
But Twitter’s cash infusion and exospheric valuation are not easily reduced to the level of the blind bets of past dot-com bubbles. In its three and a half years, Twitter has become a magnet for media attention, and its Web site now attracts 54 million visitors a month, according to comScore, the tracking firm. Along with Facebook, it is helping to remake the Web as a forum for the perpetual sharing of even the most trivial bits of information about people’s lives.
“There have probably been less than five examples of companies that have grown like Twitter has,” said John Borthwick, the chief executive of Betaworks, which created the link-shortening service Bit.ly. (Betaworks also invested in Summize, a Twitter search engine that Twitter acquired last year, and it now owns a small stake in the company.)
Mr. Borthwick lists Google and Facebook as other examples. Twitter “represents a next layer of innovation on the Internet,” he said. “This investment is happening because it represents a shift.”
The new investors include Insight Venture Partners, a venture capital firm based in New York; T. Rowe Price, the mutual fund company, which is not normally known for placing such bets; and the current Twitter backers Spark Capital and Institutional Venture Partners.
The investment is likely to kick off more discussion about the heady valuations investors are assigning to some Internet start-ups, even as the United States economy struggles to emerge from a deep recession and the window for initial public offerings remains weak.
Twitter is what insiders charitably describe as pre-revenue, and the service has become known for going down periodically, although its reliability record has been improving lately.
To some, Twitter’s new valuation makes sense. Facebook, Google and Microsoft have all reportedly made entreaties to acquire the company, and its desirability to the Internet giants elevates its value.
Then there is the nonstop media attention, with everyone from Oprah Winfrey to local radio stations increasingly using the service to communicate with fans. “There is so much media hype around them, it was probably easy to go to mainstream investors and find someone who would be interested,” said Jeremiah Owyang, a social media analyst at the Altimeter Group.
Twitter has not yet commented on the investment, so it is not clear how it will use the new cash. The company does not appear to need the capital. It previously raised $55 million and has said it still has $25 million of that in the bank. But it is known to have wide aspirations to ultimately reach one billion users and become “the pulse of the planet,” according to internal documents that were illicitly obtained by a hacker and published on the blog TechCrunch earlier this year.
Twitter could use the investment to build the technology infrastructure required to grow to that scale. It also might use the cash to acquire one or more of the companies that are writing Twitter programs for mobile phones and computer desktops.
Twitter could even find a business model for itself if it were to buy one of the several start-ups devoted to helping companies manage their Twitter presence and monitor how their brands are being discussed.
But close followers of Twitter do not sense that the company is in any great rush to prove itself as a profitable venture.
“It would be trivially easy for them to turn on a revenue source today,” said Steve Broback, founder of the Parnassus Group, which runs conferences on Twitter and other business topics. “I don’t see that they are in a big hurry to start generating revenues, mostly because they want to minimize any sort of negative effect on their community.”
Twitter’s newly lined pockets may have the biggest impact on its chief rival, Facebook. Executives from the two companies often claim that their services do not quite overlap and can peacefully coexist. But both firms are essentially on the same mission: to allow people to share with friends and fans what they are doing now, in real life and on the Web.
Despite their protestations, the companies appear, especially recently, to be taking swipes at each other. Last week, when Facebook announced that it had signed up its 300 millionth member and that its finances were strengthening, executives used the occasion to play down any threat posed by Twitter. Chamath Palihapitiya, a Facebook vice president, told the technology blog VentureBeat that Twitter was now in “the rearview mirror.”
Even more pointedly, earlier this year Facebook redesigned the stream of updates from friends that each user sees to be more of a constant flow of information, similar to Twitter. And earlier this month, Facebook began allowing users to “tag” messages about particular friends with the @ symbol, a familiar convention on Twitter.
For its part, Twitter has previewed a new way to allow people to see when other Twitter users have “retweeted” or relayed their messages. The feature looks eerily similar to the display of friends who have commented on, or indicated that they liked, an update on Facebook.
Twitter’s ascension has clearly clouded Facebook’s aspirations to dominate the market for sharing over the Web, said Keith Rabois, an Internet entrepreneur and vice president of strategy at Slide, a Web social entertainment firm.
“Twitter is so likely to be successful at this point, it is almost impossible to envision a way in which Facebook can truly monopolize online content-sharing,” he said.
Thursday, September 24, 2009
Starbucks Turns an iPhone Screen into a Gift Card
Starbucks is introducing two new applications for the iPhone that will make it easier for java junkies to get their fix—and make it possible to pay right from the phone, which has broad implications for mobile commerce.
The myStarbucks application has a slew of features that make it easier to remember your friend’s favorite drinks and to locate nearby Starbucks stores. More interesting is a test of a Starbucks card, which will allow people in select West Coast stores to pay for coffee using a bar code on a phone’s screen.
The myStarbucks app, which is usable anywhere, lets you store the recipe for your favorite coffee concoction and to share it with other people. You can send your request for a Grande Skinny Caramel Macchiato with two sugars to the office coffee slave and be assured that they get it right.
Don’t know what you want to drink? A flavor selector helps you choose a coffee based on flavors like earthy, balanced or nutty. You can also look up the nutritional information, like the calorie count of your drink (you’ll soon be switching to skim).
The myStarbucks application has a slew of features that make it easier to remember your friend’s favorite drinks and to locate nearby Starbucks stores. More interesting is a test of a Starbucks card, which will allow people in select West Coast stores to pay for coffee using a bar code on a phone’s screen.
The myStarbucks app, which is usable anywhere, lets you store the recipe for your favorite coffee concoction and to share it with other people. You can send your request for a Grande Skinny Caramel Macchiato with two sugars to the office coffee slave and be assured that they get it right.
Don’t know what you want to drink? A flavor selector helps you choose a coffee based on flavors like earthy, balanced or nutty. You can also look up the nutritional information, like the calorie count of your drink (you’ll soon be switching to skim).
Tuesday, September 22, 2009
Netflix Awards $1 Million Prize and Starts a New Contest
Netflix, the movie rental company, has decided its million-dollar-prize competition was such a good investment that it is planning another one.
The company’s challenge, begun in October 2006, was both geeky and formidable: come up with a recommendation software that could do a better job accurately predicting the movies customers would like than Netflix’s in-house software, Cinematch. To qualify for the prize, entries had to be at least 10 percent better than Cinematch.
The winner, formally announced Monday morning, is a seven-person team of statisticians, machine-learning experts and computer engineers from the United States, Austria, Canada and Israel. The multinational team calls itself BellKor’s Pragmatic Chaos. The group — a merger of teams — was the longtime frontrunner in the contest, and in late June it finally surpassed the 10 percent barrier. Under the rules of the contest, that set off a 30-day period in which other teams could try to beat them.
That, in turn, prompted a wave of mergers among competing teams, who joined forces at the last minute to try to top the leader. In late July, Netflix declared the contest over and said two teams had passed the 10-percent threshold, BellKor and the Ensemble, a global alliance with some 30 members. Netflix publicly said the finish was too close to call. But Netflix officials at the time privately informed BellKor it had won. Though further review of the algorithms by expert judges was needed, it certainly seemed BellKor was the winner, as it turned out to be.
But the race was even closer than had been thought, as Netflix’s chief executive, Reed Hastings, explained for the first time at a press conference in New York on Monday. The BellKor team presented its final submission 20 minutes before the deadline, Mr. Hastings said. Then, just before time ran out, The Ensemble made its last entry. The two were a dead tie, mathematically. But under contest rules, when there is a tie, the first team past the post wins.
“That 20 minutes was worth $1 million,” Mr. Hastings said.
The Netflix contest has been widely followed because its lessons could extend well beyond improving movie picks. The researchers from around the world were grappling with a huge data set — 100 million movie ratings — and the challenges of large-scale predictive modeling, which can be applied across the fields of science, commerce and politics.
The way teams came together, especially late in the contest, and the improved results that were achieved suggest that this kind of Internet-enabled approach, known as crowdsourcing, can be applied to complex scientific and business challenges.
That certainly seemed to be a principal lesson for the winners. The blending of different statistical and machine-learning techniques “only works well if you combine models that approach the problem differently,” said Chris Volinsky, a scientist at AT&T Research and a leader of the Bellkor team. “That’s why collaboration has been so effective, because different people approach problems differently.”
Yet the sort of sophisticated teamwork deployed in the Netflix contest, it seems, is a tricky business. Over three years, thousands of teams from 186 countries made submissions. Yet only two could breach the 10-percent hurdle. “Having these big collaborations may be great for innovation, but it’s very, very difficult,” said Greg McAlpin, a software consultant and a leader of the Ensemble. “Out of thousands, you have only two that succeeded. The big lesson for me was that most of those collaborations don’t work.”
The data set for the first contest was 100 million movie ratings, with the personally identifying information stripped off. Contestants worked with the data to try to predict what movies particular customers would prefer, and then their predictions were compared with how the customers actually did rate those movies later, on a scale of one to five stars.
The new contest is going to present the contestants with demographic and behavioral data, and they will be asked to model individuals’ “taste profiles,” the company said. The data set of more than 100 million entries will include information about renters’ ages, gender, ZIP codes, genre ratings and previously chosen movies. Unlike the first challenge, the contest will have no specific accuracy target. Instead, $500,000 will be awarded to the team in the lead after six months, and $500,000 to the leader after 18 months.
The payoff for Netflix? “Accurately predicting the movies Netflix members will love is a key component of our service,” said Neil Hunt, chief product officer.
The company’s challenge, begun in October 2006, was both geeky and formidable: come up with a recommendation software that could do a better job accurately predicting the movies customers would like than Netflix’s in-house software, Cinematch. To qualify for the prize, entries had to be at least 10 percent better than Cinematch.
The winner, formally announced Monday morning, is a seven-person team of statisticians, machine-learning experts and computer engineers from the United States, Austria, Canada and Israel. The multinational team calls itself BellKor’s Pragmatic Chaos. The group — a merger of teams — was the longtime frontrunner in the contest, and in late June it finally surpassed the 10 percent barrier. Under the rules of the contest, that set off a 30-day period in which other teams could try to beat them.
That, in turn, prompted a wave of mergers among competing teams, who joined forces at the last minute to try to top the leader. In late July, Netflix declared the contest over and said two teams had passed the 10-percent threshold, BellKor and the Ensemble, a global alliance with some 30 members. Netflix publicly said the finish was too close to call. But Netflix officials at the time privately informed BellKor it had won. Though further review of the algorithms by expert judges was needed, it certainly seemed BellKor was the winner, as it turned out to be.
But the race was even closer than had been thought, as Netflix’s chief executive, Reed Hastings, explained for the first time at a press conference in New York on Monday. The BellKor team presented its final submission 20 minutes before the deadline, Mr. Hastings said. Then, just before time ran out, The Ensemble made its last entry. The two were a dead tie, mathematically. But under contest rules, when there is a tie, the first team past the post wins.
“That 20 minutes was worth $1 million,” Mr. Hastings said.
The Netflix contest has been widely followed because its lessons could extend well beyond improving movie picks. The researchers from around the world were grappling with a huge data set — 100 million movie ratings — and the challenges of large-scale predictive modeling, which can be applied across the fields of science, commerce and politics.
The way teams came together, especially late in the contest, and the improved results that were achieved suggest that this kind of Internet-enabled approach, known as crowdsourcing, can be applied to complex scientific and business challenges.
That certainly seemed to be a principal lesson for the winners. The blending of different statistical and machine-learning techniques “only works well if you combine models that approach the problem differently,” said Chris Volinsky, a scientist at AT&T Research and a leader of the Bellkor team. “That’s why collaboration has been so effective, because different people approach problems differently.”
Yet the sort of sophisticated teamwork deployed in the Netflix contest, it seems, is a tricky business. Over three years, thousands of teams from 186 countries made submissions. Yet only two could breach the 10-percent hurdle. “Having these big collaborations may be great for innovation, but it’s very, very difficult,” said Greg McAlpin, a software consultant and a leader of the Ensemble. “Out of thousands, you have only two that succeeded. The big lesson for me was that most of those collaborations don’t work.”
The data set for the first contest was 100 million movie ratings, with the personally identifying information stripped off. Contestants worked with the data to try to predict what movies particular customers would prefer, and then their predictions were compared with how the customers actually did rate those movies later, on a scale of one to five stars.
The new contest is going to present the contestants with demographic and behavioral data, and they will be asked to model individuals’ “taste profiles,” the company said. The data set of more than 100 million entries will include information about renters’ ages, gender, ZIP codes, genre ratings and previously chosen movies. Unlike the first challenge, the contest will have no specific accuracy target. Instead, $500,000 will be awarded to the team in the lead after six months, and $500,000 to the leader after 18 months.
The payoff for Netflix? “Accurately predicting the movies Netflix members will love is a key component of our service,” said Neil Hunt, chief product officer.
Subscribe to:
Posts (Atom)
