eBay, which runs a gigantic data warehouse for internal business intelligence (BI) analytics, is considering taking it outside its firewall and offering it as a Web service to interested companies.
That would duplicate a move by Amazon.com, which built its family of Amazon Web Services for in-house use but is now marketing them to outside customers. Most notable among these services are its Elastic Compute Cloud (EC2) application hosting service and its S3 hosted storage service.
In eBay's case, the online auctioneer has built a 5 petabyte data warehouse that adds 50 TB of new data each day.
One terabyte every 5 seconds
With the Teradata-based data warehouse able to turn over a terabyte of data in just 5 seconds, eBay has taken advantage of that speed to enable business analysts to build their own "virtual" data marts, according to Oliver Ratzesberger, senior director of architecture and operations at the company.
The virtual data marts are used by about 5,000 business analysts in 100 groups inside eBay. These data marts run off the central data warehouse but are created without the help of central IT, said Ratzesberger.
Business analysts create and upload their own mini-data warehouses using standard Web and analytical tools such as those from Business Objects, SAS and Microstrategy -- and even Microsoft's Excel. This lets the analysts rapidly create and test prototypes of the BI analyses they think they want. After 90 days, successful prototypes are brought to the data warehousing managers, who convert them into production data marts with a minimal level of rewriting.
"We cut the amount of time needed to build a data mart at least in half, and in some cases up to three to five times," said Ratzesberger.
So is eBay thinking of turning its data warehouse into a utility that can be used by outside subscribing firms?
"Yes we are thinking about it," said Ratzesberger. He acknowledged one problem is how to minimize the time it would take customers to upload large amounts of data to eBay's data warehouse.