Petabytes

The Petabyte Frontier

Excerpted from CommWeb's Telecom Weekly Bulletin - 28 Feb 2002

Forget Megs, Gigs or Terabytes - The Data Frontier Is Even Bigger

A petabyte of data is difficult to fathom. Think of it as the equivalent of 250 billion pages of text, enough to fill 20 million four-drawer filing cabinets. Or imagine a 2,000-mile-high tower of 1 billion diskettes. Whatever you do, don't stop there -- because it's the amount of data many businesses will be managing within the next five years.

The amount of data the average business collects and stores is doubling each year. If that holds true at a company such as Sears Roebuck & Co., which is combining its customer and inventory data warehouses to create a 70-terabyte system, the retailer will hit the 1 petabyte threshold -- 1,000 terabytes -- within four years.

So far, no one's cracked the petabyte milestone. But somewhere between five and 10 databases, mostly in government and university laboratories, store several hundred terabytes and are quickly approaching 1 petabyte. One of the biggest resides at the Stanford Linear Accelerator Center in Menlo Park, Calif., a laboratory at Stanford University where particle-physics researchers study the reams of data generated by the laboratory's particle accelerator in an effort to understand the relationship between matter and antimatter.

Researchers add 2 terabytes of data every day to an already-brimming 500-terabyte database, leaving the IT staff to wrestle with the task of managing all that information. The laboratory keeps 40 terabytes of data on disk for quick access and the rest on tape, yet still connected to the database. It expects to hit the petabyte mark early next year.

(Thanks to Liam Leahy for passing this on.)