While digital transmission rates may be approaching terabit speeds, more precisely "tebibit" (2^40 or 1,099,511,627,776 bits per second), data storage facilities such as Apple's new North Carolina facility, scheduled to go online Q2, 2011, have already surpassed exbibit capacities—an exbibit representing 2^60 or, in decimal, 1,152,921,504,606,846,976 bits. In more human terms, this number may be thought of as (in short scale countries such as the United States) "one-hundred forty-four quadrillion, one-hundred fifteen trillion" typewritten characters.

HI Mark,

The storage and bandwidth are the easy part, relatively speaking of course. As long as there is healthy competition (plus consumer and corporate demand) in the storage and telecom markets both the cost, density and amount of data that we can securely store and efficiently move from point a to point b will continue to improve exponentially, and probably with some significant breakthroughs in either or , or both, that within the next decade will exceed the current thinking of most people in the field.

There are already technologies in what I call the “breakthrough category” that are in use today, but not have made their way into the main stream commercial markets.

That said, the catch-up field, imho, is in the computational dynamics, for analyzing and sifting through massive non-static data pools. This will involve greater leaps in parallelism and distributed computing to bring down the cost and make its usage fully transparent to the consumer. The cost involved in providing I/O or real compute power is not as cheap as storage or bandwidth, this is where the technology needs to focused, otherwise all of the data storage is for naught or possibly backup, stuff hanging around in a closet that you can’t really use.

As we talk about terabytes in the same manner that we talked about megabytes, and are now talking about petabytes the way we use to refer to gigabytes, the need to perform computational, sophisticated search and linguistics functions across both structured, semi-structured and unstructured data in order to yield useful information increases and if you need 400 blade servers to process commands or 24 hours to yield results, well that just won’t cut it in today’s world.

Everything has to simultaneously become smarter in order to be effective, storage and bandwidth are core technologies, we need a couple of software layers, and smarter computing efficiencies to make use of the data that we can now store and move.

Good Luck,
Peter
http://www.csi1000.com

April 2011