Big Data vital to CERN Large Hadron Collider project, says CTOParticle physics generated big data long before we began thinking about analyzing social data.
For example, proton and anti-proton collisions at the recently closed Tevatron accelerator at Fermilab generated an estimated 100 MB of data per second since its opening in 1967 - and that data volume represents only 100 Hz of a 1.7 MHz collision due to the limited bandwidth the data was able to be captured.
Imagine the data volume a 40 MHz collision at the Large Hadron Collider (LHC) at CERN can produce.
It is not surprising that scientists at the LHC keep a close eye on Big Data solutions. According to CTO Sverre Jarp, the LHC generated about 30 TB of data in 2012. Techworld quoted Jarp saying that Big Data has taken the organization to put one terabyte of data on one physical disk or tape cartridge," that "we are safely in the domain of petabytes and we are moving to exabytes in the future."
Like the researchers at the Tevatron, LHC scientists do not capture all data, also because of "disk failures every day." In addition, virtually all the data that is captured is unstructured, making the LHC environment a prime customer for Big Data solutions. However, he said that it is important to move unstructured into a structured form as "Big Data management and analytics require a solid organizational structure at all levels."
Jarp noted that the LHC will be running for another 15 to 20 years and produce data in the exabyte range. And the next collider is already waiting. The staff at Fermilab has built a Superconducting RF (SRF) test facility as a model for a new super collider to be built in Illinois - a collider that would exceed the capabilities of the LHC in particle physics and data generation.
Wolfgang GruenerWolfgang Gruener is a contributor to Tom's IT Pro. He is currently principal analyst at Ndicio Research, a market analysis firm that focuses on cloud computing and disruptive technologies, and maintains the conceivablytech.com blog. An 18-year veteran in IT journalism and market research, he previously published TG Daily and was managing editor of Tom's Hardware news, which he grew from a link collection in the early 2000s into one of the most comprehensive and trusted technology news sources.
See here for all of Wolfgang's Tom's IT Pro articles.
Check Out These IT Videos