Particle physics has an ambitious and broad experimental programme for the
coming decades. This programme requires large investments in detector hardware,
either to build new facilities and experiments, or to upgrade existing ones.
Similarly, it requires commensurate investment in the R&D of software to
acquire, manage, process, and analyse the shear amounts of data to be recorded.
In planning for the HL-LHC in particular, it is critical that all of the
collaborating stakeholders agree on the software goals and priorities, and that
the efforts complement each other. In this spirit, this white paper describes
the R&D activities required to prepare for this software upgrade.
Historically, high energy physics computing has been performed on large
purpose-built computing systems. These began as single-site compute facilities,
but have evolved into the distributed computing grids used today. Recently,
there has been an exponential increase in the capacity and capability of
commercial clouds. Cloud resources are highly virtualized and intended to be
able to be flexibly deployed for a variety of computing tasks. There is a
growing nterest among the cloud providers to demonstrate the capability to
perform large-scale scientific computing. In this paper, we discuss results
from the CMS experiment using the Fermilab HEPCloud facility, which utilized
both local Fermilab resources and virtual machines in the Amazon Web Services
Elastic Compute Cloud. We discuss the planning, technical challenges, and
lessons learned involved in performing physics workflows on a large-scale set
of virtualized resources. In addition, we will discuss the economics and
operational efficiencies when executing workflows both in the cloud and on
The Contribution for the Computing for the Energy Frontier as part of the
Snowmass study is discussed.
In this presentation the experiences of the LHC experiments using grid
computing were presented with a focus on experience with distributed analysis.
After many years of development, preparation, exercises, and validation the LHC
(Large Hadron Collider) experiments are in operations. The computing
infrastructure has been heavily utilized in the first 6 months of data
collection. The general experience of exploiting the grid infrastructure for
organized processing and preparation is described, as well as the successes
employing the infrastructure for distributed analysis. At the end the expected
evolution and future plans are outlined.
The CMS Integration Grid Testbed (IGT) comprises USCMS Tier-1 and Tier-2
hardware at the following sites: the California Institute of Technology, Fermi
National Accelerator Laboratory, the University of California at San Diego, and
the University of Florida at Gainesville. The IGT runs jobs using the Globus
Toolkit with a DAGMan and Condor-G front end. The virtual organization (VO) is
managed using VO management scripts from the European Data Grid (EDG). Gridwide
monitoring is accomplished using local tools such as Ganglia interfaced into
the Globus Metadata Directory Service (MDS) and the agent based Mona Lisa.
Domain specific software is packaged and installed using the Distrib ution
After Release (DAR) tool of CMS, while middleware under the auspices of the
Virtual Data Toolkit (VDT) is distributed using Pacman. During a continuo us
two month span in Fall of 2002, over 1 million official CMS GEANT based Monte
Carlo events were generated and returned to CERN for analysis while being
demonstrated at SC2002. In this paper, we describe the process that led to one
of the world's first continuously available, functioning grids.