Accomplishments IceCube Project Monthly Report - December 2009 Drill camp setup and commissioning was completed three days ahead of the schedule for this season. Deep drilling commenced on 4 December 2009. A total of 11 new strings were safely installed in December. This is more strings installed in a one-month period than in any previous installation year. The equipment, cables, Digital Optical Modules (DOMs), and personnel required for a successful season continue to arrive at South Pole as needed. IceCube researchers at South Pole interacted with hundreds of students in the northern hemisphere via conference calls and blogs. Increased efficiency in drilling will result in an approximately 20% savings in fuel usage (25k gallon savings) and costs. Technology Review featured the paper, Atmospheric Variations as Observed by IceCube. "A neutrino telescope buried beneath the ice at the South Pole is giving researchers a unique glimpse of temperature changes in the ozone layer." 1
Cost and Schedule Performance The project is 94.3% complete. Remaining contingency is $6.8 million. There has been no change to the NSF MREFC funding requirements of $242.1 million since the project performance baseline was established in February 2004. 2009/10 String Installation Status and Plans December 2009 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 String# 08 09 16 25 85 84 82 Total# 60 61 62 63 64 65 66 December 2009 January 2010 23 24 25 26 27 28 29 30 31 01 02 03 04 05 06 07 08 String# 81 86 35 34 24 15 23 33 Total# 67 68 69 70 71 72 73 74 January 2010 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 String# 43 42 51 32 41 Total# 75 The schedule variance at the end of November 2009 was -$82.9k (compared to $13k last month). The cost variance at the end of November 2009 was $2,071k (compared to $1,889k last month). This favorable variance is principally due to lower labor and on-ice support costs for Raytheon Polar Services Corporation and the ANG in FY2009. In 2
addition, training costs for drilling were less than budgeted (Implementation) and there are still several lagging invoices for cables (Instrumentation). The baseline schedule for string installation is 18 strings in December 2009 January 2010 and 3 strings in the final season, December 2010 January 2011. A total of 19 strings were installed last season. UW and the IceCube collaboration requested NSF approval to complete the 80-string array originally proposed. The 80-string array can be completed within the planned contingency and no additional funding was requested. The six DeepCore strings were approved and five were installed this month. The resources are in place for the successful completion of the MREFC project upon completion of the final installation season in FY2011. Fuel consumption for drilling is running about 20% below the budget baseline. This is the result of more efficient drilling and drilling holes with shorter freeze-back times (an estimated 24 hours available to successfully install a string). If twenty strings are installed we anticipate savings relative to the support budget plan of about 25k gallons. UW-Madison and the IceCube collaboration submitted revised budgets for M&O for FY2011-2015. The revisions were made in response to guidance from the NSF and the IceCube International Oversight & Finance Group (IOFG). Included in the submittal were responses to the recommendations from the NSF panel that reviewed the original proposal. Contingency Status and Plans No change requests were implemented this month. No. Description Change Log - IceCube Total Project Budget Baseline ($K) Approval Date Total Baseline Allocated Budget Allocated Budget Change Contingency Budget Budgeted Cost of Work Remaining NA Status as of July 2009 275,503 267,908 0 7,596 19,036 CR154 ICL Rooftop Access 09/29/09 275,503 267,908 0 7,596 CR155 Data Storage Additions 09/30/09 275,503 268,371 464 7,132 CR156 Southern Univ. Distributed Computing 09/28/09 275,503 268,409 37 7,095 NA Status as of August 2009 275,503 268,409 0 7,095 18,541 CR157 CR159 NA Additional Data Acqusition Software Support Effort Remaining Surface-to- DOM Cable Assembly at Sea Con Status as of September 2009 CR134 Non-US Institutions - Deep Core Contribution NA NA Status as of October 2009 Status as of November 2009 10/29/09 275,503 268,516 107 6,988 11/03/09 275,503 268,657 141 6,847 275,503 268,657 0 6,847 17,837 11/25/09 279,472 272,626 3,969 6,847 279,472 272,626 0 6,847 16,577 279,472 272,626 0 6,847 15,426 3
Item Risk Assessment & Potential Contingency Adjustments Contingency potentially required for technical, cost and schedule risks associated with the approved scope of work. Risk assessments are made at WBS-Level 4 to determine the value of the risk exposure as a percent of the cost of work remaining. Pre-Operations costs for additional capacity to the data storage and network systems, both at the South Pole and in the North, and for extending software development efforts. Estimate ($K) $1,447 $2,100 RPSC Estimated FY09 Closeout. -$750 RPSC estimate of base cost to support the installation of six strings during the 2010/11 drilling season. The current RPSC baseline already includes support for 80 strings. The cost to retro IceCube equipment/materials from the South Pole at the end of the project in FY2012. This is a conservative estimate based on Rev. 8 of the RPSC budget. Potential cost to retain experienced key personnel in the final year to ensure the success of the last drilling and installation season. $1,340 $1,050 Potential exposure to higher fuel costs during the last season of drilling (FY2011). $450 $730 Total $6,367 Available Contingency as of Nov 30, 2009 $6,847 Drill Operation and Installation December was an outstanding month for drilling. Drilling began three days ahead of schedule, and by the end of the month eleven strings were safely installed. The graph below shows the drilling depth and deployment depth vs. time for the 10 th, 11 th, and 12 th strings installed this year. The turnover time between holes was reduced to the point where drilling the next hole began well before the previous string installation was complete. Drill depth for strings 35, 34, and 24 (holes 10-12 of the season). Red represents drilling depth and green represents string depth during deployment. 4
We are also pleased to report that fuel efficiency procedures and systems were implemented, resulting in a 20% savings of fuel as compared to what was budgeted. We are currently 15,224 gallons under budgeted fuel use for the season. South Pole Drilling Activities: Final Enhanced Hot Water Drill (EHWD) set-up completed in early December. New load cells were installed and calibrated in the towers. Software and motor control reconfigurations were completed on the Independent Firn Drill (IFD). The IFD drilled 13 firn holes in December (4 in Nov). The Enhanced Hot Water Trencher was assembled, tested, and used for cable trenching. Surplus and damaged equipment retrograde was started and continued through December. Generator repairs and troubleshooting are on-going. Replacement prototype heater (Sioux) was installed, testing completed, and brought online. EHWD User Manual (Turn-Over Package) documentation filming is on-going. South Pole Installation Activities: Deployment installed 11 Main Cables with Digital Optical Modules (DOM s) in December, including 5 deep core strings. DOM testing and staging continues. Off-Ice Drilling Activities: Cargo: Delivery of equipment, main cables, DOM s, etc was on target for December. The Northern Hemisphere Support Group (NHSG) received and completed 19 requests for urgent procurements/shipments to support South Pole. Detector Commissioning and Verification - Verification continues to run smoothly with nothing major to report. Calibration and Monitoring Work is ongoing in several areas. We are undergoing a comparison of Photonics data with the GEANT simulation code, looking at DOM occupancy on a single string for a 10 TeV cascade. No firm results yet. Some work on the Standard Candle 2 is also underway, comparing data and simulation. We are also exploring hardware communication issues with Standard Candle 2 as well. University of Wisconsin-Madison researchers are working on additional Single Photon Electron (SPE)-level runs in the dust layer. The issue is to determine which LED is pointing in a given direction, which is made more difficult by the ice in the dust layer. Finally, at University of California-Berkley, IceCube scientists are working on ice measurements from new flasher data. 5
Data Acquisition Hardware & Software Due to a delay in satellite transfer of Data Acquisition (DAQ) log files it is not possible to give a precise accounting of the DAQ up time fraction this month. Ignoring the days where reporting is missing the rest of December saw an up time fraction of 97%. DAQ activities during December were concentrated on testing the improvements in the DAQ data formatting. The DAQ release El Farolito, deployed to Pole on November 30, was running on version four data format as other software systems were preparing to move to the more efficient version five. At the end of the month, a successful test of version five was carried out and the data has since been verified to be good quality so the DAQ group is obtaining collaboration sign-off to switch the default mode to version five. Detector Up-Time: 98.1% IceCube (in-ice) clean runs Up-Time: 96.3% Unscheduled Downtime: 0.7% Definition of the terms: Detector Up-Time is the percentage of the time period for which the pdaq data acquisition was acquiring data and delivering at least 500Hz of event rate. This uptime measure therefore includes periods in which the detector was taking data with a partial detector enabled or with light contamination from calibration sources. Clean run Up-Time(s) is the percentage of the time period considered to have pristine data (standard hardware and software configurations) with the full nominal detector enabled, not contaminated with light from calibration sources and for which no serious alerts were generated by the monitoring, verification or other. The criteria applied are not algorithmic but rather represent the Run Coordinator s overall impression of the quality (including uniformity) of the runs/data. December data-taking was relatively smooth, with only the usual interruptions from short power outages and hubs failing. The detector uptime was 98.1% and the clean run uptime was 96.3%. Various special runs to test a new version on the data format (V5) from the DAQ (and that are decoded by the JEB/PnF) were taken. After a few iterations, the new data format was proved to work correctly. On Dec 24 th, the dust logger was operated 6
on a newly made hole, the light from the device blinded the DAQ and required a few hour run on testdaq. Data Handling - South Pole systems in the IceCube Laboratory began the 59-string physics operation on May 20 th and continued smoothly through December. Daily satellite transfer rates in megabytes per day. Filtered physics data dominates total bandwidth, shown here in red. Information technology work continued with South Pole systems upgrades to handle the additional strings and data rate from the 2009-2010 installation season. Major upgrades to the data warehouse and offline-processing infrastructure were completed in December. The upgrade included commissioning of the new NPX3 compute cluster, which brings significant new and much needed processing in the data center. Online & Offline Filtering, Software & Database - The online filtering system for the 59 string configuration run, which began in May, continues smoothly with daily satellite transmission of filtered data to the northern data warehouse at University of Wisconsin as logged in the figure above. Offline filtering activities include the full release of IceTray "v3", with continuing minor changes in new releases and continuing work on new a packaging system for release and distribution of IceTray software to users. We have been busy preparing for data from the 77-string detector configuration, with filter proposals due in December. To accommodate increased data from 77 strings, we have begun testing new data acquisition payload structure, intending to lower the data size per event. Additional testing on final PnF/Online software is ongoing. This will be 7
the final delivered major release of PnF system, with remaining work to be in documentation for long-term maintenance. A major milestone in production processing of data came in November when final offline processing of data from the 40-string was complete. Simulation - Steady progress was made on improved detector simulation. Studies continued on the systematic error for physics analysis due to the modeling of ice properties. Several new special simulation data sets using various ice properties for the 40-string configuration were produced to facilitate the studies. The graph below shows the total worldwide simulation production jobs for the month. Although there was a lull in the production during some IT work in the north on the central data center, the total average number of jobs was approximately 30% above that of the previous month. Education and Outreach With all the activity down at the Pole, there have been lots of opportunities for education and outreach. In December, IceCube was involved in webcasts with classrooms, on-site South Pole tours, and more. Outreach and Education highlights: California High School teacher Casey O Hara, as part of the PolarTREC program, conducted a phone conference with several classrooms. Students were able to ask questions to Casey, who also provided a slideshow about IceCube, IceTop, and working at the South Pole. Links to his phone call and a pdf of his slides are here: http://www.polartrec.com/archives/11196. In a call arranged by the University of Delaware, researchers connected with 500 students of ages in a Phone Call from the Deep Freeze. Students submitted questions in advance, and archives of the questions and answers can be found at: http://www.udel.edu/udaily/2010/dec/antarctic121709.html. 8
UW researcher Mark Krasberg and graduate student Laura Gladstone participated in a webcast hosted by the Exploratorium. Krasberg and Gladstone were part of a panel of South Pole researchers asked to talk about their projects in conjunction with the Fall 2009 American Geophysical Union Meeting. The webcast is at: http://icestories.exploratorium.edu/disp atches/index.php/. Graduate student Laura Gladstone at the South Pole Undergraduate students from UW performing DOM tests. River Falls continue to blog from the icebreaker Oden on its way to McMurdo, where they are conducting research on an IceTop tank. Blog entries from the students can be read at: http://www.uwrf.edu/icecube/icebreaker.htm. In late December, Prof. Jim Madsen of UW River Falls was interviewed on Dr. Kiki's Science Hour, a video podcast from UC Davis. Dr. Madsen answered questions about neutrinos, in general, and IceCube for nearly the entire hour of the show. A copy of the interview can be heard at: http://twit.cachefly.net/odtv/1223-dksh28.mp4. This month IceCube hosted several visitors and an open house at the Pole. Five members of the National Science Board and senior members of the National Science Foundation visited the project site. The visitors were given a tour of the drill camp, tower operations site, and B2 where the live event display can be seen. During the summer season, IceCube hosts many distinguished visitor tours. This was the third such tour this season. On December 13th, IceCube held an Open House attended by 25 people that included a tour of the Seasonal Equipment Site. Open h ouse a ttendees a re s hown a c ompleted s tring. Quality Assurance and Safety This month IceCube took steps to increase safety measures and to discuss potential safety issues within the project and with Raytheon Polar Services Corporation support staff that work on IceCube. These actions were prompted by some minor incidents that had the potential for safety implications. No injuries resulted from any of the incidents and the experiences provide the basis for continuous improvement. 9