This is a crosspost from Jonathan Dursi, R&D computing at scale. See the original post here.
This document is a whitepaper I wrote for the CASCA Computing and Data committee outlining the computing needs for the Canadian astronomy community for the coming several years. It does a fairly decent job of laying out the diverse range of large-scale R&D computing needs for the national community.
Advanced research computing resources have never been so essential to the Canadian Astronomy and Astrophysics research community. In the past few years, astronomical researchers have benefited greatly from modern large-scale computing systems; a diverse range of resources, which are a good match to the diverse computing needs of our scientists; and good working relationships with existing providers, allowing flexibility and collaboration between these centres and research groups.
However, CASCA has concerns about the near future of advanced research computing available to its researchers. Here the Computers, Data, and Networks Committee of CASCA present, on behalf of the Society, a summary of the current state of the computing needs, successes, and concerns of our researchers taken from previous consultative summaries and their updates. This is the first step of a process that will continue through the first half of 2013, which will include a comprehensive survey of research computing needs of the Canadian Astronomy and Astrophysics community, and will investigate a variety of strategies for meeting those needs.
Early systems funded by the CFI NPF are already showing their age; in many cases they are out of their maintenance contract and are already starting to fail. The lack of any clear signs of new investment on the horizon means that even if existing systems were to continue operating perfectly, as other nations continue to invest in new research computing platforms, our researchers, using stagnant computing hardware, will not only fall behind our international competitors as data volumes continue to increase, but also be unable to make full use of prior investments.
When new funding does become available, the Canadian astronomy community would like to see changes in emphasis taken as lessons learned from the CFI NPF procurement. Previous investment focused largely on computing hardware. While this addressed a real and pressing need resulting from years of underinvestment, the research endeavor requires a more holistic approach. Computing hardware investments must be balanced with similar investments in storage, highly qualified personnel, software development, and networking to maximize results.
In this report, we recommend an urgent search for new and sustainable sources of funding for advanced research computing funding; an increased focus on personnel, software development, and storage; maintaining a diverse range of systems; enabling major longer-term projects by committing resources for longer than the one-year allocation window currently of the RAC process; continuing to enable close working relationships with research groups and computing providers, preferably as close to the researchers as possible. In addition, we recommend that CCI’s board, through the proposed Researcher Advisory Committee or otherwise, establish a direct relationship with CASCA (and similar professional groups), with via persons charged with representing the needs of these research communities in planning for Compute Canada.