New website gives community cluster users a window to their work
A few clicks on a graph, a table and a couple pull-down menus in a Web form and Kevin Colby knows this particular piece of computational research used 800 processors on 100 different machines, how long it took — and a lot more.
A new website created by ITaP allows faculty and their labs not only to more easily track the status of jobs they’re running on Purdue’s community clusters, but also to examine in detail how they use supercomputers from a variety of perspectives.
Among other things, this should be useful in planning a lab’s workflow, planning for its future computational needs and potentially in refining algorithms researchers run on the clusters so the programs can run more efficiently and produce results faster.
Moreover, the website should be a good tool for generating data, charts and graphs covering a lab’s computational resources and use, which are often needed for grant proposals and reports to funding agencies.
“The key is that this website provides faculty access to data on their computational usage whenever they need it,” says John Campbell, associate vice president for academic technologies, who oversees the Rosen Center for Advanced Computing, ITaP’s research computing unit. “Over time, the site will be expanded based on feedback from the researchers.”
ITaP likewise will be able to use information generated by the system in planning for future community clusters and other research computing resources.
The new website is accessible through the Rosen Center’s site at www.rcac.purdue.edu. Click on the User Info menu and Usage Reporting.
The usage website opens with a series of graphs providing an overview of current activity on the community clusters, along with Purdue’s DiaGrid distributed computing system, which harnesses temporarily idle processors in office, student computer labs and elsewhere for research jobs.
From there, researchers can rapidly drill down by clicking through graphs, tables or web-based forms to look at work run by their labs in myriad ways — from a single job, to all the jobs by a member of a lab, to all the jobs run by a lab as a whole; which machines they ran on; the number of processors used; how long they took; and more. Search results can be viewed online or downloaded as standard comma-separated value files, ready to use in spreadsheet, database and other software packages with more extensive data visualizing capabilities.
While the website’s front page displays some overall views of activity on Purdue’s research computing systems, only faculty members and members of their labs whom they’ve given permission can view and work with a specific researcher’s or lab’s usage data.
Previously, Rosen Center staff could generate similar information for cluster users, but the process was time consuming and labor intensive, says Colby, an ITaP scientific applications analyst.
Since faculty and ITaP began partnering to build the community clusters, Purdue’s research computing use has grown exponentially. The Steele, Coates and Rossmann clusters built in 2008, 2009 and 2010 increased Purdue’s research computing capacity by more than 10 times.
A new cluster, called Hansen, is in the process of being built and is expected to be operational in September. Purdue faculty researchers wishing to see available hardware and to order capacity in the supercomputer can visit the Hansen cluster order website.
The three existing clusters had delivered nearly 300 million research computing hours to faculty and their students as of this summer. But the tools available for analyzing how research computing systems were being used dated from a time when that usage was measured in tens of thousands of hours rather than hundreds of millions.
At this point, the new website generates individualized information about community cluster usage, but the functionality will be extended to DiaGrid and other research computing systems in the future.
Writer: Greg Kline, science and technology writer, Information Technology at Purdue (ITaP), 765-494-8167 (office), 765-426-8545 (mobile), gkline@purdue.edu