CCV Newsletter 2009-1/2

Welcome

Welcome to the quarterly newsletter from the Center for Computation and Visualization (CCV). This has been a long time coming. As you will see shortly, this long silence is not based on a lack of activity. To the contrary, we have been very (VERY) busy developing new services for the community as well as preparing for some major changes, all of which will happen during the next few months.

While the changes are major and will impact the way CCV operates, it also marks an expansion of services. The transformation will be fast and we anticipate problems along the way. Your feedback in this process is essential, and I encourage you to contact me with feedback and constructive criticism so we can continue to improve our services and support for research and teaching across the institution.

Again, I encourage you to contact me with examples that display work, which directly or indirectly, relies on the CCV facilities and staff expertise. The CCV Newsletter is an outlet for this, and I will make every effort to highlight as many of these activities to the Brown community as possible.

CCV moving to 94 Waterman Street

Unless history repeats itself, it should now be for certain – CCV is moving to a new location. After having been located in rented space at 382 Brook Street for several years, the Center staff will relocate over the summer to an improved facility at 94 Waterman Street. The staff will be located on the 2^nd floor of this beautiful old building while the basement will function as development space.

The move should be completed by the end of July 2009 and we look forward to welcoming you to our new space at the beginning of the fall semester.

For further information, please contact Sharon King (Sharon_King@brown.edu).

Major changes in the CCV machine room

As many of you are aware, the majority of the hardware in the CCV machine room is rapidly aging and quickly becoming obsolete. This is naturally limiting the level of research than can be accomplished.

 It is therefore a tremendous pleasure for me to announce that a new and substantial shared computing platform will be installed over the summer, replacing a significant part of the aging installations and providing a capacity not previously available on campus. This platform is brought to campus after a long and productive collaboration between Brown and IBM and aims to provide a stable and homogeneous shared computing and storage platform to the whole community. It also provides a nucleus for a substantial future growth through a condominium ownership model (see below).

The nucleus of the new system is a 166-node Linux cluster purchased from IBM. Each node has two Intel Xeon 5540 (2.53 GHz) quad-core Nehalem processors, 24 Gigabytes of DDR-3 memory (1333 GHz) and a 40 GHz Quad-Data-Rate (QDR) Infiniband messaging interface.  The nodes are diskless, with I/O provided by the IBM GPFS parallel filesystem accessed through the Infiniband interface. The dual-processor compute nodes are packaged in IBM’s iDataPlex system, which is designed for high reliability and reduced power consumption.

In addition to the dual-processor Nehalem nodes, there are 7 quad-processor AMD Opteron nodes with Shanghai-series processors clocked at 2.6 GHz. Six of these nodes have 64 Gigabytes of memory; the seventh has 128 Gigabytes. All have QDR Infiniband interfaces.

Two dual-processor management nodes provide network provisioning, monitoring and other system management utility. Two login nodes provide access to the cluster for application development, debugging and job submission.

Access to the GPFS filesystem is provided by a 12-node service cluster and associated hardware RAID disk storage subsystems totaling roughly 390 Terabytes (30 Terabytes of SAS and 360 Terabytes of SATA disks), and with aggregate performance of more than 10 Gigabytes/s. Each RAID subsystem is connected to two of the service nodes in order to allow configuration for fail-over redundancy as supported in GPFS. The GPFS system is integrated with a Tivoli Storage Manager (TSM) 2 Petabyte tape archival and backup system. Policy-based Hierarchical Storage Management (HSM) facilities control automatic data migration among the SAS, SATA and tape storage pools. In addition, GPFS supports snapshot filesystem imaging to allow easy retrieval of accidentally deleted files.

The compute nodes access GPFS directly via their Infiniband interfaces. Users can also access the filesystem from other campus systems via the CIFS fileservice and through the Web via WebDAV. Each of the service nodes has both QDR Infiniband and 10-Gigabit Ethernet interfaces, and a 10-Gigabit Ethernet switch connects them to the Brown campus networking through a 10-Gigabit Ethernet backbone connection.

The compute and GPFS cluster both run the CentOS build of the RedHat Enterprise Linux (RHEL) operating system, and are provisioned with the xCAT open source cluster management software. Workload management is performed using the Moab Cluster Suite, a commercially supported system for managing queuing and scheduling systems and for user job submission.

Both the computational and disk storage clusters can easily be expanded or “scaled out” with additional nodes.

The system will be operated with management policies intended to provide a high degree of availability (i.e., with little downtime). All non-emergency system maintenance will be undertaken during a regular monthly window (typically 4 hours).

For questions and requests for further information, please contact Associate Director Sam Fulcomer (Samuel_Fulcomer@brown.edu).

Introduction of the Condominium model in CCV

The new IBM system is intended as the nucleus of a new “condominium model” of high-performance computing infrastructure at Brown. Under this model, CCV will continue to provide research account access at the rates listed below; however, researchers who obtain computer equipment funding will be able to “buy into” the system by providing funding to enlarge it. They will then be guaranteed high priority access to that portion of the system without incurring additional account maintenance fees. In addition, they will have access at an increased level of priority to the entirety of the shared system. When resources are not utilized they must be shared with other users, which is the only “cost” to contributors.

Offers of existing hardware for integration with the core shared system will be considered on a case-by-case basis (primarily with a view to the technical feasibility of integration). For future purchases of hardware, coordination with CCV staff will be required to ensure proper integration and utilization of the shared resources.

The condominium model will apply to both computing and storage resources and the terms of the agreement will be outlined in a contract between CCV and the faculty or faculty group.

For questions and requests for further information, please contact Director Jan S Hesthaven (Jan.Hesthaven@brown.edu).

Digital Dropbox now available for sharing of large files

CCV is pleased to announce a new service to the community – a digital dropbox for sharing of large files. Through a simple Web interface, users can upload files and obtain hyperlinks for these files. These hyperlinks can then be mailed to collaborators who can download the files by simply activating the hyperlink. The dropbox also allows outside collaborators to upload large files through a link provided by Brown collaborators.

This system was created to allow sharing of files that are too large to send via e-mail. Files are kept on the system for no longer than 30 days, and then deleted. Quotas are currently set to 10GB per user and files are limited to 2GB each.

Check out https://dropbox.brown.edu/ to access the dropbox. A brief userguide is also available there.

This service is free and has been sponsored by the Office of the Vice President for Research, the Department of Ecology and Evolutionary Biology through Professor Beth Brainerd, and CCV.

For questions and remarks on the digital dropbox, please contact Nathan Wood (Nathan_Wood@Brown.edu).

CCV begins to offer virtual Linux server hosting

Have you ever had to set up a large database or some other server, but found it difficult to find the time to understand how to do this in a robust and flexible manner?

CCV now offers a service where we set up and run virtual servers based on Linux. We will help to install and configure software needs, establish access protocols, and discuss security concerns. We will particularly focus on SQL databases.

The advantage of the virtual server model is that everything runs on CCV hardware and you can focus on using the data as needed. There is a small fee associated with this service but there is no direct hardware cost unless your project has substantial and specialized needs.

Additional services such as full/replicated backup can be added to the basic configuration as required by your research.

For questions and remarks on the digital dropbox, please contact Nathan Wood (Nathan_Wood@Brown.edu).

Summer transition and retirement plans for CCV clusters

As we prepare for the arrival of our new cluster and storage hardware this summer, the "ted" and "lou" clusters will be retired.  The “troy” cluster will be available starting mid June for job submissions using PBS.  At that point the “troy” frontend will be running on a former compute node with upgraded system software. 

During the transition, idle compute nodes will be re-assigned to the new frontend, and ultimately the old frontend will be reinstalled as a compute node.  We will announce when the new frontend is available, possibly with a limited number of compute nodes for user trials. Once the new “troy” cluster is back on line, “ted” and “lou” will be retired permanently.

During the trial period there might be a need to reboot or rearrange things on short notice, but we hope to make the transition smoothly.  Please let us know if it turns out we have overlooked any important software packages formerly on “ted” or  “lou” or “troy” and they don't show up on the new troy.

For comments and suggestions, please contact Dave Johnson (David_Johnson@Brown.edu).

Phasing out of legacy cluster management services

With the addition of the new IBM core computing facility, CCV is announcing a sunset period for provision of cluster management services. CCV will continue to manage clusters for private use at the prevailing rate of $10,000/yr.; however, discussion with current cluster owners will be undertaken to identify a reasonable limit to the term of operation for each cluster, and management of new clusters will generally not be considered unless there are very special circumstances.

For questions and requests for further information, please contact Associate Director Sam Fulcomer (Samuel_Fulcomer@brown.edu).

Best wishes for a productive and enjoyable summer – fall will bring a number of exciting changes to Brown and CCV so stay tuned.

                                                                                    Jan S Hesthaven

                                                                                    Director of CCV