The Big Red Cluster

Usage Policies

Accounts

Access to Big Red is provided to all IU faculty and their sponsored graduate students and staff. Instructional use is limited to courses which have been approved by the Director for Research and Academic Computing.

Passwords

Your account password on the Big Red Cluster is your IU Network ID password. Account passwords are intended for the owner of the account only and should not be shared with others. If a password is determined to be insecure, the account will be temporarily disabled. Passwords may be changed using the IU Password Maintenance page. Account passwords for the Big Red Cluster will not be given out via email or by telephone.

Home directories

Home directories reside on the RAC storage system and have a default quota of 10GB. If additional permanent storage space is required, users should apply for a Mass Store account.

Login and File Transfer Access

Login and file transfer access will be limited to the designated interactive nodes via Secure Shell Version 2 clients and commands (ssh, scp, sftp). Telnet and ftp are not supported for security reasons. To transfer files to and from the Mass Store, use pftp (Kerberized parallel ftp) and hsi (Hierarchical Storage Interface).

CPU Limits and Batch Jobs

On the Big Red login nodes (b519-b522), interactive and backgrounded jobs are limited to 20 minutes of CPU time. Interactive and backgrounded jobs exceeding this limit are automatically terminated with no warning. Jobs requiring more than 20 minutes of CPU must be submitted as batch jobs to the LoadLeveler batch queueing system via the llsubmit command.

There are also four "interactive" nodes: b509-b512. These are accessible from the Big Red login nodes. Processes run on these nodes are subject to a 24-hour CPU time limit, effective February 3, 2009. These nodes are intended for short-term, interactive debugging and test runs.

Computational Resources

Jobs run on Big Red should be submitted to one of three queues:

  • BIG
  • MED
  • FAST
Each queue has properties defined in order to maximize job throughput and minimize time spent waiting in queue. Jobs should be submitted to the queue most appropriate to the specific job requirements.

BIG Queue Properties

  • Nodes: 256
  • Maximum nodes per job: 256 (1024 cores)
  • Maximum walltime: 48 hours (2 days)
  • Maximum cpu time: 48 hours per core per job
Intermediate Queue Properties

  • Nodes: 248
  • Maximum nodes per job: 32
  • Maximum walltime: 336 hours (14 days)
  • Maximum cpu time: 336 hours (14 days)
Fast Queue Properties

  • Nodes: 4
  • Maximum nodes per job: 4
  • Maximum walltime: 2 hours
  • Maximum cpu time: 2 hours
  • Maximum jobs per user: 1
Default walltime in all queues is 2 hours. Request more if you need more. The default limit on running jobs is 16 per user. Please contact the administrators if you feel you need a higher limit.

Mail Usage

We do not provide a production mail service on the Big Red Cluster, however LoadLeveler communicates via e-mail. Mail forwarding should be configured during account creation, but if you are receiving mail on the Big Red Cluster, you may need to set up a .forward file for your account; see this Knowledge Base Document for more information on setting a .forward file.

Scheduled Downtime

Big Red maintenance windows will be scheduled for the first Tuesday of each month. A window will only be used if necessary; an announcement regarding the status of maintenance will be posted the preceding Friday morning in both the Big Red Message of the Day and on the Scheduled Downtime page.

Software Requests

Users wishing to use public domain or commercial packages may request the installation on publicly accessible disk space by filling out the Request for Software Installation on Research Systems.

Questions and Comments

The policies described in this document were established with the goal of providing a stable, powerful and efficient research computing environment for IU faculty, graduate students, and staff. If you have any questions or comments about these policies, we welcome your input. Please contact the High Performance Systems group, hps-admin@iu.edu.