After 5:00 PM on Friday, October 30, 2026, RCAC Staff will not be able to retrieve any data left in the Bell scratch filesystem.

RCAC recommends Gautschi as the successor platform for communities currently using or considering Bell. Gautschi is in production and was introduced as RCAC’s most powerful community cluster to date, with CPU and AI partitions designed to support both traditional simulation workloads and newer AI-oriented research workflows.

For faculty comparing the two platforms, Gautschi provides more cores per CPU node, more standard memory per node, higher large-memory capacity, newer GPU options, and faster NDR InfiniBand connectivity. This makes Gautschi the natural path for Bell users who need continued access to Purdue’s community cluster model after Bell is retired.

Please feel free to contact us at rcac-help@purdue.edu if you have any questions or need support.

What's Changing

The following changes will take effect on Monday, November 2, 2026.

1. Proportional GPU Limits: The per-account limit of 24 GPUs is being reduced to double the number of GPUs purchased or 2 GPUs, whichever is larger.
2. Reduced Max Walltimes: The current two-week max walltime for normal QOS jobs is being reduced to 4 days.
3. New burst QOS: A new quality of service will be created called burst with the following policies:
   1. Jobs using the burst QOS can use up to 8 GPUs at once for up to 8 hours at a time.
   2. Jobs with this QOS will have the highest priority on the cluster for more immediate start time, however they will only be able to preempt preemptible jobs. Normal QOS jobs will remain unaffected.
   3. Only one job from each account on the cluster can use this QOS at a time.
   4. Jobs using this QOS will be charged at 2x the rate for the GPU hours consumed by their job.
4. New Preemptible Features:
   1. Preemptible jobs will have a GraceTime value of 10 minutes which will allow preemptible jobs a guaranteed 10 minutes to clean-up and checkpoint their work before being interrupted.

Why are New Limits Needed?

As of October 1st, Gautschi has been in production for 19 months, and over that time, we have seen utilization continue to ramp up. Due to Gautschi's allocation method of allowing groups to purchase resources based on GPU hours rather than limiting them to the number of GPUs purchased, groups have been able to use many more GPUs in parallel than purchased. This is by design as we'd like for groups to be able to use the massively parallel capabilities of Gautschi, but while an overly generous limit of 24 H100s may have been suitable for a fledgling cluster with few users, it's become clear that it's impacting fair access to resources and consequently wait times for current users.

Below, we'll provide some of the data that are informing these limit changes as well as provide some discussion on the expected outcomes of each change.

Enforcing Proportional GPU Allocation

Over the past nine months, we've watched the total utilization of the AI partition creep towards 100% while simultaneously watching the percentage of jobs using the "preemptible" QOS fall. This indicates that we are at 100% utilization while there are always "normal" QOS jobs that are waiting to start which has resulted in longer and longer wait times for "normal" QOS jobs and rendered the "preemptible" QOS obsolete due to essentially never having idle cycles.

In large part, the reason for this is because we have allowed all groups on Gautschi to use up to 24 GPUs at once regardless of the size of their purchase into Gautschi. This is distinctly different from how our other community clusters operate. On those clusters, the amount of resources purchased by a group directly affects the priority of their jobs on that cluster through the use of Slurm's "Fairshare" feature by increasing the group's respective "share" of the cluster. For reasons explained in this blog, Gautschi cannot use "Fairshare". This presents a challenge for groups who want to capacity plan for their usage on Gautschi.

In order to address this, we'll be reducing the maximum number of GPUs that can be used by a group under the normal QOS from a flat 24 GPUs per group to a limit proportional to the size of a group's purchase. By scaling proportionally, this will allow us to schedule more fairly while guaranteeing capacity to those who choose to invest in more resources within Gautschi. To enable proportional allocation, we will set each group's limit of concurrent H100s under the "normal" QOS to be twice the number of GPUs purchased or two GPUs, whichever is larger.

When reviewing the average usage of all 97 groups on Gautschi since the start of the semester, 78.3% of groups would not notice this limit, and the new configuration would ensure larger availability of resources for all users, including the current largest consumers of Gautschi.

Why was "2x" the purchase chosen as the scaling factor?

Initially, we thought to simply reduce the number of GPUs each group can use from 24 to 16, but upon looking at the data, we saw that this would not practically change anything. We next considered allowing groups to use up to 4x the number of GPUs they purchased, and again we saw that this would only increase the availability of GPUs to serve normal jobs by a little more than 5%. Finally, when looking at a factor of 2x we saw that this would greatly improve the availability of GPUs to "preemptible" jobs and still allow groups who need all eight GPUs on a Gautschi node the ability to be able to purchase more GPUs and expect that there be idle GPUs for them to run on.

How to request larger quantities of GPUs?

We're introducing a new QOS called burst that will, in exchange for using twice the GPU hours as a normal QOS job, allow groups to use up to eight GPUs for a maximum walltime of eight hours. This will allow users to leverage the fast GPU-to-GPU connection provided in 8-GPU nodes. These jobs will have the highest priority of any job on the cluster. While a burst job cannot preempt a normal QOS job, it will always start before a normal QOS job. Because of the higher resource limits and priority, each group will only be allowed to have one burst job running and one accruing priority at a time. If you expect to need large numbers of GPUs frequently, consider purchasing enough to cover up to half of your regular GPU usage.

We hope that in addition to meeting the resource needs of groups who only need whole node jobs occasionally that this will also be a tool for groups who are willing to pay for more "interactive" access as this has been a request from many PIs.

Reducing Max Walltime per Job

Another limit that we are reducing in this tuning pass is the maximum walltime for each job under the normal QOS. There are a couple of reasons for this, but it essentially boils down to the fact that, empirically, users are requesting many, many more hours/days than their jobs require, and this makes it harder for Slurm to give estimates on start time to users.

Reducing the maximum walltime should allow Slurm to be more efficient in backfilling users' jobs. For a more in-depth discussion as to why that is the case, see the discussion on backfill scheduling in this blog. We recommend that users continue to implement and leverage checkpointing as a best practice to make efficient use of shorter walltimes and preemptible jobs.

New Preemptible Feature for Enabling Checkpointing

As discussed above, in recent months the preemptible QOS has been rendered unusable by the fact that there is always an eligible "normal" QOS job ready to preempt any preemptible job that starts. We expect under these new limits on Gautschi, the resources available to run preemptible jobs without interruption will increase considerably, and we encourage groups, particularly those with multi-GPU needs, to take advantage of these resources by submitting jobs under this quality of service.

To help make this transition smoother for groups that haven't used the preemptible QOS before (and for those that have), we will be adding a GraceTime of 10 minutes to preemptible jobs. What this means is that once your job has been selected to be interrupted by a higher priority job, it will be "told" through a Unix signal, that it has 10 minutes to gracefully shut down. We hope that this will provide an easy mechanism for users to checkpoint their work.

Faculty, staff, and students are invited to present or attend!

Date: October 23, 2026 @ 10am EST

Speakers:

Nikhila T. Suresh, PhD – computational cancer genomics and bioinformatics scientist
"Genome Wide Methylation Profiling of Testicular Germ Cell Tumors"
This seminar will explore computational and multi-omics approaches to identifying methylation patterns, gene signatures, and potential biomarkers associated with chemotherapy resistance. The research aims to improve understanding of tumor biology and support the development of new approaches for detecting challenging disease subtypes, including post-chemotherapy teratoma.

Paridha Talmale - Engineering Management graduate from Purdue University
"Inside Manufacturing Engineering - Turning Data, Process, and Constraints into Decisions"<b/r> A look inside manufacturing engineering and how engineers turn observations, data, and real-world constraints into meaningful solutions. Explore approaches to improving production capacity, material flow, and data-driven operations while learning how communication, practical decision-making, and continuous learning contribute to engineering success. Whether you're exploring engineering or looking to broaden your technical perspective, this talk offers valuable insights into solving problems and creating impact in manufacturing.

To attend, please register here: REGISTRATION

What’s changing

• Starting July 20 when you connect to datadepot.rcac.purdue.edu, you will be using our newer Data Depot system (Depotv3) instead of the older one (Depotv2).

• The older system will still be available behind the scenes during this transition.

• This is mostly a technical change. It updates how accounts are authenticated and brings Data Depot in line with other RCAC services that already use our newer login system. The goal is to keep Data Depot reliable and consistent with our current environment.

What you need to do

• If you normally use Data Depot by connecting to datadepot.rcac.purdue.edu (for example, a mapped drive like “depot\mylab” on Windows or smb://datadepot.rcac.purdue.edu/depot/mylab on macOS), you should not need to change anything.

• Your existing shortcuts, mapped drives, and scripts that point to datadepot.rcac.purdue.edu should continue to work after the change.

• If you maintain service accounts or automated jobs that use Data Depot, please check them after the change to ensure continued functionality.

• Saved credentials may cause problems with your connectivity to the Data Depot.  If you believe this to be the case, simply remove the saved credentials and re-connect.

If something goes wrong

• If we see problems after the change, RCAC can quickly switch Data Depot back to the older system.

• If you notice issues accessing Data Depot or your mapped drives after this update, please contact us at rcac-help@purdue.edu.

Following good data practices makes scratch more performant and more useful for all users. By keeping master copies of important data in backed‑up storage, staging only the inputs and outputs you need for active jobs into scratch, and promptly moving valuable results back to a persistent location when your work completes, you help keep scratch fast, responsive, and available for high‑throughput workloads across the system.

To support this, we’ve updated our scratch documentation with clearer guidance and policies, including:

• An emphasis on scratch as temporary, high‑performance workspace, not long‑term storage.
• Clearer explanation of purge behavior and age limits (including that files may be removed at any time after they become eligible).
• Removal of legacy “purge warning email” language; users should no longer expect email notifications before files are purged.
• Explicit guidance that using tools like touch or other methods to defeat purge mechanisms is a violation of Acceptable Research Resource Use.
• Practical tips and examples for staging data into scratch, monitoring file ages with tools such as purgelist , and moving important results back to Data Depot or Fortress.

Scratch filesystems are engineered for capacity and performance and are not protected by backup technology; some types of failures can result in permanent data loss. If losing a file in scratch would significantly impact your research, that file should have a current copy in a more durable storage location such as Data Depot (for active data) or Fortress (for long‑term archival).

You can review the updated scratch storage guidance here: https://www.rcac.purdue.edu/policies/scratchpurge

If you have questions about how to adapt your workflows or where to store particular data, please contact us at rcac-help@purdue.edu and we’ll be happy to help.

**Issue: **Some users may not see Microsoft Authenticator push prompts unless they manually open the app. This can cause sign-ins to time out.

Workaround: If you do not see a push notification, open the Microsoft Authenticator app and approve the pending request. Also, please verify that your default sign-in method is set to ‘App based Authentication – notification’ You can follow the steps in this Knowledge Base article to check and update your settings: https://service.purdue.edu/TDClient/32/Purdue/KB/ArticleDet?ID=1788

Still Need Help? - Contact us at rcac-help@purdue.edu

Beginning May 11, 2026, RCAC will move from Duo Mobile to Microsoft Authenticator for logging in to Purdue’s research computing and supercomputing systems.

To ensure uninterrupted access, please complete the steps below before May 11.

What you need to do:

Enroll in Microsoft Multi-Factor Authentication (MFA) using the Microsoft Authenticator app (if you have not already)

Set your default sign-in method in Microsoft MFA to your preferred method

After May 11, Duo Mobile will no longer work for accessing RCAC systems.

Please contact rcac-help@purdue.edu If you have any questions or need help.

A full catalog of available modules with usage instructions is at the Biocontainers Documentation site. For step-by-step guides on running common genomics workflows on HPC systems, visit the RCAC Bioinformatics Tutorials site, which includes documentation for tools like HiFiasm, BRAKER3, Trinity, and Nextflow, along with best practices for job submission and resource optimization. Full workshop materials are also available for RNA-seq analysis, genome assembly, and genome annotation. For upcoming training sessions and community programs, including the biweekly Genomics Exchange discussion series and the Midwest Bioinformatics Showcase seminar series, see the Computational Biology Services page. For questions or support with bioinformatics workflows on Anvil, contact rcac-help@purdue.edu.

This will affect the following services:

• CMS Front-End (aka 'Login') nodes;
• EOS storage;
• XCache;
• CVMFS;
• Hammer cluster SLURM jobs;
• Analysis Facility will operate in reduced capacity:
  • fewer CPUs, RAM and GPUs will be available;
  • CVMFS and EOS mounts will not be available;
  • XCache will not be usable.

Given the scale of the event, we have scheduled a 'full downtime' for our computing resources.

Users will be able to do some interactive work in the AF and the Community Clusters (other than 'Hammer'), as home-directories and Depot storage are not expected to be affected (they are hosted in the MATH Data-Center), but are strongly advised to not depend on the reliability of any services during the power outage period.

The current plan is to have all machines powered down before the end of day (5pm) on Wed. March 4th, and then have them powered back up by start of business (9am) on Thu. March 5th.

Please - plan your use of the resources accordingly. CMS Support Team

During the Purdue AI Research Showcase, IPAI and RCAC will be hosting a poster session. We are calling on all Purdue researchers to submit your posters today!

Poster Session Callout:

Poster Topic: AI research and applications across Purdue—new models, systems, and interdisciplinary breakthroughs solving real‑world challenges. Submissions can include early-stage ideas, works in progress, group projects, or class assignments, not only completed research.

Poster presentation date: April 14-15, 2026; Exact Date and Time: TBD

Poster presentation location: Envision Center, Purdue University West Lafayette Campus

Participation eligibility: Posters authored or co-authored by undergraduate or graduate students, faculty, and staff from any Purdue campus

Awards: Best poster will receive travel expense coverage of up to $1,000 to a conference of their choice!

Parameters: Posters must be 30" wide by 40" long or smaller. Posters may be vertically or horizontally oriented, but the width must be less than 30". More information about wide printing at the Purdue West Lafayette campus can be found here: https://it.purdue.edu/facilities/instructionallabs/printing/wide_format_printing.php

Make sure you submit this *survey to ensure your spot in this event! Once we receive your submission you will receive a follow up email with additional details for the event. Submissions are due no later than Monday, April 13th.

* Registrations are now closed due maximum capacity.

We look forward to your participation and learning more about your research in AI, and don't forget to save the date for the Purdue AI Research Showcase!

During this maintenance window, several clusters will experience a temporary outage so that hardware can be safely powered down while facility work is performed:

• Gautschi, Gilbreth, Negishi, Bell, and Anvil cluster nodes will be powered down.

• The Gilbreth’s legacy V100 GPUs, that are well past their lifetime, will be decommissioned.

• Hammer (Math nodes) and Geddes: A subset of nodes will be powered down but the services will be available, unless communicated separately.

How does this maintenance impact you?

• Clusters listed in this message won’t be available to run jobs during the maintenance.

• Any jobs requesting a walltime which would take them past the start of the maintenance will not start and will remain in the queue until after the maintenance is completed.

• Users can continue to access their data.

• GenAI studio will remain available. This maintenance will position Purdue to support growing computational needs. Users should see long‑term benefits in system reliability and our ability to support future computing and AI resources.

If you have questions about how this outage will affect your work or need support, please contact rcac-help@purdue.edu.

Research Computing staff members will monitor the status of all computing and data resources in an effort to ensure continuous availability.

Research Computing staff members will monitor the ticketing system throughout the holiday period and answer critical issues and problems. Non-critical user issues and questions will be addressed beginning January 5th, 2026. There will also be no coffee hour consultations during this break.

Scratch file purging (on community clusters with scratch space) will continue as normal during the break, so be sure to archive your files in scratch storage. This does not apply to Data Depot or home directories -- only scratch storage.

Have a wonderful break, everyone, and we look forward to great things in the new year!

• All jobs on the cluster will be required to explicitly specify a partition and an account (i.e. your group's name) at submission time. You can find the names of the available partitions and accounts from the showpartitions and slist commands respectively. Any job that does not specify an account and a partition will be rejected at submission time.
• All jobs on the cluster must include an explicit memory request through use of the --mem option. Jobs that do not make an explicit memory request will be rejected and a reasonable default will be suggested.
• Accounts will be renamed to no longer include a suffix designating the type of resource the account contains. This means that groups with multiple types of resources broken into multiple accounts will be consolidated into a single group.
• GPU type will be requested through specifying a partition containing the GPU type rather than submission to an account with a suffix designating that partition. i.e. jobs previously submitted to -A mylab-k will now be specified as -A mylab -p a100-40gb
• The output of slist and the default output of squeue will be modified to reflect the new scheduler design.
• All "shared accounts" such as standby that represent resources outside of your typical "group accounts" will continue to exist but will require a different request syntax.
  • Standby will become a Quality of Service (QoS) and jobs that previously ran under the "standby" account, will now be submitted to your "group account" and be tagged with the standby QoS. I.e. if your job previously used the -A standby option, you would now use -A mylab -q standby

How will this affect you:

1. You will need to change your jobscripts and your method of invocation to include the required options outlined above.
2. If you have any scripts or tooling that rely on the current output of slist or squeue, those scripts will need to be modified to use the new formatted output.

You can prepare for this maintenance by reviewing the new Slurm organization in our user guide's Queues Page.

If you have any questions about these upcoming changes, please reach out to us at rcac-help@purdue.edu.

Get ready for a 16-week hands-on training series focused on artificial intelligence, machine learning, and large language models (LLMs). Led by expert instructors, each weekly session builds practical skills for research, development, and deployment.

Weekly Topics & Dates:

Aug 29 – Intro to Python, Numpy, and Pandas (Ashish)
Sep 5 – Python Data Visualization (Ashish)
Sep 12 – Intro to R: Tidyverse, ggplot2, dplyr (Mihir)
Sep 19 – AI Day: Prompt Engineering, RAG, Agents, Finetuning (All)
Sep 26 – ML Fundamentals I: Supervised Learning (Haniye)
Oct 3 – ML Fundamentals II: Unsupervised Learning (Haniye)
Oct 10 – Intro to PyTorch & TensorFlow (Christina)
Oct 17 – Intro to NLP + Hugging Face Transformers (Mihir)
Oct 24 – Intro to LangChain for LLM Apps (Haniye)
Oct 31 – Real-Time LLM Apps with Streamlit & Gradio (TBD)
Nov 7 – Evaluating LLMs: Benchmarks & Metrics (TBD)
Nov 14 – Vector Databases 101: Weaviate, FAISS, Pinecone (Mihir)
Nov 18 – Using LLMs in Scientific Research (TBD)
Nov 22 – Building Research Chatbots (Non-RAG) (TBD)
Dec 5 – Ethics & Governance in Generative AI (Ashish)
Dec 12 – From Data to Deployment: MLOps for LLMs (TBD)

Bonus Module (Pre-recorded):

Intro to Purdue GenAI Studio UI & API – Hands-on demo and custom RAG training

Registration and connection details will be announced soon.

Stay tuned for updates and secure your spot in this exciting series!

• All jobs on the cluster will be required to explicitly specify a partition and an account (i.e. your group's name) at submission time. You can find the names of the available partitions and accounts from the showpartitions and slist commands respectively. Any job that does not specify an account and a partition will be rejected at submission time.
• The output of slist and the default output of squeue will be modified to reflect the new scheduler design.
• All "shared accounts" such as standby, highmem, etc. that represent resources outside of your typical "group accounts" will continue to exist but will require a different request syntax.
  • Standby will become a Quality of Service (QoS) and jobs that previously ran under the "standby" account, will now be submitted to your "group account" and be tagged with the standby QoS. I.e. if your job previously used the -A standby option, you would now use -A mylab -q standby
  • The highmem and gpu shared accounts will become partitions and jobs that previously ran under these accounts will now be submitted to your "group account" and should be submitted to the appropriate partition, i.e., -A highmem will become -A mylab -p highmem
  • Groups with access to the interactive account will now submit to their "group account" and the interactive partition. i.e. -A interactive will become -A mylab -p interactive

How will this affect you:

1. You will need to change your jobscripts and your method of invocation to include the required options outlined above.
2. If you have any scripts or tooling that rely on the current output of slist or squeue, those scripts will need to be modified to use the new formatted output.

You can prepare for this maintenance by reviewing the new Slurm organization in our user guide's Queues Page.

If you have any questions about these upcoming changes, please reach out to us at rcac-help@purdue.edu.

• All jobs on the cluster will be required to explicitly specify a partition and an account (i.e. your group's name) at submission time. You can find the names of the available partitions and accounts from the showpartitions and slist commands respectively. Any job that does not specify an account and a partition will be rejected at submission time.
• The output of slist and the default output of squeue will be modified to reflect the new scheduler design.
• All "shared accounts" such as standby, highmem, etc. that represent resources outside of your typical "group accounts" will continue to exist but will require a different request syntax.
  • Standby will become a Quality of Service (QoS) and jobs that previously ran under the "standby" account, will now be submitted to your "group account" and be tagged with the standby QoS. I.e. if your job previously used the -A standby option, you would now use -A mylab -q standby
  • The highmem and gpu shared accounts will become partitions and jobs that previously ran under these accounts will now be submitted to your "group account" and should be submitted to the appropriate partition, i.e., -A highmem will become -A mylab -p highmem

How will this affect you:

1. You will need to change your jobscripts and your method of invocation to include the required options outlined above.
2. If you have any scripts or tooling that rely on the current output of slist or squeue, those scripts will need to be modified to use the new formatted output.

You can prepare for this maintenance by reviewing the new Slurm organization in our user guide's Queues Page.

If you have any questions about these upcoming changes, please reach out to us at rcac-help@purdue.edu.

In order to aid in that transition, we have made a test environment available to users so they can begin re-building their applications under the new toolchains and exploring the new software stack. We expect that this environment will not change much ahead of the maintenance next week, but we may end up making small changes to the application stack before then.

This test environment can be accessed by using the command ssh gilbreth-fe05 from any Gilbreth's front end nodes. The Slurm queues you are used to should be present and will submit to test nodes. Wait times may be larger for these test nodes due to their only being a limited number of nodes available for testing.

If you encounter issues or have questions about this maintenance, please reach out to us at rcac-help@purdue.edu.

If you have any questions about the upcoming maintenance, please reach out to us at rcac-help@purdue.edu.

In order to aid in that transition, we have made a test environment available to users so they can begin rebuilding their applications under the new toolchains and exploring the new software stack. We expect that this environment will not change much ahead of the maintenance next week, but we may end up making small changes to the application stack before then.

This test environment can be accessed by using the combination Slurm options -A debug --reservation=rocky8test in your current job submission with sbatch or sinteractive from any Bell’s front end nodes. This will make sure to submit to test nodes. Wait times may be larger for these test nodes due to their only being a limited number of nodes available for testing. The limitation on the debug queue is: 1 concurrent running job and 4 job submissions for each user with a 30-minute’s max walltime.

If you encounter issues or have questions about this maintenance, please reach out to us at rcac-help@purdue.edu.