Folding@home and how we put our idle PCs to work

I work at a major research university here in the Midwest, and like many institutions, we started working from home mid-March when stay-at-home orders were issued in St. Louis. That shift left hundreds (if not thousands) of idle machines back on campus in labs, classrooms, and offices. It didn’t take long for folks to start thinking about putting those machines to use for the Folding@home project.

What is Folding@home?

I will let the Folding@home team describe the project themselves with this concise summary from their website:

Folding@home (FAH or F@h) is a distributed computing project for simulating protein dynamics, including the process of protein folding and the movements of proteins implicated in a variety of diseases. It brings together citizen scientists who volunteer to run simulations of protein dynamics on their personal computers. Insights from this data are helping scientists to better understand biology, and providing new opportunities for developing therapeutics.

Folding@home About page (https://foldingathome.org/about/)

The project originally came out of the Pande Lab at Stanford University, but after Dr. Vijay Pande moved to private industry and joined Andreesen Horowitz as a venture capitalist, management of the project moved to Washington University in St. Louis (spoiler alert, that’s the University in the Midwest I mentioned earlier) under Dr. Greg Bowman.

Ars Technica has a great article on the rising fortunes of the Folding@home project (The coronavirus pandemic turned Folding@Home into an exaFLOP supercomputer by Andy Patrizio, April 14, 2020). Check it out if you want to know more about where the project has been and where it is going with all their newfound attention.

Deploying Folding@home at WashU

Back to those idle machines in labs, classrooms, and office I mentioned earlier. With Folding@home now resident at the University, people wanted to put those machines to use running the client, and specifically running them as members of the University’s team (40211 if you’d like to join). I was asked to look at how we could deploy the client using Configuration Manger to our managed PCs and what settings to use.

I was familiar with the client from running it back in the day, and I had just recently reinstalled it along with everyone else who rediscovered the project. But I still needed to get up to speed since running it as a user on your own PC at home is different than deploying it en masse. The Folding@home support forum was invaluable, providing tips and tricks from others who have deployed the client as we were considering

For example, the client normally runs as a startup item in a standard installation, running when the user logs on and closing when they logoff. That wouldn’t work for us with idle machines without any users, however, so we elected to run the client as a service instead, giving us 7×24 processing. It turns out, however, that if you install the client silently (which you need to do for mass deployments), the setup does not populate the registry setting the client uses to find its data directory. Without that setting, the service will fail to start as a result. Thanks to the forum, I didn’t spend cycles trying to troubleshoot that. So, take it from me, the forum is your friend.

A Friendly Competition

Early on I suggested using department-specific usernames for the clients. Folding@home provides statistics pages that ranks teams, users, and users within teams. Rather than combine all our clients under one user, I thought it would be more interesting to have each department get credit for the work done by their computers. I wrote a PowerShell script that runs at install to identify the department from the computer name and generate the appropriate username to specify in the client’s configuration file. If you look at the WashU team pages on Folding@home and ExtremeOverclocking.com you can see the departments listed there (and see how we’re doing overall). Here’s a peek at the current stats for some of the department users. Please note that I’ve removed any individual team members (such as myself).

GPUs and running as a Service vs. Scheduled Task

Something else I learned from the forum was that the client can only use the GPU for processing if it runs as a user. For security reasons, Microsoft does not allow the client to access the GPU when it runs as a service. Running as a GPU is desirable because they can provide a 20-30x increase in speed compared to projects which use the CPU. Most of the PCs we were planning to use don’t have GPUs, so this limitation wasn’t a problem for us.

Our peers in the McKelvey School of Engineering, however, were in a different situation with labs full of GPU-equipped workstations. They figured out that if they run the client as a scheduled task, the client could use the GPUs without preventing people from logging on and using the workstation. They loaded the client in those labs using that method, and they have been the top producer at the University ever since, currently ranked 2,814 of all Folding@home users. (You can check out their Folding@home and ExtremeOverclocking team pages to see how they’re doing.) I’m not one to let a clever idea go unused, so I borrowed that approach (thanks Brian!) and incorporated it into my installer (more on that later). Even if we don’t need it now, we may eventually find more clients with GPUs (fingers crossed!).

Our Deployment

I handed off my instructions, settings, and script for naming to our packagers, and they created a wrapper that does the following:

1. Install the client silently (the version was 7.6.9 at the time, but the latest is now 7.6.13)
   fah-installer_7.6.9_x86.exe /S
2. Create a central data directory at C:\ProgramData\FAHClient
3. Copy a standard config.xml to the data directory where we specify these values which are the same across all clients.
   • cause – The preferred project type. (“COVID_19” for COVID-19 research)
   • team – Team number (40211 – feel free to join us!)
   • passkey – A unique identifier to tie your contributions to your user and which nets bonus points. (Get your own: https://foldingathome.org/support/faq/points/passkey/)
   • password – Password used for remote management of the client
   • allow – Network ranges to allow to connect and manage the client
4. Run the PowerShell script to generate the correct username and replace the placeholder value for user in config.xml.
5. Install the client service
   FAHClient.exe --install-service
6. Add the data directory path to the registry key HKLM\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\FAHClient\DataDirectory ="C:\ProgramData\FAHClient"
7. Adds a FAHClient program group to the system Start Menu with shortcuts specifying the data directory as the working directory.

The data directory and shortcuts are normally created by the installer, but we had to create them manually. When you deploy to the device using Configuration Manager, the installer runs as the System account, and neither the directory not the shortcuts are created under System. Our installer takes care of those as well as populate the empty registry key that I mentioned previously.

We deployed the client to a total of 332 systems. I am hopeful that we will get to push the client to more systems in other departments, but even if we stop here, that’s still a considerable number of systems processing work units every day. You can see the impact of our efforts along with other units at the University. On March 28, the WashU team was processing sixty-six work units per day and was ranked 9,301. Today we are processing 3,100 work units each day and our ranking is 370 and still climbing.

An Improved Installer

After we deployed these systems, I decided to work on an improved installer that I could share with others who are looking to do mass deployments of the Folding@home client. I used the PowerShell App Deployment Toolkit (PSADT) for the framework and made some refinements to make it less specific to our use case. The biggest changes from what I described above (aside from using PSADT) were in the creation of config.xml and the installation of the service/task.

I’m going to save the discussion of the new and improved installer for another post. I suspect the audience for that is less interested in the story and just wants to see the code. If after all this, you want to see more, hop over to Deploy Folding@home using PowerShell App Deployment Toolkit.

I hope you found this interesting and even useful. Please consider installing the Folding@home client on your own computers. If you want to mass deploy it on systems you own or manage, consider giving my PSADT installer a try.