An Open Letter to the HPC Community

We’re releasing PBS Professional under an open source license… why?

More than 20 years ago, I was just starting my career in HPC at NASA, where, along with a great team, we developed the original PBS software. Today, PBS Professional is among the most-used technologies for HPC job scheduling. But scheduling is hard — every site has unique processes, unique goals, and unique business requirements. In fact, there are at least a dozen different software packages that are in use across the globe for HPC job scheduling, in traditional HPC centers and in both private and public clouds. A dozen? Why?

Looking back over the last couple of decades, I believe a big reason is that the HPC world is really two worlds:

• The public sector – researchers and scientists from universities and national labs are breaking new ground and pushing the limits of HPC – natural collaborators, early adopters and risk takers, which leads to a strong preference for open source software, and
• The private sector – engineers and businesspeople from corporations are building products and delivering services leveraging HPC as a tool for competitive advantage – natural competitors, later adopters and risk-averse, which leads to a strong preference, often a requirement, for commercial software.

Because the community is split, innovations do not easily flow from one sector to the other, and rather than advancing the state of the art, efforts are wasted re-implementing the same old capabilities again and again. This is a HUGE missed opportunity.

Altair is making a big investment toward uniting the whole HPC community to accelerate the state of the art (and the state of actual production operations) for HPC scheduling. We are dual-licensing PBS Professional®, both open source and commercial. We are opening the full core of the software, not just a weak subset or older version. We are continuing to offer a hardened, commercial version for commercial customers. We are committing to being aggressively open and community oriented. We are adding staff and reorganizing our engineering organization to behave as one of (hopefully) many contributors to the project. We are using community-accepted best practices (e.g., Github, JIRA, open discussions for new features). We are joining the OpenHPC project with PBS Pro. And, we are focused on longevity – creating a viable, sustainable community to focus on job scheduling software that can truly bridge the gap in the HPC world.

Excited? Honestly, this is one of the most exciting activities I’ve been involved in since bringing PBS Pro out of NASA in the first place. Getting the software in shape for release has required a lot of effort. We are on track for releasing open source PBS Pro 14.0 in mid-2016. It’s not only opening the source code, but also creating the tools and processes for nurturing a community – documentation, ticketing system, continuous integration tools, wikis, email lists, etc. We want to “do it right”. Of course, once we go live, I’m sure we’ll learn a great deal (from people like you), and we will work hard to best support the needs of the community.

Obviously, we cannot do this alone!

We want your input, your help; please join us as part of the PBS Pro community.

(This post originally appeared on InsideHPC on 3 May 2016.)