Univa Grid Engine Does Windows, Linux Containers 

Grid Engine, the popular workload management tool for distributed computing environments, now does Windows. And it is also doing Linux control groups, or cgroups, containers, a technology that was championed by Google many years back as the basis for its internal resource control on its massive clusters.

The support for Windows was made at the behest of some big customers who want to be able to run their distributed applications across collections of Windows-based workstations and servers. Grid Engine has supported various Unixes and Linuxes from day one, and it was possible to run Grid Engine inside of Microsoft's Services for Unix, a Unix-alike user-mode environment that runs on top of the Windows kernel and that makes it easier to port applications from Unix to Windows. While the SFU 3.5 extensions are still available from Microsoft as a download, general support for Windows SFU ended in 2011 and extended support for it runs out this year. Microsoft has also stepped back a little bit on a big direct push in the high performance computing market as well, and is more inclined to partner.

And so, explains Gary Tyreman, CEO at Univa, the company decided a while back to do a native Windows port of Grid Engine, and the decision was driven by a number of software vendors telling Univa that Windows was growing in importance, particularly in the oil and gas, electronics, and manufacturing industries where Unix and then Linux tended to dominate. One of the company's biggest customers – which Tyreman cannot name – also asked Univa to do a port to Windows, too. The idea was to do a true port of Grid Engine to Windows so it would not require any emulation software or compatibility libraries.

Grid Engine is written in C, not C++, and so the port to Windows was a bit easier than it might have been but it was still a pretty big task. Moreover, the Windows, Linux, and Unix kernels are all different enough that the expansion to Windows gave Univa a chance to go back into the Grid Engine code base and make some performance tweaks. "It was not an easy thing for us to do, but we have improved services and a lot of other things come along for the ride," explains Tyreman. "We went in and cleaned up some code in the protocol areas that we normally would not have changed because if it works, why touch it. These performance improvements and really core, tough fixes we had to take time for, and that's why it is a little later than we wanted it to be." Univa was shooting for early summer instead of late summer, but in the grand scheme of things, this is not much of a delay at all.

Pretty much if a machine doesn't run Windows Home Edition, it can run either the Grid Engine 8.2.0 client or be a server in a grid engine cluster to support workloads dispatched from Grid Engine. This includes Windows XP SP3, Windows Vista, Windows 7, and Windows 8 on desktops and Windows Server 2003, Windows Server 2008, and Windows Server 2012 on servers.

Among the other tweaks in the Grid Engine 8.2.0 release, Grid Engine can now do automated binding jobs to specific cores on a chip or sockets in a NUMA cluster of processors to ensure not only the best performance, but also that this performance is repeatable as the jobs come and go and come back again on the cluster. The update has a Postgres database for job spooling to run hundreds of thousands of small jobs concurrently in the Grid Engine scheduler and it also has a new feature called Qmaster, which creates a thread pool that sits between the client requests coming in to submit jobs and the scheduler and run queries against the scheduled work about twice as fast as was possible in the prior release. These queries eat up resources because people want to know the status of their jobs as conditions change out on the cluster. (A job scheduler is not so much like playing a game of Tetris with workloads as is it like trying to control the weather while predicting it.)

Grid Engine 8.2.0 also has support for Linux cgroups, a resource container that provides strict settings for CPU, memory, and processes running atop the Linux kernel. Now you can submit jobs to Grid Engine and it can fire them up in cgroups if that is how you want to do it on a Linux machine. Now you can take advantage of the restrictions on CPU and memory that the cgroup provides, thus allowing for applications to run side-by-side on the same physical platform without being selfish about resources.

The Grid Engine scheduler does not yet support Docker containers, which is a variant on the cgroup theme that is taking off like wildfire out there in hyperscale land, but support for Docker is on the Grid Engine roadmap, says Tyreman. The company is discussing internally how to do this now, and the odds are that it will treat the Docker machine much as it treats server virtualization hypervisors, meaning that the Docker machine will be created by its UniCloud tool, but the starting and running of applications will be managed by Grid Engine. The UniCloud support for the Docker machine could be available by the end of the year, with Grid Engine support for deploying into Docker containers coming perhaps by March 2015. That is plenty of time to catch the Docker wave.

Univa Grid Engine 8.2.0 is also able to schedule jobs on Cray's high-end XC30 massively parallel supercomputers and integrate with its Cray Linux Environment, which is a version of the SUSE Linux Enterprise Server tailored to Cray's "Aries" XC system interconnect. Adaptive Computing Moab, Altair PBS Pro, IBM Platform LSF, and the open source SLURM workload managers are already available for Cray systems, and once again, customers in the oil and gas industry where Grid Engine is popular have asked for Univa to bring its scheduler to a new platform.

The 8.2.0 release also includes initial support for the 2.0 specification of the Distributed Resource Management Application API, or DRMAA for short, a standard that is still evolving and that will codify a means of submitting jobs to a scheduler in a way that spans all schedulers. This standard is being developed through the Open Grid Forum. Version 1.0 of DRMAA gave a means of submitting jobs and then to do a basic query to check and see how a job is going, but according to Tyreman it did not allow you to run sessions and do some other things that are necessary in job schedulers. Software vendors that create code to run atop of Grid Engine want a standard way to submit jobs that spans various job schedulers. Univa has its own APIs for this, and will be releasing a RESTful API set probably in October of this year to expose those APIs in a programmatic fashion. The initial DRMAA 2.0 support will also come at this time.

Grid Engine has an annual license fee of $100 per core, with volume discounts for both long-term contracts and for increasing number of core counts. Tyreman says that most customers opt for a three or five years contract for their Grid Engine licenses, which gives them a better deal and which gives them certainty over the price over a longer term.