Redmond (WA) – As much as Microsoft hyped its Vista operating system, its success is rather underwhelming so far. But the company is developing a new operating from scratch – a software that takes a new approach in the battle against viruses and will be able to run on many-core processors and take advantage of GPGPUs. A first version of the code was made available today .
Singularity was first announced more than two years ago as a project to rethink the way operating systems should work. Now almost five years in development, tech-savvy users can take a first look at Singularity and Microsoft’s ideas how a Windows of the future could look like.
The operating system isn’t ready to run on mainstream PCs by any stretch. It is provided as a basic framework and research project to developers. Microsoft said that it hopes to collect feedback and ideas from developers to shape this operating system, which at least at this time is not intended to replace Windows on client PCs in the foreseeable future. As bare as the software is (61.0 MB for the code) at the moment, the ideas behind Singularity is revolutionary and if the company can translate this concept into a mass-market product, there is suddenly an exciting operating system in our future again.
According to Microsoft, the basic idea behind Singularity is to create a more dependable OS and, in doing so, leave traditional operating system strategies behind and create a completely new architecture. The key developers of Singularity, Galen Hunt and Jim Larus, said that all current operating systems such as Windows, Linux and MacOS can be traced back to an operating system called Multics, which has its origins in the 1960s. In effect, current operating systems are still based, in part, on thoughts and criteria from 40 years ago, Hunt claims.
Of course, the challenges the of computing environment are very different today, which led to the idea to build Singularity from scratch, with modern technologies based on current challenges. “Our goal was to make Singularity small enough, simple enough, and well-designed enough that you can try out radical new ideas quickly,” Hunt says. “Our thinking is that a researcher with a great new idea for improving operating systems could get from idea to published paper in under a year using Singularity.”
So, while Windows Vista is an operating system that has evolved over decades, Singularity is entirely new. It is based on a so-called micro-kernel, which performs the crucial role of dividing systems resources among competing programs and abstracting the complexities of system hardware. According to Microsoft, more than 90% of the OS kernel is written in Sing#, an extension of the C# high-level programming language. The developers claim that this fact alone hands Singularity security advantages over operating systems written in lower-level languages such as C or C++. For example, by using C#, the researchers say they can prevent a class of errors known as buffer overruns, thereby eliminating a key area of vulnerability typically exploited by worms and viruses.
Microsoft said it uses “sound verification tools” that ensure that “entire classes of programmer errors are removed from the system early in the development cycle” of software, which should enable a clean development environment.
Three ideas to carry the development of Singularity
The kernel is the core technology supporting new ideas and technologies introduced by Singularity: It provides the base abstractions of the software-isolated processes (SIPs), contract-based channels, and manifest-based programs. So called SIPs have been created to break down programs into components that are isolated from other software components running on the same device. This enables pieces of the operating system to fail without the whole system to crash. You may know this concept from virtualization, where one operating system could face trouble, but leaves every other software on a system completely intact.
“In the past, creating many processes in an operating system was impractical from a cost perspective, largely because they required special support from the hardware,” Hunt says. “Rather than use hardware to build boundaries between processes, we figured out a way to build processes using a software technology called static analysis.” This analysis inspects a program’s source code in advance to make sure a process obeys certain rules that guarantee it’s isolated from the rest of the system.
The second architectural feature of Singularity are “contract-based channels” which enables SIPs to communicate with eachother and share information to achieve a stated goal. Microsoft believes that this approach will allow Singularity to avoid errors and “breakdowns” that are occurring today. “We figured out how to describe the form in which communications should take place between two processes, and using static analysis, we can check the codes of the processes at compile time,” Hunt explains. “So before the code ever runs, we can confirm that the processes communicate correctly.”
Third, there are “manifest-based programs” which has the goal to inform the operating system with basic parameters of the applications it runs. “We basically say, if you want to install a program to run on a Singularity system, you have to provide some information about it so we can preserve certain properties that make the system more reliable,” Larus explained. “You have to provide a manifest of the program pieces and how they fit together, and we’re going to check it. More important, we reserve the right to say ‘no.’ If a certain program doesn’t follow the rules set down for the system, you can’t install or run it.”
Read on the next page: Singularity's huge performance potential
From today’s view we would be already happy, if the promise of closing common security holes would come true. But deep in Singularity, there’s a new idea of increasing system performance and surely hope that Microsoft isn’t kidding about this one: Singularity goes multithreading-crazy.
It isn’t surprising that the Singularity kernel and SIPs are multi-threaded and it apparently can scale much further than to the 16 CPU cores, which are currently viewed as the limit where the integration of more cores will make more sense than optimizing the code-base, instructions and extensions. The big news about Singularity really is that Microsoft wants the OS to support heterogeneous microprocessors – microprocessors that not only integrate classic CPUs, but other processing cores such as GPUs.
“Thanks to physical constraints, it is easier to replicate processors on a die than to increase processor speed. With vendors already demonstrating prototype chips with 80 processing cores, we have begun experiments in support for so-called many-core systems in Singularity. These experiments build on the SMP support already provided by the Singularity kernel,” the developers wrote in a research paper. Singularity in fact can specialize many-core processors by dedicating them to specific SIPs, which should result in a greater “dynamic specialization” of the processor, thereby enhancing its performance.
Even more interesting, Singularity is the first operating system we hear of that explores the opportunity of exploiting the massive floating point horsepower of graphics chips. AMD and Nvidia have begun pitching their Tesla and FireStream stream processor cards as integral parts of deskside supercomputers, which offer about 20 times the speed of a common dual-core CPU in compute-intensive applications, such as financial modeling or scientific simulations. This area is probably the most significant untapped horsepower resource in the modern computing world: the home-user is locked out of this technology since there are no mainstream applications that take advantage of “general purpose GPU” computing.
Both AMD and Nvidia limit their stream processor products for high-margin products at this time. AMD has shown some experimental consumer GPGPU applications, but is far from offering such a product. Nvidia’s CEO Jen-Hsun Huang told TG Daily that his company would only take Tesla-like consumer products into the mainstream, if Microsoft would provide a Windows API for it.
It appears that Singularity will offer such an API. In their research paper, Hunt and Larus write: “Within Singularity, we see an opportunity to pursue a new course for programmable I/O processors. We agree (…) that programmable I/O processors are here to stay due to the better performance-per-watt of specialized processors. (…) We are exploring the hypothesis that programmable I/O processors should become first-class entities to OS scheduling and compute abstractions.”
The developers believe that the architecture of Singularity is uniquely positioned to enable this feature: SIPs minimize the need for elaborate processor features on I/O cores, contract-based channels explicitly define the communication between a SIP on an I/O processor and other SIPs, the technology’s memory isolation invariant removes the need for shared memory between SIPs on co-processors and CPUs, the small, process-local ABI isolates operations that may be safely implemented locally from services which must involve other SIPs and Singularity packages manifest-based programs in the abstract MSIL format, which can be converted to any I/O processor’s instruction set.
“Our hypothesis is that Singularity binaries can target legacy-free cores on many-core CPUs as easily as legacy-free cores on programmable I/O processors,” the developers said.
If the Hunt and Larus are right, Singularity could become an extremely flexible operating system that could take advantage of many of the hardware ideas floating around today. Of course, and you may not be surprised, there’s a catch.
The software is not only far from being usable for the average PC user, it is not even intended to make its way to the desktop. Instead, Singularity is described as “a laboratory for experimentation in new design” and not as a “design solution.”
“While we like to think our current code base represents a significant step forward from prior work, we do not see it as an ideal system or an end in itself. A research prototype such as Singularity is intentionally a work in progress; it is a laboratory in which we continue to explore implementations and trade-offs,” the developers wrote.