Opinion - For low-level software developers, having a machine emulator is an absolutely essential tool. It is often a necessity to step through lines of code one by one via a debugging process in some way that can't always be done easily or economically on a physical machine. One such example is developing software for mobile devices. These are typically developed and tested completely in an emulator, only to eventually be deployed to the real device. In the past few weeks I've spent some time migrating old 80386 code to now run inside of VMware Server. In the process I've discovered a serious limitation in the debugging capabilities of the software API. These limitations exist because of business practices and not because of hardware or software limitations.


While VMware's virtualization technology has taken the machine from a physical, cold and distant box somewhere, and brought it onto our desktops in a concise package, the full implementation of that migration is not yet known to developers.

VMware is not providing a comprehensive set of debugging tools for their virtual machine platform. And what I'm referring to is not something along the lines of their developers going to great lengths to make it happen through months of development. I'm talking about native machine abilities that already exist in the host OS, inside of VMware, and on the physical and virtual machines. And yet, those abilities are intentionally being kept from the development community.

Today, VMware provides debugging abilities through an implementation of software protocols, installed helper applications and library toolsets. But VMware is ignoring the most powerful debugging abilities present at the very heart of the x86 ISA, and denying developers access to those abilities.

These limitations keep VMware's virtualization technology from being the most comprehensive debugging tool I've ever seen. One which would allow any non-traditional x86 development effort to proceed at a pace not previously possible outside the world of expensive in-circuit-emulators and specialized or customized abilities.

Basically, VMware could help bring consumers better products through super-rapid development available to all developers. This would lower the cost of application development, lowering the cost of applications, and allowing for much faster turnaround times. And, from a developer's point of view, it would provide a much easier and more pleasant debugging and development experience.

VMware's decision not to include these abilities keeps the door closed between machine-level developers and VMware's otherwise amazing product.


Background

Recently, I've been in contact with VMware through their community forum. Specifically with Ramesh Dharan, a VMware employee/developer who's been with VMware ever since he graduated from Princeton University in 2002 with a BSE in Computer Science.

During the migration of some of my old code written for the 80386 into VMware Server, I had discovered the need to be able to do a particular thing. In the old days, there used to be physical areas of machine memory which were traditionally allocated to specific devices. For example, the VGA cards used memory locations 0xb8000 through 0xbffff for text, and 0xa0000 through 0xaffff for graphics. Well over the years I had developed a monochrome monitor debugger which utilized monochrome card memory space between 0xb0000 and 0xb7ffff.

My debugger has become an intregral component of my low-level software development. And to accomodate the fact that VMware does not provide a virtual monochrome graphics card, I posted a few questions to the VMware community asking for help.

I was looking to discover if there was a virtual Hercules Monochrome Graphics Adapter (MGA) that had already been developed for VMware. And if not, then I offered my services to write one for free, even desiring to release my work as open source when finished.


Hercules Monochrome Graphics Adapter - A powerhouse

The Hercules MGA is an exceedingly simple device. It was developed in the 1970s and comprises three data ports and a small 32 KB block of memory used for displaying pages of text and graphics. It has basic display abilities, such as displaying text with normal intensity, bright intensity, blinking, and in reverse video variations of those attributes in text mode. In graphics mode it presents a 720 x 350 pixel array of a single color, allowing for 90 x 43.5 characters in an 8x8 font. Its memory requirements are hard-coded to 0xb0000 through 0xb7fff for both text and graphics, and therein lies its awesome utility for debugging.

I had developed a low-level graphical debugger called Debi (DEBugger for Intel). It includes a mouse pointer, flashing cursor, several different fonts, reverse video and a host of useful abilities, such as watches, breakpoints, memory, register displays, disassembler, etc. I have used that debugger for stepping through custom developed operating system code, even boot-up code loaded from a floppy disk. And, after years of tweaking, it works perfectly and is very robust. Even more so than Microsoft's old CodeView debuggers for DOS. And, it has some abilities that even modern day Windows debuggers don't have.

The benefits of using a monochrome monitor for debugging may not be readily apparent. But even Microsoft's CodeView debugger for DOS can run on a monochrome monitor with the "/2" switch, as in "cv /2 myapp.exe".

Since the monochrome monitor always operates in the memory range of 0xb0000 - 0xb7fff, it does not interfer with the 0xb8000 - 0xbffff or 0xa0000 - 0xaffff ranges used by the VGA card. This allows algorithms which use the screen to be debugged without affecting what's on the screen.

Similar modern efforts allow for remote debugging. Instead of using something like the monochrome monitor for showing debug information without affecting what's on the display, the machine's state is communicated over Ethernet to a remote system which displays the information. While this implementation is faster and works exceedingly well, it requires that specific OS tools be installed to function. And if a developer is not coding in Linux or Windows, for example, or hasn't developed those tools for themself, then such abilities are not possible.

The hardware solution available from the monochrome display is always present, always possible, provided the hardware is there.


VMware responds

In response to my posts to VMware's discussion boards asking about me writing a virtual Hercules MGA, Dharan stated, "We don't have any APIs for arbitrary I/O port interception/redirection, or for arbitrarily reading/writing guest physical memory. So unfortunately I don't think what you want to do is possible using any existing VMware products."

Dharan went on to state, "VMware's focus has always been on virtualization, not emulation. It's true that we have to emulate a bunch of hardware devices, but our focus is on high performance virtualization of modern production workloads and hardware, not on helping people emulate hardware that no longer exists."

It may be worth pointing out that VMware currently emulates a complete virtual machine, the Intel 440BX chipset and motherboard, keyboard controllers, mouse controllers, internal abilities such as timers and programmable interrupt controllers, a VESA compatible video card, SoundBlaster sound card, hard drive, floppy drive, USB ports and much, much more. It is an amazingly sophisticated piece of technology. And all of these virtual devices are presented to the executing OS in near real-time.


Virtualization could make it so easy

The fact that VMware is already doing so much is what makes what I'm after possible. And, as a low-level software developer I realize what's involved in exposing these abilities through something like VMware's API. In fact, the ability to do what I'm looking for already exists in VMware in a feature they call "snapshot."

Snapshots take a complete picture of the running virtual machine state exactly as it was when the "snapshot" button is pressed. It writes everything to disk, including the machine's complete state and everything in memory. Two specific files are created, one called "snapshot.vmem" which is the entire virtual machine's memory, and one called "snapshot.vmsn," which is presumably the internal hardware state. The memory file alone proves that the host can interact with a running machine, pulling down its state and reading all of its memory.

The snapshot feature is so completely powerful and robust that it actually serves as a type of "safe harbor anchor" prior to moving forward with tests, such as installing a particular setup or configuration for deployment. It works like this:

A snapshot is made before a new program is installed, or before a change of some kind is made (like a registry modification or disk change). The snapshot exists as the machine was at a particular point and time, and allows it to revert back to its previous state should there be any undesired results from a failed install or bungled attempt at modifying something. No longer is there just a crashed machine requiring hours to reinstall the OS and applications. Instead, there is an almost immediately restored machine to its previously functioning state.

The differences between having this existing ability that exists today via the snapshot button, and the the ability to read blocks of memory from the virtual machine, are so small that they are practically non-existent. In fact, all that would be required to expose this amazing ability through the API is simply doing it, and then releasing a new version which contains it. All of the algorithms necessary to expose this ability have already been written. Third party developers just need a way to access and direct it to happen on demand.

Read on the next page: VMware's business practices fail low-level software developers