VMware: Potentially the world's most powerful low-level debugging tool

Posted by Rick C. Hodgin

Opinion - For low-level software developers, having a machine emulator is an absolutely essential tool. It is often a necessity to step through lines of code one by one via a debugging process in some way that can't always be done easily or economically on a physical machine. One such example is developing software for mobile devices. These are typically developed and tested completely in an emulator, only to eventually be deployed to the real device. In the past few weeks I've spent some time migrating old 80386 code to now run inside of VMware Server. In the process I've discovered a serious limitation in the debugging capabilities of the software API. These limitations exist because of business practices and not because of hardware or software limitations.

While VMware's virtualization technology has taken the machine from a physical, cold and distant box somewhere, and brought it onto our desktops in a concise package, the full implementation of that migration is not yet known to developers.

VMware is not providing a comprehensive set of debugging tools for their virtual machine platform. And what I'm referring to is not something along the lines of their developers going to great lengths to make it happen through months of development. I'm talking about native machine abilities that already exist in the host OS, inside of VMware, and on the physical and virtual machines. And yet, those abilities are intentionally being kept from the development community.

Today, VMware provides debugging abilities through an implementation of software protocols, installed helper applications and library toolsets. But VMware is ignoring the most powerful debugging abilities present at the very heart of the x86 ISA, and denying developers access to those abilities.

These limitations keep VMware's virtualization technology from being the most comprehensive debugging tool I've ever seen. One which would allow any non-traditional x86 development effort to proceed at a pace not previously possible outside the world of expensive in-circuit-emulators and specialized or customized abilities.

Basically, VMware could help bring consumers better products through super-rapid development available to all developers. This would lower the cost of application development, lowering the cost of applications, and allowing for much faster turnaround times. And, from a developer's point of view, it would provide a much easier and more pleasant debugging and development experience.

VMware's decision not to include these abilities keeps the door closed between machine-level developers and VMware's otherwise amazing product.

Background

Recently, I've been in contact with VMware through their community forum. Specifically with Ramesh Dharan, a VMware employee/developer who's been with VMware ever since he graduated from Princeton University in 2002 with a BSE in Computer Science.

During the migration of some of my old code written for the 80386 into VMware Server, I had discovered the need to be able to do a particular thing. In the old days, there used to be physical areas of machine memory which were traditionally allocated to specific devices. For example, the VGA cards used memory locations 0xb8000 through 0xbffff for text, and 0xa0000 through 0xaffff for graphics. Well over the years I had developed a monochrome monitor debugger which utilized monochrome card memory space between 0xb0000 and 0xb7ffff.

My debugger has become an intregral component of my low-level software development. And to accomodate the fact that VMware does not provide a virtual monochrome graphics card, I posted a few questions to the VMware community asking for help.

I was looking to discover if there was a virtual Hercules Monochrome Graphics Adapter (MGA) that had already been developed for VMware. And if not, then I offered my services to write one for free, even desiring to release my work as open source when finished.

Hercules Monochrome Graphics Adapter - A powerhouse

The Hercules MGA is an exceedingly simple device. It was developed in the 1970s and comprises three data ports and a small 32 KB block of memory used for displaying pages of text and graphics. It has basic display abilities, such as displaying text with normal intensity, bright intensity, blinking, and in reverse video variations of those attributes in text mode. In graphics mode it presents a 720 x 350 pixel array of a single color, allowing for 90 x 43.5 characters in an 8x8 font. Its memory requirements are hard-coded to 0xb0000 through 0xb7fff for both text and graphics, and therein lies its awesome utility for debugging.

I had developed a low-level graphical debugger called Debi (DEBugger for Intel). It includes a mouse pointer, flashing cursor, several different fonts, reverse video and a host of useful abilities, such as watches, breakpoints, memory, register displays, disassembler, etc. I have used that debugger for stepping through custom developed operating system code, even boot-up code loaded from a floppy disk. And, after years of tweaking, it works perfectly and is very robust. Even more so than Microsoft's old CodeView debuggers for DOS. And, it has some abilities that even modern day Windows debuggers don't have.

The benefits of using a monochrome monitor for debugging may not be readily apparent. But even Microsoft's CodeView debugger for DOS can run on a monochrome monitor with the "/2" switch, as in "cv /2 myapp.exe".

Since the monochrome monitor always operates in the memory range of 0xb0000 - 0xb7fff, it does not interfer with the 0xb8000 - 0xbffff or 0xa0000 - 0xaffff ranges used by the VGA card. This allows algorithms which use the screen to be debugged without affecting what's on the screen.

Similar modern efforts allow for remote debugging. Instead of using something like the monochrome monitor for showing debug information without affecting what's on the display, the machine's state is communicated over Ethernet to a remote system which displays the information. While this implementation is faster and works exceedingly well, it requires that specific OS tools be installed to function. And if a developer is not coding in Linux or Windows, for example, or hasn't developed those tools for themself, then such abilities are not possible.

The hardware solution available from the monochrome display is always present, always possible, provided the hardware is there.

VMware responds

In response to my posts to VMware's discussion boards asking about me writing a virtual Hercules MGA, Dharan stated, "We don't have any APIs for arbitrary I/O port interception/redirection, or for arbitrarily reading/writing guest physical memory. So unfortunately I don't think what you want to do is possible using any existing VMware products."

Dharan went on to state, "VMware's focus has always been on virtualization, not emulation. It's true that we have to emulate a bunch of hardware devices, but our focus is on high performance virtualization of modern production workloads and hardware, not on helping people emulate hardware that no longer exists."

It may be worth pointing out that VMware currently emulates a complete virtual machine, the Intel 440BX chipset and motherboard, keyboard controllers, mouse controllers, internal abilities such as timers and programmable interrupt controllers, a VESA compatible video card, SoundBlaster sound card, hard drive, floppy drive, USB ports and much, much more. It is an amazingly sophisticated piece of technology. And all of these virtual devices are presented to the executing OS in near real-time.

Virtualization could make it so easy

The fact that VMware is already doing so much is what makes what I'm after possible. And, as a low-level software developer I realize what's involved in exposing these abilities through something like VMware's API. In fact, the ability to do what I'm looking for already exists in VMware in a feature they call "snapshot."

Snapshots take a complete picture of the running virtual machine state exactly as it was when the "snapshot" button is pressed. It writes everything to disk, including the machine's complete state and everything in memory. Two specific files are created, one called "snapshot.vmem" which is the entire virtual machine's memory, and one called "snapshot.vmsn," which is presumably the internal hardware state. The memory file alone proves that the host can interact with a running machine, pulling down its state and reading all of its memory.

The snapshot feature is so completely powerful and robust that it actually serves as a type of "safe harbor anchor" prior to moving forward with tests, such as installing a particular setup or configuration for deployment. It works like this:

A snapshot is made before a new program is installed, or before a change of some kind is made (like a registry modification or disk change). The snapshot exists as the machine was at a particular point and time, and allows it to revert back to its previous state should there be any undesired results from a failed install or bungled attempt at modifying something. No longer is there just a crashed machine requiring hours to reinstall the OS and applications. Instead, there is an almost immediately restored machine to its previously functioning state.

The differences between having this existing ability that exists today via the snapshot button, and the the ability to read blocks of memory from the virtual machine, are so small that they are practically non-existent. In fact, all that would be required to expose this amazing ability through the API is simply doing it, and then releasing a new version which contains it. All of the algorithms necessary to expose this ability have already been written. Third party developers just need a way to access and direct it to happen on demand.


Read on the next page:
VMware's business practices fail low-level software developers


VMware's business practices fail low-level software developers

VMware's focus seems to be only on big business. They are not interested in addressing and exposing some of the most powerful aspects of their virtual machine's abilities. They are leveraging their time and energy in support of tools that allow current, mainstream products to be effectively debugged and communicated with from inside the running virtual machine only. This allows modern compilers, debuggers and operating systems to integrate very well with VMware's debugging abilities. But it also leaves anyone doing non-traditional low-level x86 hardware development out in the cold. And again, it doesn't have to be that way. It's just a business decision to make it that way. 

VMware is leaving out these crucial features even though they are 1) simplistic, 2) fundamental, 3) naturally indicated by the very nature of the product, and 4) proven to already exist within their released executable (via snapshots).

Summary

VMware has the potential to be the de facto standard for all x86 software development, regardless of scope or intent. They could open up a completely new market. Their product is already one of the best (if not the best) virtual machine manager out there. And, they keep improving it every day. Were they to include these four basic abilities:

1) Read a memory range
2) Write to a memory range
3) Intercept or hook I/O data port reads
4) Intercept or hook I/O data port writes

then VMware could market themselves not only as the best virtual machine in the world, but also as the most powerful x86 debugging tool ever created for public consumption - which is exactly what VMware's product would be at that time.

One last feature request

There has to be some way for the virtual machine to communicate with the host in a natively provided for way by the x86 ISA. As such, I would also propose the addition of one more feature to VMware's debugging arsenal, one which would be the capper.

I propose VMware integrates a programmable interrupt vector to be setup in the console, one which would default to INT 0xff. That vector would be intercepted by the host and used to communicate a message pointed to by ds:[esi/rsi], of ecx/rcx bytes, and identified by the value 0 in eax/rax for a send (and a value of 1 for a receive), and an application-specific handle or value stored in ebx/rbx to any third party application which inquires of the host if such a message has been sent/received from the guest OS.

Such an ability would allow the low-level code to "call home" to its parent application running alongside the host. Such a feature could be used to indicate where the guest setup memory blocks for reading and writing, for example.  And, a similar ability with a value of 1 in eax/rax would poll for any data sent to the guest from a third party operating outside the host.

These features would be part of the machine-native communications abilities VMware would then expose between itself and third party apps. No special protocols, no drivers, no required operating systems or installed tools would be necessary to make it all happen. It would simply be abilities which already exists in both the physical and virtual machine, working together in harmony.

Conclusion

Such abilities would present a most powerful, generic and flexible way of cross-platform communication that's ever been possible. It's one which leaves the decision as to how they should be implemented to the developer. One that requires no drivers whatsoever. No special coding of any kind. Just a few function calls issued as easily as anything else in software development and voila: fully robust communications and debugging abilities.

Why VMware hasn't already added abilities like this truly baffles me as the very nature of the beast not only lends itself purposefully, directly and exactly toward those abilities, but it almost demands they exist.

Anyone care to guess why they're not including it?