I began by converting this skeleton into C++, since I would have to implement a COM dll with it. I looked at some of our other C++-based COM dlls, and I found some code that I copypasted into the skeleton. This code used the ATL module system as a means to implement the basic COM functions for DLLs. That is, DllRegisterServer, DllGetClassObject, DllUnregisterServer, and DllCanUnloadNow.
The module, when initializing, takes a special “table” which maps COM classes and their CLSIDs, in order for them to get registered and loaded by the module, when the functions mentioned above are called. To fill in this table, I added a class, which I named (like the windows equivalent) “CSysTray”.
A small aside here, there is a big misconception that the icons next to the clock, in the taskbar, are called “tray icons” or similar. This is a mistake that originated in the young times of Windows 95 development, where the taskbar was instead a tray. That is, a folder docked to the edge of the screen, which the user could open and close as needed. Because of that, they came to be known as “tray icons”. Their proper name, for future reference, is “notification icon”. But bear with me if I still prefer to call them tray icons instead. Nearly 20 years of habit are hard to change. I mentioned this because, as you can see, the class name still follows the ancient naming, which it has inherited across all versions of windows, since the times it was still a separate program that would run at startup.
As a Shell Service Object, the CSysTray class is only expected to have one interface implemented: IOleCommandTarget. Of this interface, only one method is used by SSOs. When it is called in this context, the Exec method can have two values in the command ID: 2 and 3, which correspond to “new”, and “save” in the ole terminology. These two commands are repurposed to mean “init” and “shutdown” (respectively) by the SSO manager.
It was around this point, that the weekend came, and I started keeping the burn uncovered. While still cleaning it and treating it. I didn’t write the report yet, because I had not much to show, but also because I completely forgot about it until this past Tuesday.
Over the week, I continued working on the DLL. Inside this skeleton CSysTray, I implemented an “icon manager”, which would run a thread with an hidden window, used as a target for the messages sent by the icons. It would also take care of a list of icon handlers, and send them notifications about the state of the icons, while the handlers could call the methods to add, modify, or remove icons.
Meanwhile, I also programmed a Volume icon handler, initially as a skeleton, which would receive those calls, and print debug messages. This allowed me to ensure incrementally that each part was working as expected.
To properly test this skeleton, I made it show one of the icons that Robert had included in his initial skeleton, and I ensured that the functions were being sent when the mouse was moved over the icon and such.
Later, I added an update timer to the CSysTray handler, which would notify the handlers periodically, letting them refresh the state of the icons. For now, I hardcoded the interval to 2000 milliseconds (that is, of course, 2 seconds). To test that the timer was working, and also that icons could be modified, I made the volume handler toggle between the normal volume icon, and the muted volume icon.
Looking at the imports and pdb function lists from the windows DLL, I could easily guess that it was using the winmm API to obtain the mixer controls. Tracing the usage of these functions, I saw that the primary usage was to obtain the MUTE control, and use it to choose which icon to display. The popup with the volume is handled by a different set of functions, so I left that aside. I looked for some examples of how to use this API, and tried to replicate the windows behaviour by obtaining the default waveOut device, obtaining the mixer id from the device id, then obtaining the line control id corresponding to the MUTE control type. This control id would then later be queried to obtain the mute status, through the update function called by the timer.
With this code, I managed to load the DLL in Windows 2003, by “hijacking” the registry to change the path of the DLL in the COM registration. This allowed me to debug and verify the code. Sadly, the equivalent winmm code in ReactOS has some missing features, which means the part where I try to obtain the default waveOut device, isn’t working. Given that we have no waveOut device to obtain the mixer id from, I decided to have a fallback case where the first available mixer is used, even if it could be the wrong one.
With this fixed, I could manually register the DLL in reactos, and have it show a dummy icon, since the audio driver from the VMware Tools (my existing and preferred VM software) does not appear to install a mixer in ReactOS. This meant, all I had left to do in this phase of the implementation was to ask the reactos setup process to create the necessary registry keys, and then on the second phase of the setup (the graphical wizard), run the registration function from the DLL.
This is when things got a bit confusing, since although the registration worked perfectly well when building with MSVC (Visual Studio 2013), when I tested the MingW-GCC compilation and setup, there was no icon to see afterwards.
It took me – with the help of some other reactos developers – quite a few hours of head-scratching to realize the answer was simpler than it looked. If nothing worked, it was simply because the ATL module didn’t have anything to do at all. Because the DllMain function wasn’t running, and the ATL module hadn’t been initialized with a COM class table, it believed there were no classes to register or load, so it just returned without doing anything!
I traced the problem to the naming convention of the DllMain function, which was supposed to be in the C style and not C++ style (missing ‘extern “C”’). So you may ask why it worked with msvc. And you do well to ask, because it turns out the Microsoft C compiler has it hardcoded to always treat a function called DllMain as “C”, even if you didn’t ask for it! And not even a warning to make people aware of the automatic change!
Until next week!