Network enabled apps such as news aggregators, blogging clients often load lots of small files, such as thumbnails, trough HTTP connection. One of them is my app – Kozaczek, that loads thumbnails for every UITableView item when it is about to be displayed. Usually we expect that such application is: (1) responsive (does not block scrolling while loading), (2) it does load thumbnails almost realtime.

I suppose the first thing everybody tries is calling [UIImage imageWithData:[NSData dataWithContentsOfURL:someURL]]. There is nothing wrong with it, except it make your UI stutter and load images painfully slow. After trying that one may think of using background threads, NSOperationQueue, or something like that, but still any modern web browser loads those images much faster when they are embedded into regular web page than your application. Wonder why?

Man behind the bar at HTTP server

HTTP 1.1 introduces feature called persistent connections – later backported also to HTTP 1.0 in the form of an extra header Connection: keep-alive sent by the client. This is the clue of the solution here.

You may think of HTTP server working thread serving you as bartender taking your order and bring back drinks. First your connection is waiting in the queue to the bar, once bartender dispatches all orders of clients in the front of you, he asks for yours. Then (during the connection) you have the bartender (almost) exclusively serving you. Once you disconnect, the bartender focuses his attention on the next client, so if you decide to make another request short while afterwards you need to go back in the queue and wait again for your turn.

Back in HTTP 1.0 times, server was taking only one order, sending back response and disconnecting. But since HTTP 1.1 you may send many orders to the server one after the other (or even parallel). You can also stay connected while you are non sending requests, just in case you want something extra. All modern web browsers follow simple rule of trying to execute all the requests using existing persistent connection while not making more than 2-4 connections to single server.

Thats why putting your thumbnails download in background threads or NSOperationQueue does not do any better, but just pollutes remote HTTP server with many connections, that anyway will not be served altogether. Moreover the overall time you gonna spend waiting for response for all requests will be significantly longer than time use with single persistent connection sending requests one-by-one.

CFNetwork HTTP connection pooling

It is better then to keep your application single threaded, but instead use good event driven download mechanism based on persistent connections. The first candidate for that is NSURLConnection which unfortunately has some problems with keeping persistent connections to HTTP 1.0 servers via keep-alive header, even it uses internally CFNetwork. This is the reason I will focus on more low level solution directly based on CFNetwork‘s CFReadStream object and CFReadStreamCreateForHTTPRequest function.

CFNetwork documentation states that this framework tries to pool and reuse existing HTTP connections for newly created CFReadStream if there is still other CFReadStream connected to the same server. This implies two facts: (1) CFNetwork does queue all read requests to the same server where possible trough single connection (2) if only you keep the CFReadStream reference to the server (i.e. using global variable), so every time you open new CFReadStream, it will reuse connection kept inside previously used CFReadStream.

Below I attach ready to use implementation of Fetch class (under MIT-like license) that does all of that for you. All you need to do is to [Fetch fetchURL:someURL delegate:self tag:someTagForYourComfort]. The connection itself does retain Fetch instance and releases it once the request if over, so you do not need to retain it yourself. The delegate (also retained in case it is deallocated before request is over) should implement 3 methods:

- (void)fetch:(Fetch *)fetch didFailWithError:(NSError *)error;
- (void)fetchDidFinishLoading:(Fetch *)connection;
- (void)fetch:(Fetch *)fetch didReceiveStatusCode:(NSInteger)statusCode contentLength:(NSInteger)contentLength;

HINT #1: Once you receive didReceiveStatusCode, you should assign fetch.data property (retained) some NSMutableData instance otherwise it will not store downloaded data anywhere.

HINT #2: If you wish to cancel Fetch request you may use -[Fetch cancel].

• Fetch.h downloader class interface
• Fetch.m and implementation

Since iPhone/iPod Touch is pretty quick to do image decoding in main thread once you get the data, creating image with [UIImage imageWithData:fetch.data] in fetchDidFinishLoading and binding the image to UITableView cell works for me fine.

Implementation part that deserves the attention is initialization part:

- (id)initWithURL:(NSURL *)_url
         delegate:(id<FetchDelegate>)_delegate
              tag:(NSInteger)_tag
{
    // Copy properties
    self.delegate = _delegate;
    tag = _tag;

    CFHTTPMessageRef request = CFHTTPMessageCreateRequest(
                                                          kCFAllocatorDefault,
                                                          CFSTR("GET"),
                                                          (CFURLRef)_url,
                                                          kCFHTTPVersion1_1);
    CFHTTPMessageSetHeaderFieldValue(request, CFSTR("Keep-Alive"), CFSTR("30"));

    stream = CFReadStreamCreateForHTTPRequest(kCFAllocatorDefault, request);
    CFRelease(request);
    CFStreamClientContext context = {
        0, (void *)self,
        CFClientRetain,
        CFClientRelease,
        CFClientDescribeCopy
    };
    CFReadStreamSetClient(stream,
                          kCFStreamEventHasBytesAvailable |
                          kCFStreamEventErrorOccurred |
                          kCFStreamEventEndEncountered,
                          FetchReadCallBack,
                          &context);
    CFReadStreamScheduleWithRunLoop(stream,
                                    CFRunLoopGetCurrent(),
                                    kCFRunLoopCommonModes);

    // In meantime our persistent stream may be closed, check that.
    // If we won't do it, our new stream will raise an error on startup
    // FIXME: This is a bug in CFNetwork!
    if(persistentStream != NULL) {
        CFStreamStatus status = CFReadStreamGetStatus(persistentStream);
        if(status == kCFStreamStatusNotOpen || status == kCFStreamStatusClosed || status == kCFStreamStatusError) {
            CFReadStreamClose(persistentStream);
            CFRelease(persistentStream);
            persistentStream = NULL;
        }
    }

    CFReadStreamSetProperty(stream, kCFStreamPropertyHTTPAttemptPersistentConnection, kCFBooleanTrue);
    CFReadStreamOpen(stream);

    if(persistentStream != NULL) {
        CFReadStreamClose(persistentStream);
        CFRelease(persistentStream);
        persistentStream = NULL;
    }

    streamCount++;

    return self;
}

Few notes about initialization part of Fetch instance:

1. It forces using 30 seconds keep-alive trough CFHTTPMessageSetHeaderFieldValue
2. It retains itself using CFReadStreamSetClient
3. It uses main thread (current) runloop with CFReadStreamScheduleWithRunLoop
4. It maintains persistentStream global reference to last used stream, and replaces it by its own connection (so persistentStream always points to the last one)
5. Finally it forces persistent connections, even they may be default (or not?) using CFReadStreamSetProperty(stream, kCFStreamPropertyHTTPAttemptPersistentConnection, kCFBooleanTrue);

So this is all folks. Waiting for your comments. I hope you will find this post useful.

Seems it is very hard to spread the news about CMSity around the world, and acquire some user base of this project. Till now I did few commercial deployments of CMSity, however the cmsity.com page hits look really miserable.

I wrote mails to many CMS magazines asking to drop a note about my project. However only two of them replied and posted something on their sites about CMSity. So I still miss some more reviews that can put a breathe into the project. But it seems my mails are ignored by majority of big CMS related sites making very hard to promote new project like CMSity nowadays.

Since I believe this software presents great value and outperforms other popular solutions I consider releasing it as open-source, however probably not GPL but some license that will force the code to stay by the author (me), deny all the forks, making all community changes happen to be included in the source code base.

Open-source is a great idea, but I have seen too many projects that were just bloated, forked and spoiled by masses of developers that were just about put something of themselves in the project regardless of quality of the included changes, missing overall sense and directions of the project itself. That is why I decided to keep the code closed initially, while giving the licenses for free, but it seems non-open source project are just kicked out of the focus today, and releasing the source code is a “must” to gain the momentum.

So I decided to create Haml “mod” that uses GetText (FastGettext alternatively) to automagically translate static texts from Haml templates during precompilation stage. So something that you don’t see and you don’t need to worry about.

Normally you would use _(text) calls to translate string in your templates as below:

%h1= _('Items')
- items.each do |item|
  .item
    %h1= item.title
    %a{ href='/edit' }= _('Edit')
    %a{ href='/delete' }= _('Delete')

You can see this is pretty ugly and unreadable. If you gonna have multi-language site you want to translate all of the static texts there anyway, without doing anything extra. So you would prefer to type:

%h1= Items
- items.each do |item|
  .item
    %h1= item.title
    %a{ href='/edit' } Edit
    %a{ href='/delete' } Delete

And have “Items”, “Edit”, “Delete” just translated, right? This is what my mod does. It translates all of those at precompilation stage which also is much better for your application performance, since GetText routines are called only once at precompilation stage. Here it goes.

haml_gettext.rb

# Haml gettext module providing gettext translation for all Haml plain text calls
#
# http://pastie.org/445295

class Haml::Engine
  # Inject _ gettext into plain text and tag plain text calls
  def push_plain(text)
    super(_(text))
  end
  def parse_tag(line)
    tag_name, attributes, attributes_hash, object_ref, nuke_outer_whitespace,
      nuke_inner_whitespace, action, value = super(line)
    value = _(value) unless action || value.empty?
    [tag_name, attributes, attributes_hash, object_ref, nuke_outer_whitespace,
        nuke_inner_whitespace, action, value]
  end
end

All you need is to choose GetText or FastGettext and load my mod later on.

require 'haml'
require 'fast_gettext'
include FastGettext::Translation
require 'haml_gettext'

You probably want to know the way to your gettext pot/po files out of the tempaltes. I have prepared here Rake tasks for you as well (based on work I found somewhere over Ruby blogs) to parse Haml templates extended to include static texts in the translation.

Rakefile

desc "Update pot/po files."
task :updatepo do
  require 'gettext/tools'
  require 'haml_parser'
  GetText.update_pofiles("rbigg", Dir.glob("{lib,views}/**/*.{rb,haml}") << "rbigg.rb", "rbigg 1.0.0", :po_root => 'locale')
end

haml_parser.rb

# Haml gettext parser module
#
# http://pastie.org/445297
require 'gettext/tools/rgettext'
require 'gettext/parser/ruby'
require 'haml'

class Haml::Engine
  # Overriden function that parses Haml tags
  # Injects gettext call for plain text action.
  def parse_tag(line)
    tag_name, attributes, attributes_hash, object_ref, nuke_outer_whitespace,
      nuke_inner_whitespace, action, value = super(line)
    @precompiled << "_(\"#{value}\")\n" unless action || value.empty?
    [tag_name, attributes, attributes_hash, object_ref, nuke_outer_whitespace,
        nuke_inner_whitespace, action, value]
  end
  # Overriden function that producted Haml plain text
  # Injects gettext call for plain text action.
  def push_plain(text)
    @precompiled << "_(\"#{text}\")\n"
  end
end

# Haml gettext parser
module HamlParser
  module_function

  def target?(file)
    File.extname(file) == ".haml"
  end

  def parse(file, ary = [])
    haml = Haml::Engine.new(IO.readlines(file).join)
    code = haml.precompiled.split(/$/)
    GetText::RubyParser.parse_lines(file, code, ary)
  end
end

GetText::RGetText.add_parser(HamlParser)

So this is it. Have fun with nice and clean templates that are translated on the fly

See & get more…

You can find more recent and complete code at my Sinatra Hat GitHub hosted Ruby gem project.

1. It uses now click menu instead of keystroke “System Events” command, because in some cases when you had this script assigned to shortcut that used Ctrl or Shift modifiers, those modifiers were sent together with Cmd to “Terminal” producing invalid behavior.
2. It waits 0.5 second when window is busy just in case Terminal.app was not running and it is just loading the shell which makes it busy for short while too. Fixes incorrect behavior of opening extra tab when Terminal.app was not running.

If you ever wondered how to open specified path in new tab of Terminal.app or reuse current one if it is not busy (running a command), here’s a script you may use:

tell application "Terminal"

    activate

    set windowCount to (count of the windows)

    -- Terminal may be just launched loading the shell, wait a bit
    if windowCount is greater than 0 and first window is busy then
        delay 0.5
    end

    -- Still busy / no windows? open new tab
    if windowCount is greater than 0 and first window is busy or windowCount is 0 then
        tell application "System Events" to tell process "Terminal"
            click first menu item of first menu of second menu item of first menu of third menu bar item of first menu bar
        end tell
    end if

    do script "cd #{e_as(e_sh(dir))}" in first window

end tell

Where #{e_as(e_sh(dir))} is your desired folder. This is modified chunk of TextMate‘s command script found at Mark Eli Kalderon’s Blog. Thanks Mark!

Note: Mark’s TextMate command script does not open anything when Terminal.app has no windows or it is not running.

It is hard to be Switcher and totally forget/abandon Windows projects, since most of my clients… well almost all of them are on *indows. So since I switched, I am constant user of VMWare Fusion + XP (Win7, Win98) as a guest OS having Visual Studio installed inside. But sometimes if you want to create small cute application for *indows I prefer do it 100% on Mac with TextMate, then just test it on Windows. This is what MinGW cross compiler is perfect for. Not to mention I did Miranda’s MinGW Makefile that successfully builds miranda on Linux/Mac using cross-compiler, so it can be queued for nightly builds on SF.net servers.

We cannot count on free “performance lunch” anymore, but how about at least a pie for free? Do we need to throw all our old source-code into the trash bin and start over again? 

Certainly, not. We may think of our old software as a zombie of the new multi-core era. Still there is a way to make the zombie walk, even walk faster. Of course we will need to rewrite our code sometime, but we may postpone this nasty need for a while. 

The computing paradigm shift is now a fact we need to learn how to live with. It is unquestionable that processor manufacturers hit the barrier of 3 GHz. There pretty many news about spinning the CPU up to 5 GHz or so, but do not try to do so at home unless you got liquid nitrogen cylinder around. Over 3 GHz heat emission grows unreasonably making the CPU economically worthwhile. 

So the only sensible way is now horizontal performance improvement, doubling the number of processing units. This means we can expect soon 128 core CPUs. Oh, wait they are here already; nVidia GeForce 9 series are perfect example of 128 core streaming processor. 

Now how to keep up with things that happen so fast. First we need focus more on performance of our code. Something that was not important before, now cannot be ignored.

Want more, read my article draft below:

• A Pie for free.pdf 256KB, 2008-06-16

After spending few years using WordPress, MODx, PHPnuke I found out those systems getting more and more complicated, well yes sophisticated too, but first of all complicated. Anyway, feeling that this is not right direction I have decided to rewrite something by my own; simple, but powerful enough to provide functionality that you expect from most of the modern sites – blog, forums, RSS feeds.

Such functionality was requested by few of my clients, asking me: “Adam, we need something like WordPress, but we would like to have forums too, also site should be multi-lingual and finally we would like to limit the access to some parts of the site.” Just to mention those clients were running websites at PHP virtual hosting. I said, “Roger that” and started looking for some (any) CMS that could satisfy their needs.

I use to browse for new CMSes time to time. I did it again last October. Nothing new. Okay, maybe I could somehow tailor MODx, or WordPress. Nah. Then I found BBpress, well I was excited. A forum system made by WordPress creators, that integrates with WP. Great! Maybe this will fit.

Unfortunately I was hitting so many problems syncing WP and BBpress user database and logins, cookies (Why the hell it is so complicated?), so I gave it up.

I decided to write my own system, having in mind that forum, blog, they just differ at presentation level! There is always some parent page and subpages, post and replies, forum and topics. All having some hierarchy relations, and different presentation model. So why we have to install BBpress over WP if it is just a matter of presentation. Forum should look more like a table… while blog more like list of post excerpts. Why we cannot just turn WP sub-blog into the forum?

The difference is the presentation

So thing that makes forum or blog special is the way it is presented to the user. But from structure point of view both are the same. So this is the idea of the CMSity. For CMSity all pages (document tree nodes) are the same, they just carry the type which is used to determine what template to use to render them – what is their presentation.

Why PHP? Even I know more optimized/fast languages such as Ruby (1.9 is really fast), Java, C++.. started learning Erlang, still PHP is kind of standard that is present on most of the web servers and you can expect your clients will have PHP on their web servers. So I have chosen PHP; not really my favorite language, but the one I know pretty well and have spent quite a while with.

So this is genesis of CMSity… hope it will find many users and will be useful for you all.

Recently I’ve received two papers from GNU (originated from U.S.) and few great stickers with gnus I had to sign it back after carefully studying them. And now I am officially assigned and my changes went to the official repository.

Can I call myself genuine OpenSource developer now !?

I’m an author of few patches for both Tortoise SVN and Tortoise CVS that makes them display the explorer’s context menu icons nicely on XP and Windows 2000. Both programs are implementing IContextMenu and using QueryContextMenu function to create items of popup menu of explorer. Briefly the called extension must fill menu items with InsertMenuItem using suplied HMENU hmenu parameter.

During development of those few patches I’ve learnt some few new things about way we make icons displayed next to menu items I want to share with you…

How to get icons in context menus

Methods described here are related to shell context menu extension, however they can be used in any Windows application.

hbmp(Un)checked method

Initially both Tortoises were filling hbmpUnchecked & hbmpChecked fields of MENUITEMINFO that is passed to InsertMenuItem with HBITMAP created from icon to get icons on menu item. This solution works on all Windows since 95. However the strong limitation is that HBITMAP must be SM_CXMENUCHECK x SM_CYMENUCHECK (usually 12 x 12). So if you are using 16 x 16 icon, the icon gets squished and looks awfully. The function used to convert icon to bitmap is:

HBITMAP CShellExt::IconToBitmap(std::string sIcon)
{
    RECT rect;

    rect.right = ::GetSystemMetrics(SM_CXMENUCHECK);
    rect.bottom = ::GetSystemMetrics(SM_CYMENUCHECK);

    rect.left = rect.top  = 0;

    HICON hIcon = (HICON)LoadImageA(g_hInstance, sIcon.c_str(), IMAGE_ICON, rect.right, rect.bottom, LR_DEFAULTCOLOR);
    if (!hIcon)
        return NULL;

    HWND desktop = ::GetDesktopWindow();
    if (desktop == NULL)
    {
        DestroyIcon(hIcon);
        return NULL;
    }

    HDC screen_dev = ::GetDC(desktop);
    if (screen_dev == NULL)
    {
        DestroyIcon(hIcon);
        return NULL;
    }

    // Create a compatible DC
    HDC dst_hdc = ::CreateCompatibleDC(screen_dev);
    if (dst_hdc == NULL)
    {
        DestroyIcon(hIcon);
        ::ReleaseDC(desktop, screen_dev);
        return NULL;
    }

    // Create a new bitmap of icon size
    HBITMAP bmp = ::CreateCompatibleBitmap(screen_dev, rect.right, rect.bottom);
    if (bmp == NULL)
    {
        DestroyIcon(hIcon);
        ::DeleteDC(dst_hdc);
        ::ReleaseDC(desktop, screen_dev);
        return NULL;
    }

    // Select it into the compatible DC
    HBITMAP old_dst_bmp = (HBITMAP)::SelectObject(dst_hdc, bmp);
    if (old_dst_bmp == NULL)
    {
        DestroyIcon(hIcon);
        return NULL;
    }

    // Fill the background of the compatible DC with the given colour
    ::SetBkColor(dst_hdc, RGB(255, 255, 255));
    ::ExtTextOut(dst_hdc, 0, 0, ETO_OPAQUE, &rect, NULL, 0, NULL);

    // Draw the icon into the compatible DC
    ::DrawIconEx(dst_hdc, 0, 0, hIcon, rect.right, rect.bottom, 0, NULL, DI_NORMAL);

    // Restore settings
    ::SelectObject(dst_hdc, old_dst_bmp);
    ::DeleteDC(dst_hdc);
    ::ReleaseDC(desktop, screen_dev);
    DestroyIcon(hIcon);
    return bmp;
}

Ownerdraw method

Tortoise SVN was using also owner draw method. I won’t describe here details of this method. This relays on MENUITEMINFO fType flag set to MFT_OWNERDRAW. Shell extension in HandleMenuMsg2 callback should handle WM_MEASUREITEM and WM_DRAWITEM. This method is generally OK, however it has several flaws:

1. We need to measure & draw menu in all stated ourselves, which makes us write plenty of code.
2. Ownerdraw menus are not respecting visual styles of Windows XP or Vista. We would need to use uxtheme functions to somehow handle rendering of menu parts on those systems.
3. We need to keep exta context information for each menu item with text, icon handle, etc.
4. Keyboard shortcuts doesn’t work automatically, we must handle WM_MENUCHAR to make them work.

HBMMENU_CALLBACK method

Since Windows 98 MENUITEMINFO has extra field hbmpItem. This field can be used for setting the HBITMAP with bitmap that is displayed next to the menu item. hbmpItem can be set also to HBMMENU_CALLBACK which will make menu item work like owner-draw, but WM_MEASUREITEM & WM_DRAWITEM just need to handle icon drawing, rest will be done by Windows. This method is easiest to implement and so it is used inside many application, I just name on I use or develop: wxWidgets SDK, Miranda IM. We just need to initialize menu item like that:

MENUITEMINFO menuiteminfo;
ZeroMemory(&menuiteminfo, sizeof(menuiteminfo));
menuiteminfo.cbSize = sizeof(menuiteminfo);
menuiteminfo.fMask = MIIM_FTYPE | MIIM_ID | MIIM_SUBMENU | MIIM_DATA | MIIM_BITMAP | MIIM_STRING;
menuiteminfo.fType = MFT_STRING;
menuiteminfo.dwTypeData = lpszMenuTitle;
menuiteminfo.cch = _tcslen(lpszMenuTitle);
menuiteminfo.hbmpItem = HBMMENU_CALLBACK;
menuiteminfo.wID = id;

Rest is done in WM_MEASUREITEM where we need to just make sure we have space for 16 x 16 image using:

    case WM_MEASUREITEM:
        {
            MEASUREITEMSTRUCT* lpmis = (MEASUREITEMSTRUCT*)lParam;
            if (lpmis==NULL)
                break;
            lpmis->itemWidth += 2;
            if (lpmis->itemHeight < 16)
                lpmis->itemHeight = 16;
            *pResult = TRUE;
        }
        break;

Then to draw an icon we need to handle WM_DRAWITEM, but just drawing the icon, nothing else:

    case WM_DRAWITEM:
        {
            LPCTSTR resource;
            DRAWITEMSTRUCT* lpdis = (DRAWITEMSTRUCT*)lParam;
            if ((lpdis==NULL)||(lpdis->CtlType != ODT_MENU))
                return S_OK; // not for a menu
            resource = GetMenuIconResourceID(lpdis->itemID);
            if (resource == NULL)
                return S_OK;
            HICON hIcon = (HICON)LoadImage(g_hResInst, resource, IMAGE_ICON, 16, 16, LR_DEFAULTCOLOR);
            if (hIcon == NULL)
                return S_OK;
            DrawIconEx(lpdis->hDC,
                lpdis->rcItem.left - 16,
                lpdis->rcItem.top + (lpdis->rcItem.bottom - lpdis->rcItem.top - 16) / 2,
                hIcon, 16, 16,
                0, NULL, DI_NORMAL);
            DestroyIcon(hIcon);
            *pResult = TRUE;
        }
        break;

Simple ? Yes it is. However there are some issues with this method as well:

1. When this method is used on Windows 2000 shell extension window background popup menu, then shell.dll is removing text from the menu, so we see just icons.This is obviously a bug of Windows 2000 shell.dll, because MSDN documentation states this shall work regardless of the sittuation, but we need to somehow get over it. Surprisingly it does work fine when we right-click on explorer item (file or folder). The easiest solution is using hbmp(Un)checked method when uFlags == 0 of QueryContextMenu, which indicated we clicked the background, so we fall back to most primitive method, but at least we got text and “some” icons in the menu.Note: This bug only appears in Windows 2000 explorer’s background menu of shell extension, so in every other situation as standalone program menus HBMMENU_CALLBACK method can be used without any problem. So you may not care about it unless you are shell extension developer.
2. Vista is removing menu theme when some menu item has hbmpItem set to HBMMENU_CALLBACK, so we will have nice icons and nice menu, but if we want to have nice icons with 100% themed menu on Vista we need to use last method…

Vista PARGB32 hbmpItem bitmap method (Updated)

Vista strongly relays on 32-bit pre-multiplied alpha RGB bitmaps for rendering its interface. In Vista hbmpItem can be set to PARGB32 HBITMAP and this bitmap will be nicely displayed by Vista together with theming as you can see on the screenshot at right. I got know of this possibility reading nice article Vista Style Menus, Part 1 – Adding icons to standard menus at ShellRevealed blog.

The most important question is how to we get our icon (regardless it is 32-bit with alpha, or 256-color with mask) converted to PARGB32 HBITMAP. Windows API doesn’t give such possibility straight of the box. Article from ShellRevealed proposes WIC (Windows Imaging Component) which is cool & quite simple for conversion or Vista‘s GDI method, but those require Vista SDK, which may be annoying for those using Visual Studio‘s out of the box.

As an alternative to that I’ve used Gdiplus which is present on most of the systems since Windows 2000, and most shipped with Visual Studios Platforms SDKs. This method is also much simpler than WIC or GDI method from described article.

Alternative solution to WIC is to use pure Vista GDI (UxTheme) calls as described at http://msdn.microsoft.com/en-us/library/bb757020.aspx?s=6 (GDI_CVistaMenuApp.cpp sample). It was implemented in TortoiseSVN revision 14191 by Steve King. It uses BeginBufferedPaint, EndBufferedPaint and GetBufferedPaintBits dynamically loaded from Vista‘s UXTHEME.DLL and Create32BitHBITMAP and ConvertBufferToPARGB32 from GDI_CVistaMenuApp.cpp sample.

The PARGB32 function is as follows:

HBITMAP CShellExt::IconToBitmapPARGB32(std::string sIcon)
{
    HRESULT hr = E_OUTOFMEMORY;
    HBITMAP hBmp = NULL;

    HICON hIcon = (HICON)LoadImageA(g_hResInst, sIcon.c_str(), IMAGE_ICON, 16, 16, LR_DEFAULTCOLOR);
    if(!hIcon)
        return NULL;

    SIZE sizIcon;
    sizIcon.cx = GetSystemMetrics(SM_CXSMICON);
    sizIcon.cy = GetSystemMetrics(SM_CYSMICON);

    RECT rcIcon;
    SetRect(&rcIcon, 0, 0, sizIcon.cx, sizIcon.cy);

    HDC hdcDest = CreateCompatibleDC(NULL);
    if(hdcDest) {
        hr = Create32BitHBITMAP(hdcDest, &sizIcon, NULL, &hbmp);
        if(SUCCEEDED(hr)) {
            hr = E_FAIL;

            HBITMAP hbmpOld = (HBITMAP)SelectObject(hdcDest, hbmp);
            if(hbmpOld) {
                BLENDFUNCTION bfAlpha = { AC_SRC_OVER, 0, 255, AC_SRC_ALPHA };
                BP_PAINTPARAMS paintParams = {0};
                paintParams.cbSize = sizeof(paintParams);
                paintParams.dwFlags = BPPF_ERASE;
                paintParams.pBlendFunction = &bfAlpha;

                HDC hdcBuffer;
                HPAINTBUFFER hPaintBuffer = pfnBeginBufferedPaint(hdcDest, &rcIcon, BPBF_DIB, &paintParams, &hdcBuffer);
                if(hPaintBuffer) {
                    if(DrawIconEx(hdcBuffer, 0, 0, hIcon, sizIcon.cx, sizIcon.cy, 0, NULL, DI_NORMAL)) {
                        // If icon did not have an alpha channel, we need to convert buffer to PARGB.
                        hr = ConvertBufferToPARGB32(hPaintBuffer, hdcDest, hIcon, sizIcon);
                    }

                    // This will write the buffer contents to the destination bitmap.
                    pfnEndBufferedPaint(hPaintBuffer, TRUE);
                }
                SelectObject(hdcDest, hbmpOld);
            }
        }
        DeleteDC(hdcDest);
    }

    DestroyIcon(hIcon);
    if(SUCCEEDED(hr)) {
        return hBmp;
    }
    DeleteObject(hBmp);
    return NULL;
}

So in this case instead menuiteminfo.hbmpItem = HBMMENU_CALLBACK we do menuiteminfo.hbmpItem = IconToBitmapPARGB32(lpszIconResourceID). We shouldn’t forget of initializing Gdiplus library with GdiplusStartup(&m_gdipToken, &gdiplusStartupInput, NULL) in program/DLL initialization code and shutting it down after all with GdiplusShutdown(m_gdipToken).

If we want to be compatible with older Windows versions, we shall load (map) the Vista‘s UXTHEME.DLL functions dynamically only on Vista:

typedef DWORD ARGB;
typedef HRESULT (WINAPI *FN_GetBufferedPaintBits) (HPAINTBUFFER hBufferedPaint, RGBQUAD **ppbBuffer, int *pcxRow);
typedef HPAINTBUFFER (WINAPI *FN_BeginBufferedPaint) (HDC hdcTarget, const RECT *prcTarget, BP_BUFFERFORMAT dwFormat, BP_PAINTPARAMS *pPaintParams, HDC *phdc);
typedef HRESULT (WINAPI *FN_EndBufferedPaint) (HPAINTBUFFER hBufferedPaint, BOOL fUpdateTarget);
/* (...) */
HMODULE hUxTheme = ::GetModuleHandle (_T("UXTHEME.DLL"));
pfnGetBufferedPaintBits = (FN_GetBufferedPaintBits)::GetProcAddress(hUxTheme, "GetBufferedPaintBits");
pfnBeginBufferedPaint = (FN_BeginBufferedPaint)::GetProcAddress(hUxTheme, "BeginBufferedPaint");
pfnEndBufferedPaint = (FN_EndBufferedPaint)::GetProcAddress(hUxTheme, "EndBufferedPaint");

Since we are going to use Gdiplus only on Vista, we may use Gdiplus.dll as delayed load DLL, so it won’t be loaded on older systems using previous methods, saving us some memory. Simple enough ?

Testing Windows version number with GetVersionEx and combining this method for Vista with HBMENU_CALLBACK method for Windows XP and older systems (with hbmp(Un)checked fallback on explorer extension on Windows 2000 if needed) is my opinion best method of having nice menus in all modern Windows systems. This is also current display method of Tortoise CVS & Tortoise SVN latest development versions. If you need full code to browse you may want look into Tortoise SVN SVN trunk files: srcTortoiseShellContextMenu.cpp, srcTortoiseShellShellExt.cpp and srcTortoiseShellShellExt.h.

Conclustion

All those hacks and recipes would be worthless if only there was simple consistent API for making menu item icons. Unfortunately menu icons, something that was always present in Windows and Microsoft applications, never got any decent API, moreover the methods to get those icons working change for every major Windows release, making us developers wasting our time “porting” our applications to new “shinny” Windows rather than doing something productive.

One thing that is simply unacceptable for me (even more since I now work regularly on OSX) is that Windows system apps and Microsoft regular applications are using so many UI hacks and mods that are never exposed to the developers trough API. Those are either closed libraries like one for Office’s or Visual Studio GUI, or Vista hacks with PARGB that require tricky in memory conversions rather than just pointing hbmpItem to HICON and making Vista to do the conversion on its own.