Note: While Sikuli is Free Software (under the MIT license), Mac OS X is composed mostly of non-free, proprietary software.
Sikuli (Project Sikuli) is some very cool technology from MIT that combines a simple programming framework with AI for image recognition. Sikuli can be used to automate tasks in a manner similar to Selenium. The difference between the two is Selenium is strictly geared towards the web, while Sikuli is a general solution that uses image recognition (as opposed to HTML ids, names or XPath) to identify items and can therefore be used for much more than just the web (i.e. the desktop).
You’re running Sikuli to test applications on a headless Mac OS X (10.6) machine. The machine does not have a monitor but a graphical server does exist because you remotely administer the machine using VNC.
Running Sikuli on the headless machine you get the warnings:
no full-screen pixel format
Followed by an error when the item (image) you’re looking for is not found/recognised, even though it does exist on screen.
Add a physical monitor to your Mac machine to fix the issue. You can still administer the machine remotely using VNC including running Sikuli jobs. Having the monitor plugged in appears to tell Mac OS X to use hardware/render OpenGL. See the Wikipedia article on VirtualGL for more information.