Rating storage in Canon ImageBrowser EX

I recently had to move a huge collection of photos and videos from a Windows/ZoomBrowser environment to OS X/ImageBrowser. In the translation process, most of the star ratings disappeared. Migrating this information turned out to be quite complex, mainly due to a lack of standardized ways to represent the information, and no official documentation on the topic. The mechanisms are different in different versions of the Canon software, and also between Canon's Windows and OS X software.

There was one particular case where I failed miserably at implementing a migration script: videos. No EXIF, no XMP, no zb4meta.info. As far as I could tell, no file was modified when I changed the ratings. Yet they were somehow persistent. After a closer look at my fs_usage logs, it turned out that the ratings are stored in an SQL database, located under the home directory of the current user in an SQLite3 file:

~/Library/Application Support/CANON INC/ImageBrowser EX/IBXImg.db

The relevant table is SKA_COMMON_BASE_TBL. If you are not familiar with the SQLite3 APIs and would prefer to work with the data in text format, it is easy to export the database to a text file:

$ sqlite3 IBXImg.db .dump > dump.txt

For example, in this entry the rating is 5, stored directly after the path:

INSERT INTO "SKA_COMMON_BASE_TBL" VALUES( ... 'MVI_7761.AVI','/Users/gunnar/...',5, ... );

Note that JPG still images also have their metadata stored in the database. However, in that case the data is stored in the JPG file itself, too. If you change the rating in the ImageBrowser GUI, the file is updated, not only the database. I assume that changing the rating externally (e.g. with exiftool) after ImageBrowser's initial scan would cause ImageBrowser to update the database, although I haven't tested explicitly.

Generally, I am surprised how much trouble it was to migrate all the metadata. Not only the ratings, but also e.g. Picasa face information was problematic. Imagine if you couldn't move your Word documents to a different computer, because some of its data would reside in an undocumented SQL database, or an undocumented XML file somewhere on the file system. How does the typical photography hobbyist handle this situation? Does everyone write their own little Perl script? Or transfer the metadata manually?

An open, generally accepted standard for this purpose would be good, but I guess that is a few years away. In the meanwhile, though, it would be nice if all vendors publicly documented their metadata storage system. Support for XMP ratings in Picasa would be great, too, as the current 'star/no star' scheme is inadequate (not to mention 'vendor-specific').

Note: The information above is relevant for ImageBrowser EX 1.1.0.17.


Update: I found the scalability of ImageBrowser insufficient for my dataset (about 25k photos/videos). Now I am looking into iPhoto, which also uses an SQLite3 database to store the ratings. The database schema is a bit tricker, as it uses a larger number of tables and has a versioning concept. Seems you can find a 'masterId' from the table RKMaster (in the Library DB) given the path of the file, and from there you can find all versions of the image/video in the RKVersion table. The ratings are stored per RKVersion entry.

I'm a little nervous about manipulating that database, as it contains a fair number of hooks, indices, and cross-references. However, I'm drifting towards the conclusion that I should keep the master copy of the metadata in text files anyway, and write import/export scripts for whichever photo browser I want to use at any given time. Perhaps in a few years the standard tools will be more mature. Today, they are too fragile, vendor-specific, and undocumented for my taste.


Update 2: I modified the iPhoto database as described above, and it worked as expected. I did have to use the 'Repair database' feature afterwards to get Smart Albums to work correctly, though. I suspect that iPhoto keeps an application-maintained index in the database, which was not updated by my code.


Last modified Dec 9th, 2012. Mail any comments/corrections to gunnar.gunnarsson at abc.se.

Back