Skip to content

10-100 times faster thumbnail loading

Saturday, 31 July 2010  |  blackie

Fast - tell me how do you implement a thumbnail loader - well the loading and storing of the thumbnails that is?

Classically you would likely say something that included individual JPG files for each thumbnails, and perhaps also something that included multithreading.

Well that is at least how many application does it, and Linux even have a standard for it, so applications can share the thumbnails.

I've implemented a much faster way to do it. This is something like 10-100 times faster, and I have code to prove it :-)

The Short Story
Store all the thumbnails in one big file, which is way faster to navigate for the hard disk, than seeking around just to load a few kilo bytes here and there.

The slightly longer story

First of all, I didn't come up with this idea, so kudos to whoever introduced it to me many years ago - maybe someone planted the idea in my mind while I was sleeping :-)

I'm no where close to being a hardware kind of guy, so I wont bother you with the details or statistics, but the fact is that the hard disk waste valuable milliseconds when it has to seek to a new location to load data. Just try to create an application which loads dummy data from a bunch of different files, and measure how much your application loads a second. Now compare that to an application that loads dummy data from one big file instead.

When you want to see the thumbnails in say a photo application like KPhotoAlbum, then you are very likely to load them in a specific sequence, so if you could get them located on disk in that sequence then the disk would not need to skip around when loading.

I, at least haven't found a way to tell my Linux how to store things in a specific order on disk, so the alternative is to store all the thumbnails in one big file. That at least from my test seems to be way less fragmented.

Try it out
OK, don't just trust what I say, try it. Download the demo from http://www.kphotoalbum.org/data/ThumbnailLoader.tgz

compile it using: qmake; make

run it like this: ThumbnailLoader new /path/to/the/rootdirectory/of/all/you/images

you only need to specify the path the first time you run the demo.

To compare this to the old way of loading thumbnails, you may run the demo like: ThumbnailLoader old.

To avoid being cheated by your disk cache, make sure you load in a lot of images, and that you scroll a lot around.

Cheers Jesper.