Tuesday, October 19, 2010

This aren't the benchmarks you're looking for


I was evaluating use of GObject for small plentiful short-lived objects in libgck. I wanted to see how their performance compared to custom reference counted structures. Turns out it's not as bad as I imagined.

The speed difference on my system, with a simple test program, ended up being around a factor of eight. Most of that cost is due to pthread_mutex_lock and pthread_mutex_unlock.
$ ./test-gobject-speed 
struct x 10000000 = 1.091700
object x 10000000 = 8.578848
Note that I didn't use heavy weight stuff like properties or signals in my benchmark. But I don't need those for this use case.

6 comments:

  1. A couple of tips:

    * Compile with -DG_DISABLE_CAST_CHECKS
    * Don't use G_TYPE_INSTANCE_GET_PRIVATE if not needed (just use a direct pointer)
    * If you don't need signaling/refcounting (or want to do it yourself), but still want class hierarchies... use GTypeInstance instead of GObject.
    * In your profiling, create one instance of your object outside of the iteration to avoid having the overhead of the initial/final GType class unrefs.

    ReplyDelete
  2. Just for fun, I took your test code and added a C++ class benchmark. For fairness, I used a 'pimpl' struct to get real private data, as GObjects have, and used std::shared_ptr from C++0x for atomic reference counting. I also added one create/unref of the GObject outside the loop, to get rid of the GType intialisation overhead, as bilboed suggested, and compiled with -DGDISABLE_CAST_CHECKS.

    The results on my system were:

    Struct: 2.6s
    GObject: 19.6s
    GObject-like C++ class: 15.0s

    So it looks like GObject isn't actually performing all that badly. Unfortunately, I then recompiled with -O2 and got the following:

    Struct: 1.6s
    GObject: 13.0s
    C++ Class: 2.8s

    ...which looks rather worse. Perhaps the C++ benchmark is "cheating" somehow, but otherwise a 4.5X difference is pretty damning for GObject.

    The modified code is at
    http://tristanb.net/test-gobject-speed.cc
    if anyone wants to have a play with it.

    ReplyDelete
  3. GObject creates 1 million objects per second on my computer. I realize there are faster alternatives, but it's still fast enough for almost all uses.

    ReplyDelete
  4. Tristan, when I try to look at your test-gobject-speed.cc file, I get a (javascript?) error dialog saying "/tmp/r3NOHMPK.cc.part could not be saved, because the source file could not be read."

    ReplyDelete
  5. The test is not correct. You are not only testing the creation time of gobject/struct but also g_strdup("string").

    If you remove those, you see the difference becomes a lot larger.
    (If you want to make GObject look better, just put something taking a lot of time there).

    this is without the strdup.
    struct x 10000000 = 0.550659
    object x 10000000 = 7.721579


    And this is the test (withouth strdup) on some lower end hardware (arm11):
    struct x 10000000 = 24.749612
    object x 10000000 = 339.239949

    ReplyDelete
  6. @Murray: No idea I'm afraid, I've just tried it in both my browser (Chrome) and with wget and it worked okay. It's possible my host went down briefly?

    As far as the topic goes, I remember juergbi of Vala fame saying that the biggest performance issue in the Vala compiler is in the creation of GObjects. Perhaps it would be worth having something along the lines of GstMiniObject (which, IIRC, supports inheritance and interfaces but not properties or signals) in GLib for the case when you want to create lots of short-lived objects?

    ReplyDelete