implementing ManagedObject or Serializable, and updating those objects

If you've looked at the Darkstar APIs, you're familiar with the notion of a Managed Object. These are the objects that get persisted for you in the data store. Implement the ManagedObject interface, and we'll take care of making that object durable, handle contention around it, etc. Real nice. The only requirement is that these objects must also implement Serializable, since we use Java serialization to actually store your objects.

A common pattern that you'll see in the Darkstar APIs is the ability to use a class that must implement Serializable but may optionally implement ManagedObject. What may not be obvious is why this pattern is used or what effect there is on your application code when you make one of these two implementation choices. Since I hear questions about this pretty regularly, I thought I'd try to lay out what to my mind are the two key issues involved here.

To make this a little more clear, consider the following example. The TaskManager can be used to schedule tasks:

  Task myTask = new MyTask();

In this case, myTask is an instance of MyTask, which in turn must implement the Task interface. All implementations of Task must also implement Serializable:

  public static class MyTask implements Task, Serializable { /\* ... \*/ }

According to the javadocs, implementations of Task may also choose to implement ManagedObject. Several other classes (like ChannelManager) use this same pattern. Implementing ManagedObject won't change the methods that must be implemented, but it does have a few effects. Specifically, there are two big differences from the Application code's point of view.

The first, and perhaps more obvious difference (perhaps) has to do with handling object removal, and this is why I usually suggest that Task be implemented as above. If you only implement Serializable, then Darkstar will take care of managing this object in the data store until the task runs. This also means that we'll take of removing the object once the task has finished running. So, you as an application developer can schedule this instance and then pretty much forget about it. If, however, you said

  public static class MyTask implements Task, ManagedObject, Serializable { /\* ... \*/ }

then Darkstar assumes the application has already managed this object. We also assume, therefore, that the application code will take care of removing the object. In other words, even after the task has committed, the task object will remain in the data store until your application code explicitly removes the task instance.

Sometimes this is exactly what you want. If, for instance, you have some shared object that maintains state and it's convenient to schedule this object occasionally to do some work based on that state, you wouldn't want Darkstar to remove the object when the task completes. In this case, making your class implement ManagedObject means that you can keep a ManagedReference to it, and work with that object as long as you like. It also means, however, that if you don't keep a reference to the object, that it will continue to be persisted and you'll have no way to remove the object.

So, to summarize, if you implement Serializable only, we'll take care of managing the object and removing the object when it's no longer needed. If you also implement ManagedObject, then make sure that you have a ManagedReferece (or name binding) to the object, so that you can explicitly remove the object when you no longer need it. In other words:

  public static class MyTask implements Task, Serializable { /\* ... \*/ }
  public static class MyMOTask implements Task, ManagedObject, Serializable { /\* ... \*/ }
  // ...
      // this is ok:
      AppContext.getTaskManager().scheduleTask(new MyTask());
      // this is not ok, since the object will never get removed:
      AppContext.getTaskManager().scheduleTask(new MyMOTask());

Cool so far?

I said that there are really two issues to consider here. The first is how objects are managed and removed. The second has to do with managing updates.

Recall that to get an object from the DataManager you can either do a get() or a getForUpdate(). Even after you get() an object, you can always call markForUpdate(). As the docs say, the point of "marking" an object for update is to tell Darkstar that you're going to be modifying the state of the object. This can have some significant performance implications; more on this in a minute.

Now, if you look at the docs, you'll see that application code can only "mark" an instance of ManagedObject. It cannot "mark" an object if that object only implements the Serializable interface. Fair enough, since the DataManager interface is used to manage implementations of ManagedObject. In other words, you can't ask the DataManager to manage an object that doesn't implement ManagedObject.

What does this mean for the examples we've been looking at? Essentially, it means that any time you schedule a Task (or register a ChannelListener or ClientSessionListener, etc.), Darkstar is effectively wrapping your object in something that does implement ManagedObject so that the object can be persisted in the data store. We'll take care of creating and removing these wrappers as needed, and these wrappers in turn will either have a direct reference to the object (if it only implements Serializable) or a ManagedReference (if the object implements ManagedObject). What Darkstar won't do for you is "mark" this wrapper for update.

So, going back to the original example, if your implementation of Task only implements the Serializable interface, you will never be able to "mark" your object for update. This is because the actual ManagedObject that you would have to "mark" is a wrapper object that is internal to the TaskManager, ChannelManager, or some other system component. This is why you'll see language in the javadocs suggesting that if your Task (or ChannelListener, etc.) has mutable state, you should always implement ManagedObject so that you can "mark" your object if you are changing its state. In other words:

  // state will never change, so this never needs to be marked for update
  public static class MyTask implements Task, Serializable {
      private final int someValue;
      /\* ... \*/
  // state can change, so you'll want to mark the object when modifying it
  public static class MyMOTask implements Task, ManagedObject, Serializable {
      private int someChangingValue;
      /\* ... \*/

"But wait," you may be thinking, "I thought we didn't have to mark our objects for update and Darkstar would just magically take care of things." Yes, that's true. Darkstar does implement modification checking. That is, at the end of a transaction, we can (and by default, do) check every accessed object to see if it's been modified. So, strictly speaking, you don't need to mark an object each time you modify it. That said, there are some very good reasons why you should mark all updates, and therefore why you should follow the pattern I've illustrated above. Let me try to convince you.

Like the docs say, marking for update is all about performance. Whenever an object is modified in one transaction, it may cause conflict with other transactions. The sooner that Darkstar knows about an update, the sooner it can start tracking this possible conflict, or decide that the calling transaction will have to be aborted and re-tried. If you mark your updates, then this will help optimize your Darkstar application; if you don't mark updates, then we have to wait until the transaction is ready to commit, and only then can we figure out what modifications happened and therefore decide if there was conflict. At the simplest level, this may mean that your transaction does a lot of work that may be aborted, but that could have been avoided if we'd known up-front about the update. There are lots of other neat tricks that we already do or are looking at doing with this information, but these are all optimizations that won't happen unless you mark your updates.

There's another performance issue here, however, and it has to do with how we do the modification checking. Remember way back at the start of this entry I said that all objects must implement Serializable because this is how we persist your data? Well, since we're using Java serialization anyway, this is also how we check for modifications. That is, for every object that hasn't been marked by the end of a transaction, we do partial serialization of the object to see if its state has been modified. So, even if you don't update an object, you'll still pay a sight serialization cost. It's not terrible, but it can definitely add up, especially if you access a lot of objects or have objects with large amounts of data.

What can you do about this? Well, if you're marking all of your updates correctly, you can actually turn off this modification checking. Obviously you need to be pretty careful about doing this. If you are actually modifying some objects that you don't mark correctly, you won't end up committing your changes. So, before you even consider turning off modification detection, you should see if your code is marking updates correctly. Luckily, there's a logger that will tell you just this. In your logging properties, set: = ALL

This will continue to run with modification detection enabled, and will tell you whenever a modification was detected for an object that wasn't marked for update. Obviously a compile-time tool for complete code coverage would be better, but hey, we're working as fast as we can :) For the time being, if you set this logger and run your code through vigorous tests (you do have good test coverage for your game, right?) you should be able to see which updates, if any, you're missing.

If you're feeling confident that you're correctly marking all updates, then you can try turning off modification detection altogether. This is done with a property setting:

This has the effect of only serializing and committing the objects that your application code has actually marked for update. Obviously the overall performance impact will vary depending on the game, but in practice we've found that this can make a noticeable difference. Try looking at profiling data and see for yourself if this helps with the runtime of your code.

Coming back to the original topic of this post, you won't be able to turn off modification checking unless you mark your updates correctly, and you can't do this unless your objects implement ManagedObject. So, if you have an object with mutable state (i.e., something where you'll be changing the value of some member variable), you really should be implementing ManagedObject and you should get in the habit of marking those objects each time they're updating. If your object's state doesn't change, then you may not need to implement ManagedObject. You'll have to figure out whether you need to keep the object around for an arbitrary amount of time, or keep ManagedReferences to it. If not, then it's probably easier to only implement Serializable and let Darkstar take care of managing and removing the object.

Like I said, the choice of whether or not to use ManagedObject may not be immediately obvious just by looking at the javadocs. There is some serious subtly here, but ultimately I think the few rules I've sketched out should make it fairly simple to figure out which approach to take. I hope this makes it a little easier when you're writing your games, and when you're trying to figure out how to use this pattern!


hi thanks for the good articles on your blog about darkstar, is there a way to remove the manage objects from the data store as well so that we can free up the resources..

i m not sure but does darkstar provides the way to delete the persisted manage objects as well.

Posted by puran on November 22, 2008 at 12:01 PM EST #

Post a Comment:
  • HTML Syntax: NOT allowed



« September 2016