[Feedback Needed] Advanced Message Bus for Unity

*** EDIT: I’ve decided to release this with full source for anyone who wants it

Message Bus With Source - FREE DOWNLOAD

I’m still working on some sample projects but haven’t had the time to complete them yet. In the meantime I’ve decided to offer up the message bus free of charge, both the compiled binaries and the full source code (Visual Studio 2010 / 2012 projects / solution) with unit tests. The solution should also open fine in MonoDevelop.

Two things to note:

  1. When opening the source in VS / MonoDevelop - you will need to re-add the reference to UnityEngine.dll to the DH.Messaging.Extensions project. I wasn’t sure what the redistribution policies were for UnityEngine.dll so I did not include it in the zip file.

  2. Currently no serialization attributes exist for any of the MessageBus classes. If you needed to serialize messaging you’d need to implement that in the provided source and recompile.

Here’s the link to download:
http://unity.dustinhorne.com/downloads/messagebusv1.zip

and here’s the technical documentation link again:
http://unity.dustinhorne.com/documentation/messaging

Feel free to PM me with any questions or if you need help setting it up.

*** ORIGINAL POST ***

Hello team! I’m working on a new asset that I’m going to be submitting to the Asset Store and I would like to solicit two things from you guys and below these questions is information about the asset:

  1. How much should it cost? Free isn’t an option at the moment but I don’t want to gouge people either.

  2. What do you think of the Asset and Implementation? This is really the important question. I need to know if you think the features are useful and what if any additional features you’d like to see.

Documentation (link is also at the end of the post)
http://unity.dustinhorne.com/documentation/messaging/

What Is It?
It’s a messaging system for Unity3D. There are two main pieces in the system. The first is a first-come-first-serve message pool. The second is a message bus that objects can subscribe to. It’s written in C# (Visual Studio 2012) and compiled to a single assembly. It works with both Unity Free and Unity Pro. The code has already been written and pretty heavily tested, but may change based on feedback here.

The Message Pool
The message pool is meant for registering simple messages and retrieving them with your components. The first component to grab the message removes it from the pool. This is useful if you want to just do something once for whoever might look for the message. As an example, you could have a piece of code that calls the static manager to add a message:

PoolManager.AddPoolMessage("SomeMessageKey", "I have arrived!");

You can add as many messages as you want to each key. Components looking for these messages will just grab them one at a time in the order they were added. Example:

string myMessage = PoolManager.GetNextMessage("SomeMessageKey");

The only requirement is that the message pool exists, which is done by calling:

//Create the pool with the default max size
PoolManager.CreatePool("SomeMessageKey");

//Or create the pool with a max message size
//Old messages will be truncated automatically to keep the pool size down.
PoolMaanger.CreatePool("SomeOtherKey", 500);

The Message Bus
The message bus itself is much more robust. It utilizes a subscriber model allowing individual components to subscribe to messages of a specific type and key. When messages are added to the message bus manager it looks for subscribers of that specific type and key combination and delivers the message to all subscribers. The “Message Key” is a way to allow multiple messages to be of the same type but have different meaning. Every “Type” specified for a message creates an internal message bus that handles all messages within it.

To work with the message bus, I created a component (MonoBehavior) that can be attached to any GameObject. This component (MessageBusSubscriber) handles all of the subscriptions for that game object. I also added some game object extension methods allowing you to create a default empty subscriber or you can subscribe to a specific message type. If you subscribe and the subscriber component doesn’t exist, one is automatically created for you. The subscription is automatically registered with the message bus and will start receiving messages. Here’s an example:

var subscriber = gameObject.SubscribeToMessage<Vector3>("PointOfInterest");

This code will automatically call AddComponent on the game object if the component doesn’t already exist. Furthermore, it subscribes to any message with the type of Vector3 and the key of “PointOfInterest”. The extension methods all return the MessageBusSubscriber instance (either existing or newly added) so you can perform additional options. To retrieve the MessageBusSubscriber you can call:

var subscriber = gameObject.GetComponent<MessageBusSubscriber>()

or use the extension method:

var subscriber = gameObject.GetSubscriber();

Now, we’ve subscribed to the Vector3 message with key of “PointOfInterest” above, but we want to set some additional properties on our subscriber:

var subscription = subscriber.GetSubscription<Vector3>();

Importantly, all Vector3 type messages are handled by the same subscription, regardless of the key. The subscriber let’s us configure when messages are processed (Update, FixedUpdate or LateUpdate). The MessageBusSubscriber component handles processing the messages at the appropriate time and will process all messages in an active subscription in a frame. If you’ve subscribed to multiple keys, multiple messages will be received and processed. By default, messages are processed during “Update” but we can now say:

subscription.UpdateType = UpdateType.FixedUpdate;

There are several options that can be configured. Now, to send a message, from any code we’re executing you can call:

MessageBusManager.AddMessage<Vector3>("PointOfInterest", new Vector3(1, 3, 3));

That message will be delivered to every subscriber of that type and key. Every subscriber will receive the Vector3 we supplied. Messages are generic and you can register and send any type of message you want.

On the subscription side, every “subscription” (not subscriber) has an OnMessageReceived event. This event is fired for every message that was processed during the given frame. The event gets two arguments… the a reference to the MessageBusSubscriber instance and a MessageReceivedEventArgs object. The arguments has (string) MessageKey and (T) Message properties.

You setup an event handler for every “Type” of message (not every key), then based on the key you can perform different actions from your handler. In this case your event handler will handle MessageReceivedEventArgs and the args variable will contain the “PointOfInterest” key and the Vector3 we supplied.

Why is this useful?
Being component driven, Unity allows the objects and components to access eachother in a variety of ways, such as finding objects by tag name and finding objects by id or just directly referencing objects. This system allows you to broadcast messages that will be received by anyone subscribing to that message regardless of tags or type of object. You can put your event handler logic in a prefab or script to attach. Here’s a good example:

Let’s say you have a crowd of characters. You want all characters to react to certain events. If it’s an Explosion, you want them to run away… if it’s a loud noise you want them to turn and look. The only thing they need to know is what the event was and what the location was. So, you create a script that does this:

var subscriber = gameObject.SubscribeToMessage<Vector3>("Explosion");
subscriber.SubscribeToMessage<Vector3>("LoudNoise");

var subscription = subscriber.GetSubscription<Vector3>();

subscription.OnMessageReceived += ReactToNoise;

and we setup our handler

public void ReactToNoise(MessageBusSubscriber s, MessageReceivedEventArgs<Vector3> args)
{
    switch args.MessageKey
    {
          case "Explosion":
               RunAway(args.Message);
               break;
          default:
               LookToward(args.Message);
               break;

    }
}

Ok… this has gotten to be a much longer post than I anticipated. So, I’ve put together some preliminary technical documentation of the library. I’d love to hear what people think and what questions they have. I may also open it up for some very limited beta testing before I submit it.

Here’s the documentation:
http://unity.dustinhorne.com/documentation/messaging/

Hi,

Your system seem most evolved that an simple Notification Center. I like the ‘Bus’ mode :wink:
For 15-20$, i’m in.

Awesome, thanks for the feedback! There’s much more to it than I could post in a simple forum. I plan to release the full documentation with the assembly and also provide instructionals on my blog. I want to keep it simple to use but I think I’m limiting functionality as well so I’m going to make some changes. Let me know what you think of this:

  1. Currently the MessageBusSubscriber is required to use the message bus, but I want to open it up for any script to be able to use it so I’m going to expose more of the internal members of the MessageBusManager and a more robust implementation of the MessageSubscription. In this way, you could use your own subscription manager to process messages rather than having to rely on the MessageBusManager. You could process messages then at your leisure instead of waiting for the subscriber to do it.

  2. Optionally enable / disable event based notifications. Right now, the event is fired for every message when ProcessMessages (an internal method) is called. The drawback to this is that you’re tied to that event and have to process all messages. Instead, I want to expose the methods to get the messages manually and disable individual event notifications so you can process messages as you like.

  3. MaxToProcess (or something similar) setting - When using the default message processing event, set the maximum number of events you want to process per frame for that individual subscription. This way if you have anomaly frames that have a high volume of messages, you can spread that message processing across multiple frames by not processing all of them at once.

  4. Add ProcessingMode setting. By setting it to ProcessingMode.Manual it will bypass all events and allow you to manually process messages. By setting it to ProcessingMode.Default it will use the default message processing of processing all per frame. And by setting ProcessingMode.OnDemand, you can have the event fire immediately when messages are received instead of waiting for the next frame to process all messages.

  5. Add Bulk Message processing - Add the ability to subscribe to a bulk message event (also affected by the MaxToProcess setting) which will allow you to attach a separate event and process messages in bulk. This means the event will only fire once and a BulkMessageReceivedEventArgs object will be passed to the event which contains an IEnumerable<MessageReceivedEventArgs> property so you can access the key and Message all at once.

  6. I also want to do some cleanup to the way subscriptions work. Right now there is a lot of automatic removal of Subscription objects that gets done when using the MessageBusSubscriber but there are some things that do not do any cleanup, such as clearing all subscriptions from the MessageBusManager. The Subscriptions are unsubscribed, but they are not removed from the MessageBusSubscriber in some cases. However, I want the subscriber to manager its own subscription objects as that’s not the responsibility of the Manager. So, I’ll be adding a property to the Subscription (internally settable by the Manager) to signify that the subscription is no longer attached to any message bus. This way the Subscriber can remove those if necessary (also going to add a setting for automatic cleanup) and if you use the Subscriptions in your own classes, you can decide what to do based on whether or not it is subscribed to any bus.

  7. Enable extensibility - The core classes are sealed in my current implementation, but since I’d like to expose the ability to wrap the messaging in your own code, I’d like to also make some items extensible, such as the message subscriptions, so you can add more functionality as needed.

  8. And finally, move the GameObject extensions to a separate assembly. The GameObject extensions are nice (I like using extension methods) in that they simplify the work of adding the MessageBusSubscriber to your GameObject, but some folks may not want to muddle up the member list for gameObject or may find that it conflicts with their own, so I’ll move the extension methods to a separate (but included) assembly so they can be used optionally as they are not a core requirement for the MessageBus to work. You can even drag and drop the Subscriber to your game object in the GUI (there’s no requirement for creating via code).

Let me know if there’s anything you think I’ve missed.

Thanks!

Dustin

3 points of concern:

it seems one can only use string as key? thats a killer for refactoring/obfuscation. i currently use a messaging system from the wiki (before the demonish licence change!) and have it changed to an enum. this way you also keep out typos and add intellisense for your messages.

does it only work with monobehaviors/gameobjects? i have most stuff run in the background on non-mb classes and only a small part of the “world” is displayed to the user as gameobjects. if its limited to those you limit its useablity.

on purchase we only get the assembly but no source code? thats also a nogo. one can get the code with reflector but its inconvenient. is there a certain reason for this?

other than that your package seems like a mature solution.

What advantage does the message bus give over using events and/or delegates?

@Jaimi -

This depends on how you utilize it. I’m going to reply to exiguous separately below which should give you a little more insight, but the basic premise is that it doesn’t lock you into any specific way to use it (other than the string based keys which may change). I’ve made some changes already that allow it to be used with or without a MonoBehavior. In fact, the Message Bus itself no longer has any dependencies on Unity (moved to a separate assembly). It has an option event based notification that will allow you to handle received messages in an on-demand fashion. If you use the Extensions you can use the MessageBusSubscriber which now allows you to process the messages in bulk during the Update, FixedUpdate, LateUpdate. It also allows you to process On Demand via an event subscription. Or you can periodically check and process the subscriptions without using the supplied events.

It’s meant to be a flexible system. In fact, multiple pieces of code can subscribe to the same message and handle it differently if desired. You could have another class that subscribes to the messages and processes them on demand as they come in, and you could have a MonoBehavior subscribed to the same message that processes the messages during the FixedUpdate call. Or you could have another MonoBehavior that processes the messages manually without even using the MessageBusSubscriber component. It’s really up to you, but the basic idea is that the MessageBus just acts as a messenger and delivery mechanism for the messages. The MessageBusSubscriber component is there to provide some abstraction and make implementation in game objects simpler, but isn’t required so how the message is handled after it’s been received is really up to the recipient.

@exiguous -

I haven’t made any big decisions on what all will be included yet. I actually participate in the open source community and provide a lot of source code. To date I have always kept my “commercial” products and open source projects separate, not supplying the source code. This is my first foray into the Unity community which has an Indie target so it may be reasonable to provide the source code, or provide two versions of the product… a lower priced as-is version and a slightly higher version that includes the source. I haven’t dug much into how other components are provided at this point so I need to do some research but it’s exactly why I asked for feedback here.

For the message key, right now it’s string only. I’m thinking about loosening that requirement but implementing it will also increase the complexity of implementation. For instance, right now if you want to subscribe to a message you make your calls as: Subscribe(string messageKey). If I allow generics to be used for the message key, some calls now get more complicated and have to be in the form of: Subscribe<TKey, TValue>(TKey key).

This is pretty easy for me to pick up as I do backend development in C# for a living however it may start to be overkill for some users of Unity, as there are some (maybe it’s only a few) who don’t have much C# experience and thus want something easier to use. In this case, the simpler the better… however that does an injustice to the power users who would want or require more flexibility such as yourself.

As for what it works with, I did a lot of thinking after I wrote my second post and I was unhappy with the architecture. What I didn’t have was a real clean separate of concerns as to which components had what responsibilities. For this reason I revamped the architecture a bit. It required me to add one extra layer between the MessageBusSubscriber (Monobehavior component) and the MessageSubscription but it was well worth it. I completely decoupled the MessageBus from Unity. The MessageBus project no longer references the UnityEngine.dll. Instead, I created a separate Extensions project which has the game object extensions as well as the MessageBusSubscriber pieces. All configuration related to the MessageBusSubscriber is now in a SubscriberItem that wraps the MessageSubscription.

Now, when the extensions are used, the game object can subscribe to the OnMessageReceived event of the SubscriberItem which uses the following signature:
MessageReceivedEventHandler(MessageBusSubscriber s, MessageReceivedEventArgs args);

Every frame when the messages are processed, that event is fired for every message. Alternately, the game object can set the ProcessingType to ProcessingType.OnDemand which will process the messages immediately as they are added. The SubscriberItem does this by itself subscribing to the OneMessageReceived event of the MessageSubscription. If that event has subscribers, the messages are immediately delivered through that event and bypass the queue altogether. Similarly, if you wanted, you could actually subscribe directly to the OnMessageReceived event of the MessageSubscription and you could receive the messages as they come in directly from the subscription. In this case the MessageBusSubscriber isn’t even needed, however you could still use it to more easily manage and house subscriptions for the individual GameObject. It just wouldn’t be used for processing. The OnMessageReceived event for the MessageSubscription has a nearly identical signature as the event for the SubscriberItem. The only difference is that it supplies a reference to the MessageSubscription object instead of MessageBusSubscriber since it knows nothing about the subscriber. In fact, the MessageBus assembly has no references and knows nothing about the subscriber.

I know for sure I will be putting together some samples which will also be included with the asset. I’ll include some demos into how it could be used as well as some skeleton scripts for managing and subscribing to the events.

As for the solution being mature… I’m getting there. I actually just developed it this week. Total time including development and documentation was about 2 days and I spent the rest of the week testing what I have in place. I’ve also already made the modifications I discussed in this post but I haven’t updated all of my code documentation. I’m going to post updated code documentation online in a few minutes but there will be some inaccuracies in the verbiage in the documentation until I get a chance to update my code comments.

If anyone is curious, I’m actually using the Sandcastle Help File Builder GUI to generate the documentation. If you’re interested in looking at one of my open source projects, I also developed an RSA encryption and signing library for Silverlight and Windows Phone 7 since they didn’t support it (Windows Phone added the support but Silverlight never did).:

http://scrypt.codeplex.com

Or you can check out my blog. Back when I was working with XNA I did a tutorial series on developing some basic heightmap based terrain with dynamic LOD which was an interesting project and actually still gets a lot of traffic.

http://www.dustinhorne.com

I’ve updated the documentation. The descriptions may not all be 100% accurate but the new class structure and separate and exposed methods are shown:

http://unity.dustinhorne.com/documentation/messaging/

it must not necessarily be more complex. as described i have defined an enum which contains an entry for every message i want to send/recive in my game. so you could simply provide an empty enum which is then filled by the user (requires him to have the code) with the messages he needs. this way you can also iterate over the entries when your messaging system is initialized and pregenerate a delegate for each entry. so you could simply provide 2 overloaded methods one with string one with enum. no need for generics here. but the better and more consistent way imo would be to drop strings entirely and only use the enum way. i see no single advantage of using strings as keys.

i see you have an interesting product but in my case i need to see wether the additional features (and complexity) justify the work of exchaning it with my current system. if the price is reasonable and the source included i would also buy it if i don’t directly use it just to learn from it and see what parts could improve my current system.

After digging in, doing some refactoring and looking at the architecture I’ve made some executive decisions. What I evaluated was ease of use vs flexibility. In order to keep the implementation simple I know I won’t be able to accommodate everyone who has existing systems in place but it’s a concession I’m willing to make. The primary driver for not implementing typed keys is the difficulty of implementation. I could implement them but as I expected there isn’t a good way to do it without requiring the they type to be specified on most calls to the library which would complicate things. This is due to the necessary architecture allowing for typed values and subscriptions where are handled with some simple wrappers and internal collections.

That being said, there are workarounds to this. You could, for instance, call the ToString() method of your enumeration when defining your keys and convert it back to your enum type when handling. A more simplistic approach would be to define some public constants in a separate statically accessible script and register / subscribe using your contstants. (i.e. MessageKeys.MyMessageKey) and enforce that consistency through your project.

I could take the route you suggested and provide an empty enum, but this would require extra coding to be done and in my opinion is a message and incomplete implementation. I’m a little OCD about having clean implementations at times so it’s not a route that I want to take. I know this approach won’t work for exiguous, but for others it should be pretty simple.

Once my documentation is complete I’ll push it out in two varieties. The first will be a lower cost variety (price to be determined) which will be the implementation as described. The second will have a higher but reasonable price tag and will include the full source. This should encompass users who are developing on a tight budget and the power users who want to be able to fully customize their solution. As it currently stands, the message subscriptions are inheritable and will still be handled by the message bus, so it’s still pretty simple to expand upon and add additional functionality.

I think most folks are going to already have their own system in place and not be to eager to jump ship to a new one. Making it cheap at first could go a long way to get more people to give it a tryout. Then after everyone has decided it’s the new standard for Unity you could charge whatever you wanted :wink:

How’s performance? Mobile friendly? 3.5 compatible?

Burletech -

I think you’re right. I’m thinking about the $10 to $15 range at most. Performance just depends on how you use it. I’ve worked to keep it as simple as possible and stick to some basic types underneath so it should be mobile friendly. It does rely on .NET 3.5 so I have to take a look at see what versions of Unity that encompasses but I believe it’s 4+. That being said, I haven’t actually used anything that’s not available in .NET 2.0 so once if it’s not 3+ compatible I may go back and recompile to a lower framework target.

Internally it essentially uses some generic dictionaries and Queue to store the messages. The messages themselves are a struct (Message). This was something I went back and forth on, but I settled on the Struct for performance purposes. This will help keep heap size down for any systems that don’t support generational garbage collection. Still, if you’re using reference types within your message (i.e. you have a Message you’re going to be shoveling stuff on the heap, but in most cases I think people will be passing around items like vectors.

I’m in the process of writing some sample scenes for it now. I’m scripting some use cases for it to demonstrate how it works. I also want to create a stress test scene for it that will allow it to be tested with a large volume of messages and objects. This will all be included with the Asset.

I don’t expect many will want to jump ship to the Asset if they have their own working solution unless they find it compelling enough, but that’s ok. I’m not looking to make a living off of the Asset store. :slight_smile: I just want to make my work available to the community and maybe recoupe a little for the time investment I’ve put into creating it :).

One quick question for anyone who has created assets… I’m hoping to build more Assets in the future so I’m going to build a new website to house my documentation and in depth sample code. But I’m also considering including the documentation as a .chm file with my Asset. I’ll always have the documentation online, but is .chm within an Asset package acceptable for folks who may want online documentation or should I just provide it as a separate download from my site?

Thanks,

Dustin

Just a quick update. I did some testing on a fairly beefy machine so I’ll need to test on some others as well, but I created a script and attached it to a game object that moves when you press W, A, S, D. On every single Update frame, it sends a “LookHere” message through the message bus with it’s transform:

MessageBusSubscriber.AddMessage(“LookHere”, transform);

Then I created a cylinder that subscribes to the message and attaches an OnMessageReceived event. I duplicated until I had 1,400 cylinders. The cylinders don’t do anything, but they subscribe to the event so the method fires. With 1400 cylinders, there was no drop in framerate, so the good news is, it works quickly and efficiently. To make sure the events were being received, I added transform.LookAt(args.Message, transform.up); to the event handler. As expected, as the camera flys around, the cylinders all turn to face the camera. There was a framerate drop at this point but it was all do to the calling the LookAt method on so many objects on every Update.

The problem I see you facing is that the people who are interested in this are likely to be more advanced users. Users who can write their own solution pretty easily. The only clear benefit you have described over built-in events is the ability to defer processing. This is a nice feature but the question your user base is going to be asking is will it be quicker to write my own solution than it will be to learn someone else’s.

All I really need to do is use c# events, add each work item (message) to a queue in my listener, and then process that queue in the desired method (Update, FixedUpdate, etc). Finally I might need to add some logic for discarding duplicates or discarding messages if the queue gets too big (does your system cater for this?).

This doesn’t mean your solution isn’t better, more powerful, more flexible, more robust, etc. But if I’m working on a project, I only care about the behavior I need.

One thing you might consider is integrating with popular asset store packages either generically (for example hooking in to PlayMaker), or to solve specific problems.

Thanks Johnny, appreciate the feedback and definitely see your point. My real target is intermediate users; those that are stronger on the graphics and game development side but not as strong on implementing abstractions. I’ve kept it simple enough for basic users but complex enough that it could suit anyone, but you’re right, most advanced users will just build a solution tailored to their specific needs. In fact, if you have the time and are an experienced coder you could implement something specific to your needs and it would be a little lighter weight.

Mine is a little more niche and geared more toward speed of implementation which is also why I wanted to keep it as simple as possible to leverage. In fact, you’ve described pretty much what I have. I’ve created listeners that contain message queues and a manager to distribute messages to all subscribed queues. In essence it is really simple. There is no de-duplication logic, but there is enforcement for queue size which performs automatic purge of old messages and it’s configurable on a per-subscriber basis.

It is also based on the standard event system. You can handle a message received event and process messages immediately, or you can wait and process them as desired. This is exactly how the extensions work. The MessageBusSubscriber attaches a component to further abstract away the subscriptions for you and by default will bulk process the messages in the Update method, though you could still process them on demand.

A couple of other benefits to using the extensions is that it will automatically activate and deactivate subscriptions by default when the game object it’s attached to gets deactivated and activated. This is configurable so you can have the subscriptions remain active. If the processing mode is set to Bulk (where it’s processed during one of the Update methods), then the messages won’t be processed but the subscription will still receive messages. If you’re doing OnDemand processing and you leave the Subscription active, it will still receive messages and fire the message received event, but if you auto-deactivate it, it will stay subscribed but not receive messages until it’s activated again.

You can also auto-clear messages on deactivation of the game object. And, subscriptions are automatically removed when the game object is destroyed. These are all benefits of using the MessageBusSubscriber, but in reality it’s a component that’s really meant to make it easier for more basic users to consume the bus.

All of the feedback I’ve gotten here has been good and constructive. I’m really impressed at how active the Unity community is.

Yes, by no means trying to downplay your effort, sounds like a great package.

I’m just wondering how you can best sell it!

EDIT: One thing I’ve found is that demo videos really help sell things here. So maybe think of some compelling use cases and show how easy they are to set up.

Thanks, appreciate it! And I’m wondering the same thing. :wink: I’ll have to work on putting something together video-wise. It’ll take me some time because I’m far from a graphic designer. I’m a good developer but a monkey with a package of sidewalk chalk can produce better graphics assets than I can. I’ve fallen in love with Unity since I started playing around with it but I definitely migrate more toward the code oriented bits.

Great sounds like it should be very fast for mobile and will possibly be working for 3.5. Thanks for letting us know :slight_smile:

Tested it against 4.0 (was using 4.1.3) and it worked fine as well. My only problem is that Unity < 4 (i.e. 3.3, 3.5, etc) didn’t have an Indie version. I’ll have to install as a trial and create a project to test it. Is there any other method to push asset packackes to the asset store besides the editor tools? The tools look great, but it looks like I need to push from 3.5 if I want it to support 3.5, but I won’t have a full license for it.

Hi,

Any news?