Unity Analytics Heatmaps: Official Thread

https://www.youtube.com/watch?v=usHiDE9aG0I

This is the official discussion thread for Analytics Heatmaps.

[UPDATE]
UPDATE: May 18, 2017
Hey all! Quick note that you can now turn on heatmaps yourself on your dashboard config page. And, during the month of May only, we're letting everyone – regardless of license – activate heatmaps! Once activated for a project, you get to keep it forever.
[/UPDATE]

The project is now live and we invite you to get mapping. Here's what you need to know:

Let us know here if anything's not working, or if there's something more you'd like to see!

Here's a presentation from Unite Boston on Heatmaps and the general use of custom events...

https://www.youtube.com/watch?v=aWZiB7jX7C0

2 Likes

After referencing the potentially high data usage of heatmaps, the documentation states "Your app should send heatmap data ONLY for development builds.".

Is the intention that heatmap commands should not currently be included in any release version of a game regardless of how often they'd be sent? Considering I haven't seen this referenced elsewhere, I'm assuming maybe that part of the docs is out of date and from an earlier (e.g. alpha) release?

No, that's current (and was mentioned both in the previously released Google doc and in my presentation at Unite). In theory, we can handle an awful lot of events...but we're being cautious and watching for potential pitfalls/breakages.

There will hopefully come a day when we allow you to proceed with reckless abandon and heatmap everything you like. For now, though, we're asking you to limit heatmap activities to pre-release (essentially, your test users). If you have a case where you think you really need heatmaps in your live product, talk to us and we'll see what we can do. But let's be a little careful until we see what this baby can do. :)

[quote=“marc_tanenbaum”, post:3, topic: 599676]
No, that’s current (and was mentioned both in the previously released Google doc and in my presentation at Unite).
[/quote]

Ok, all good. I watched the Unite presentation and am generally well informed about things like this, but for whatever reason I wasn’t aware of that until I was reading through the docs. It might be worth mentioning it more visibly.

[quote=“MattCarr”, post:4, topic: 599676]
I watched the Unite presentation…
[/quote]

Possibly you just saw the keynote (or were you at the conference)? I’m eagerly awaiting the release of the Unite video where I went fairly deep on heatmaps, but AFAIK it hasn’t been released yet. John Cheng did a quick intro during the keynote which probably didn’t highlight this point.

I’ll ask about highlighting the point in the docs…the thing is that there are probably lots of points we should highlight. As the guy who built it, I want to highlight everything. :wink:

Hi Marc,

I wanted to talk to you at Unite, but couldn't find you.

I think this is great, just have a suggestion. Heatmaps are most useful when metadata is attached to them, not just the size or the number of events, but rather, other things, like cause of death, or a velocity vector. We have a powerful gizmo rendering system inside unity, why not use it?

Why I propose, is that imported data is processed by an editor script (the processor), producing an array of instances of a serializable struct, that may or may not have a position as well as other data, and a renderer, that takes that array and produces gizmos on the editor. The renderer can aggregate groups of nearby points, but it's optional.

This requires that arbitrary metadata can be added (including more than one vector).

With this, we could customize colors, render fields of influnce, render velocity vectors, etc.

What do you think?

Hi @sethillgard ... sorry we didn't talk at Unite. Sounds like you have lots of interesting ideas for the project!

Let's start with this: you can in fact attach arbitrary data to your heatmap events as I mention here. What you can't yet do is apply anything useful with that data in the Heatmapper. Now, obviously by adding the capability to send the data, I'm signaling my intent to help you use it.

My intention has been to help you aggregate on the arbitrary data. For example, you might send this:

    var dict = new Dictionary dict<string, object>();
    dict["level'] = currentLevel;
    UnityAnalyticsHeatmap.HeatmapEvent.Send("PlayerDeath", transform, Time.time, dict);

This would allow you to aggregate on level and see level1 Heatmap.PlayerDeath events separated from level2, level3, etc.

But it sounds as if you're thinking of something more, and I'm not quite sure I understand what that something is. What specific gizmos are you suggesting and how would the developer access them? Are you thinking of some kind of plugin to the plugin so that a dev could create the gizmo they need? More detail might help me firm this into something actionable. (Also, this is all open source...so if you wanted to create a fork and demonstrate your idea in code, we'd happily look at the pull request!)

[quote=“sethillgard”, post:6, topic: 599676]
We have a powerful gizmo rendering system inside unity, why not use it?
[/quote]
If you’re suggesting using gizmos to represent data instead of points drawn by a mesh as it’s currently the case, I’m afraid the gizmo system doesn’t scale enough for that.
We have a path tracker in place for our project, and I did the render of the path points with gizmos. It’s fine for some playthroug, but it quickly slows down the editor.

With more options to filter down data, maybe we’d need the possibility of displaying several heatmaps at once, with different colors/materials for each. So you could for example display both the players path and where they dies, with custom icons where they die.

Sorry for the late reply.

Yeah, I am suggesting a plugin of plugins of sorts. It's not that different from what you currently have so long as we can attach data in that dict, including other vectors. I guess the only real thing I'm suggesting is the ability to have custom scripts go through the heatmap data and render their own stuff, wither on top of the heatmap or replacing it. My whole point about having a 2 step process where we first process the heatmap data and produce "points" (this part of the process takes nearby points and collapses them into a single renderable unit, selects a color, calculates the average of another field that was set in the dict, etc), and a second pass that renders those points (and the data calculated in the previous step) is just a formalization of that.

@khan-amil , good point. I didn't consider that. Still I think It'd be nice to be able to attach gizmos in certain cases. Maybe not all points have a gizmo, but we could add a gizmo to points where "players used item X", or a vector gizmo where "players's velocity average had a magnitude greater than Y"

[quote=“sethillgard”, post:9, topic: 599676]
…so long as we can attach data in that dict, including other vectors.
[/quote]
So over the weekend I added the ability to group on arbitrary data. This code isn’t on master yet for a variety of reasons, but it exists and you can now do pretty much what I outlined in my comment to you above. You won’t be able to attach a vector per se (and possibly you never will). The Custom Event Dictionary — the real one on which the HeatmapEvent is built — only supports 10 properties and I’m constrained to stay within that limit; I need to train developers never to overflow that limit. If you send position, that’s 3 properties. Rotation…that’s another 3. Add time and we’re already using 7 of 10. To the best of my knowledge, that constraint won’t go away anytime soon.

There’s no reason you couldn’t dissociate a vector (like velocity) and use that as the remaining three properties. Figuring out a generalizable mechanism that would allow me to add visual markers (as you might want with velocity) might be an interesting idea.

[quote=“sethillgard”, post:9, topic: 599676]
I guess the only real thing I’m suggesting is the ability to have custom scripts go through the heatmap data and render their own stuff
[/quote]
I’d probably need to go deep on a couple practical cases to fully get this. In the end, I’ll get time for these features only if I can demonstrate the value of my extremely limited time to the man upstairs. Feel free to PM me if you like…but what I’m looking for is a very specific case: if I render X on top of the heatmap it will tell me Y. A gizmo that ‘points where “players used item X”’ is only just so useful…since you could simply render a separate map of that data. Maybe the velocity idea is more the thing.

[quote=“sethillgard”, post:9, topic: 599676]
My whole point about having a 2 step process where we first process the heatmap data and produce “points” (this part of the process takes nearby points and collapses them into a single renderable unit, selects a color, calculates the average of another field that was set in the dict, etc), and a second pass that renders those points (and the data calculated in the previous step) is just a formalization of that.
[/quote]
I’m honestly not sure how this differs from what we currently do.

1 Like

Hi,

I add the Heatmaps_Installer.unitypackage and this day my heatmap request are activated and i follow the guide and add the :

https://analytics.cloud.unity3d.com/api/v1/batches?appid=APP&hash=HASH

For testing i add the headmap event on click and pass the position:

UnityAnalyticsHeatmap.HeatmapEvent.Send("MapPoint", pointPosition, pointDictionaryData);

But do nothing and when i touch in "Fetch and process" i have the follow error:

NullReferenceException: Object reference not set to an instance of an object
HeatmapMeshRenderer.RenderHeatmap () (at Assets/Plugins/Heatmaps/Renderer/HeatmapMeshRenderer.cs:171)
Heatmapper.Update () (at Assets/Editor/Heatmaps/Heatmapper.cs:106)
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[ ] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono-runtime-and-classlibs/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[ ] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono-runtime-and-classlibs/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:232)
System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[ ] parameters) (at /Users/builduser/buildslave/mono-runtime-and-classlibs/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:187)
UnityEditor.HostView.Invoke (System.String methodName) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:180)
UnityEditor.HostView.SendUpdate () (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:245)
UnityEditor.EditorApplication.Internal_CallUpdateFunctions () (at C:/buildslave/unity/build/artifacts/generated/common/editor/EditorApplicationBindings.gen.cs:270)

and Points in current and Points currently displayed always is = 0.

Unity Version: 5.2.1p4
SO: Windows 10 Pro

[quote=“Onsterion”, post:11, topic: 599676]
But do nothing and when i touch in “Fetch and process” i have the follow error:
[/quote]

I can see what’s causing the NPE and that fix was just posted. Basically, you’re attempting to recolor/manipulate the heatmapper while there’s no data in the set. I just put in a test so you can’t do that.

But the real issue for you is that there’s no data to display. I’ve looked at your project and I’m not seeing any HeatmapEvents. Did you check the validator to make sure they’re being received? How long ago did you start sending them? It typically takes anywhere from 2-12 hours for raw data to get exported so that it’s available to the Heatmapper.

[quote=“marc_tanenbaum”, post:12, topic: 599676]
I can see what’s causing the NPE and that fix was just posted. Basically, you’re attempting to recolor/manipulate the heatmapper while there’s no data in the set. I just put in a test so you can’t do that.

But the real issue for you is that there’s no data to display. I’ve looked at your project and I’m not seeing any HeatmapEvents. Did you check the validator to make sure they’re being received? How long ago did you start sending them? It typically takes anywhere from 2-12 hours for raw data to get exported so that it’s available to the Heatmapper.
[/quote]

Hi marc_tanenbaum!,

1- I added the new build and the first error does not appear.

2- Not passed 12 hours . But i have some form to store data in local and than click to show? I select “Local Only” and than “Process” but again :

Points in current set: 0
Points currently displayed: 0

Greetings.-

[quote=“Onsterion”, post:13, topic: 599676]
1- I added the new build and the first error does not appear.
[/quote]
Excellent!

[quote=“Onsterion”, post:13, topic: 599676]
2- Not passed 12 hours . But i have some form to store data in local and than click to show? I select “Local Only” and than “Process”…
[/quote]
Well, “Local only” is precisely that: it prevents the heatmapper from contacting the server. This speeds you up when you know your local data cache is up-to-date. But it has absolutely nothing to do with sending heatmap data. As I wrote in the documentation:

So please (1) confirm that your HeatmapEvents are showing up in the validator, and (2) give it a few more hours and try again. The data will be there, but data processing takes some time.

[quote=“marc_tanenbaum”, post:14, topic: 599676]
Excellent!

Well, “Local only” is precisely that: it prevents the heatmapper from contacting the server. This speeds you up when you know your local data cache is up-to-date. But it has absolutely nothing to do with sending heatmap data. As I wrote in the documentation:

So please (1) confirm that your HeatmapEvents are showing up in the validator, and (2) give it a few more hours and try again. The data will be there, but data processing takes some time.
[/quote]

I can’t see the documentation : https://bitbucket.org/Unity-Technologies/heatmaps/overview it’s down with error 500 :S.

How can confirm if the HeatmapEvents are showing up in the validator? In the case if not showing what can i do?

Greetings.-

That link works for me. As far as using the validator, please read the documentation for Unity Analytics.

http://docs.unity3d.com/Manual/UnityAnalyticsCustomAttributes.html

[quote=“marc_tanenbaum”, post:16, topic: 599676]
That link works for me. As far as using the validator, please read the documentation for Unity Analytics.

http://docs.unity3d.com/Manual/UnityAnalyticsCustomAttributes.html
[/quote]

For me is down xD

And yes the analytics are validated.

Greetings.-

@marc_tanenbaum

I have been playing around with the heatmaps, and have events being sent and received by the analytics server. But i have issues when trying to load that data. It always returns 0 results.

I looked into it further, and it seems for me any use of the web downloading processing fails. I had to hack a few places to remove whatever web download functionality was in place and instead use Unity's WWW calls. This was done in RawDataDownloadClient.cs in its DownloadFileAsync function (turned into a synchronous WWW download instead of using WebClient) and also in RawEventClient.cs in the FetchData function (a synchronous WWW download instead of using WebRequest).

After making these two modifications, data was finally downloading properly and I could see heatmap events.

This is running on Mac OSX if it makes any difference.


Hi @BFS-Kyle , thanks for trying out Heatmaps!

From your description, it sounds as if you're trying to run the Heatmapper at runtime (WWW is runtime only...that's why we don't use it). There's no reason to download the data at runtime. Download and render the mesh without hitting play. You can still hit play after you download if you want to see the map against your runtime setup.

[quote=“marc_tanenbaum”, post:19, topic: 599676]
Hi @BFS-Kyle , thanks for trying out Heatmaps!

From your description, it sounds as if you’re trying to run the Heatmapper at runtime (WWW is runtime only…that’s why we don’t use it). There’s no reason to download the data at runtime. Download and render the mesh without hitting play. You can still hit play after you download if you want to see the map against your runtime setup.
[/quote]

Nope, not loading it at runtime. WWW can still work outside of runtime, you just cant yield it in a coroutine (so it has to be used synchronously). Might just be a weird setup / config on my machine if nobody else is seeing the same issue.

Just as an example, this is what I changed in RawDataDownloadClient.cs:

public void DownloadFileAsync(string url, string filePath, CompletionHandler handler)
        {
            this.downloadCompletionHandler = handler;
            //base.DownloadFileAsync(new Uri(url), filePath);
            WWW _WWW = new WWW (url);

            Debug.LogError ("Now downloading data from url [" + url + "]...");

            while(!_WWW.isDone)
            {
                if(!string.IsNullOrEmpty(_WWW.error))
                {
                    Debug.LogError("Error downloading - error is [" + _WWW.error + "] from url [" + url + "].");
                }

                //Keep waiting for download to finish.
            }

            //Save the results
            Debug.LogError ("Finished downloading! Results is [" + _WWW.text + "] from url [" + url + "].");

            Debug.LogError ("Saving data to path [" + filePath + "]...");

            EditorStorage.SaveString (filePath, _WWW.text);

            downloadCompletionHandler (true, "Download complete");
        }