Funnel - display not helpful. Why no drop-off rate?

Unity Services
1

Hello,

I found the display of funnels to be quite underwhelming. It shows me the percentage of the population that reached a certain funnel step. (The other setting “playtime”: I couldn’t figure out what it means exactly and its not explained anywhere, so I ignore it for now…)

3264003--251808--upload_2017-10-24_8-51-0.png

This might be an easy way to explain funnels, but in real life, what does it help? IMHO its totally irrelevant how many percent of the total player base reached a certain step. The most important question is the drop-off in each INDIVIDUAL funnel step,e.g. “how many percent of the people which reached the previous step also reached that certain step.”

Or mathematically: the derivate of the current display:

3264003--251807--upload_2017-10-24_8-50-11.png

In other words, IMHO you want to identify “Where do player drop off the funnel?” “What is the problematic step where everybody quits when playing? (or what step is way too-easy?)”

Yes, you can eyeball the curve and look for some steep decline (too difficult/boring) or flat part (too easy). But why do calculus in your head when you have computers? I don’t get it.

So currently, to be useful, you always have to download the csv, import into excel, write some “=B4/B3” in some column to get the derivate and then plot this by yourself. (Or you do the clumsy eyeballing-thing, trying to do calculus in your head…)

Shouldn’t Unity do this on the website as default?

Ciao, Imi.

PS: The example data were after balancing, so it might not be the best data to make my point. As you can see, we are quite happy with our funnel progression. :wink:

PPS: As you can see from the unity-screenshot, the name of the custom event takes up all the labelling-space on the diagram. Would be nice if Unity would strip the custom event name if its the same for all funnel steps. :wink: (Which I would assume this is true for all funnels plotting regular checkpoints using the same custom event)

PPPS: Also, setting up the funnel was quite annoying. Fortunately you have to do this only once per game/funnel but I’d like some way to copy the last funnel step into the next one when setting up the stuff. I imagine that this would help 99% of the funnels, as any funnel step configuration is most probably “like the last funnel step with one value of one parameter slightly changed”

PPPPS: Yes, “drop-off rate” is probably misleading word for what I plotted in my graphics. But Unity’s graph also does not show a drop-off but some kind of “reached rate” where they plot how much percent reached a certain funnel step, so I thought its better to keep the graph in the same direction. In literature, its usually plotted as drop-off, which is simply “100-my values”

2 Likes
2

I agree with everything said above, especially the part about setting up the funnel and having the next step start with a copy of the previous step so you can just tweak one value.

3

The first graph that you display does provide the drop off rate. Keep in mind that funnel steps are sequential. So in this graph, 100% of the users completed Level 1 (I’m assuming, the legend is missing). Then of those 100% that completed Level 1, 72.9% went ahead and completed Level 2. If the third bar was considerably lower, you would know that of the users finished Level 2, many did not go on to finish Level 3 (a drop off). You would see right away visually, without the differential (granted a handy calculation). Pro license users have access to Raw Data Export where they can download raw events locally, and perform the degree of analysis as desired. We are planning to offer more in-depth analysis features in the Dashboard for all users over the coming months, thank you for your feedback.

4

No, it does not. It is not a drop-off RATE, its a weird graph comparing each funnel step to the very first step. Which is totally stupid, except it is kindof nice to look at.

A “rate” is something relative to a time span. PRECISELY because a funnel is sequential, this makes it possible to calculate a “rate”. The natural “time span” (doesn’t have to be “time”, to be precise) here to choose would be “since the last funnel step”. You calculate some weird graph “from each funnel step to the very first funnel step”.

“Considerably lower” is EXACTLY doing a differential calculation in your head. You compare the values of two bars to eye-balling the “difference”. How else would you define a derivate?

I am sorry if I sound a bit grumpy about this (probably a lot grumpy). But as it stands to me, Unity Analytics is near to useless for any non-Pro users. (Luckily me: I am a pro-user).

Without downloading the data, the analysis capability are laughable. They may be nice to impress some managers with shiny graphs and to quickly look up some very basic stats. TBH, I was impressed the first few hours when I played around with the website. This turned into skepticism after half an hour and into resentment when I recognize, that you basically can’t do any sensible statistics.

So basically, you are selling a data storage server with a rather clumsy download interface. Thanks for confirming this. :(.

You shouldn’t overuse this “pro-only can download - answer”. It leaves this aftertaste of being a shortcut for “lets get rid of all critic on the current analysis capabilities by claiming that people can just do all the work themself”.

Other things I hate:

  • No queries involving more than one parameter to custom events. (wtf?) I looked literally two hours to find my “surely simple mistake that I must have done. Some checkbox I missed…” before I gone in the forum and realized that it is indeed not there. This was basically the time when I realized you are selling a data storage server - not an analytic service.
  • Not more than like 10 different values for strings in custom parameter. (Even if “Others” are 80% of the distribution). I know that it gets more CPU/storage intensive for your, the more values you have to split into. But I can’t even configure this, e.g. saying “for this parameter, I like to have 20 different values and to balance it out, this other parameter is only interesting for raw-data download.” Honestly, there are a TONS of parameter only interesting for raw-data download (e.g. heatmap coordinates) that you uselessly parse at the moment. No way to trade-off your computation capacity here. This will be true ESPECIALLY if you keep answering with “pro-users should download and analyze by themself”
  • No adding of graphs together (e.g. to plot “os=windows” vs. “os=mac” + “os=linux”). Again: its not rocket science to download the CSV and write some 5-minute Excel for that, but why bother with Unity Analytics then?
  • No retrospective analysis of new segments / funnels. Why? Now we can not run new analysis on older data to confirm anything. Again: I totally understand that this might extremely stress any CPU (it could outright kill your farm if enabled carelessly). But again: Why bother with Unity Analytics then? Why do we have a “50GB preparing download packets - limit” but no “50GB retrospective data analysis - limit”?

Arr… sorry again for sounding so harsh. I have to deal with disappointed users all day long too. Its just the crash of my hopes after I realized the true state Unity Analytics is in right now…

1 Like
5

Understood and thank you for your feedback, I will ensure the engineering team is aware of your concerns. You make some good points.