General Feedback, missing functionality and bugs

[fherbst]

Hi!

I tried the IET framework a while back (after seeing it in the LEGO tutorials) and also tried creating custom tutorials, and was impressed at both the complexity of the custom inspectors and the “hackiness” in place to inject in all kinds of places in the editor ;). I personally think especially the experience for tutorial creators is not ready for a public release, but since you seem to think otherwise I’ll just list some feedback and bug reports here:

1. Seems there is currently no way to highlight menu items or point people to where to open them. Is the recommended approach here just to write this in plain text “Open Window > Custom Thing” and have a EditorWindowCriterion for completion? (Case 1320855)

2. From a UX standpoint, for masking you really need a way to “pick”, maybe that can be done in a similar way as the UI Debugger does. It’s really cumbersome to figure out what is needed here. It’s somewhat easier with source access/good decompilers (e.g. Rider)

3. The way masking works seems very hacky (separate “mask” and “outline” objects?) and breaks in simple cases already (e.g. undocking a window) - (Case 1320860) - where the mask area and the outline affect different places in the window, which breaks the tutorial for users.

4. There’s still a ton of testing code in there, e.g. the “Force default inspector” checkbox, is that really intended?

5. It’s very confusing that all paragraphs have a “Masking Enabled” checkbox and it’s not clear why that is. Seems only the masking from the first paragraph is ever used?

6. It seems that adding criterions is broken in 2021.2, seems to work in 2020.2 (Case 1320883)

7. When no scene asset is set for a tutorial, a new scene will be created instead of staying in the current user scene. Is this intended? Imagine a tutorial that explains you how to create specific types of objects for your scene, or uses some analysis tools in it.

8. Selection criteria are broken when the object reference is a child in the hierarchy; additionally, going “Next” then “Back” on a selection criterion breaks the tutorial (tutorial window is masked, user can’t progress anymore). (Case 1320889)

9. It seems that VisualElements (renamed: UI Toolkit) isn’t supported for masking - is that an oversight? Will that come? Or am I missing something… This feels weird as Unity has been advocating for moving editor UIs to VisualElements for a few years now (and some of Unity’s UIs have already moved over, e.g. Settings and PackageManager). Doesn’t look like one can make tutorials for them now. Good test case: Make a tutorial page that asks the user to select a specific package in PackMan and install it.

10. In general, I think you should “dogfood” this more in actual usecases besides the beginner-friendly workflows in e.g. the Microgames. How about you make an In-Editor Tutorial for tutorial creators, that shows people how creating tutorials work?

11. It would be great to have a slight delay between “criterion is completed” and “tutorial jumps to the next page”, otherwise it feels like the user has done something wrong (“huh - why did the page switch now? I haven’t clicked Done yet!”). Auto-advancing in itself is useful but having a slight delay of say 1s would make it less confusing.

Looking forward to feedback to the above - hope that helps! I really look forward to more in-editor tutorials
Thanks,
Felix

[RikuTheFuffs-U]

Hi Felix, thank you so much for your feedback! I’ll try to answer your questions/observations here:

@fherbst

1. Seems there is currently no way to highlight menu items or point people to where to open them. Is the recommended approach here just to write this in plain text “Open Window > Custom Thing” and have a EditorWindowCriterion for completion? (Case 1320855)

Yes. For now, IET is not able to access that part of the UI.

@fherbst

2. From a UX standpoint, for masking you really need a way to “pick”, maybe that can be done in a similar way as the UI Debugger does. It’s really cumbersome to figure out what is needed here. It’s much easier with source access/good decompilers (e.g. Rider)

That’d be a great addition. I’m not sure the picking can be done with IMGUI objects though. It looks like UIDebugger can’t do that aswell.

@fherbst

3. The way masking works seems very hacky (separate “mask” and “outline” objects?) and breaks in simple cases already (e.g. undocking a window) - (Case 1320860) - where the mask area and the outline affect different places in the window, which breaks the tutorial for users.

It is very hacky indeed, and it’s all legacy code. I hope we’ll have the chance to rewrite that in the future.

@fherbst

4. There’s still a ton of testing code in there, e.g. the “Force default inspector” checkbox, is that really intended?

The “Force Default Inspector” is basically an “emergency option for advanced debugging”. It has been helpful many many times in the past, so we left it there. I’m not sure what other testing code you’re referring to, can you please elaborate on that?

@fherbst

5. It’s very confusing that all paragraphs have a “Masking Enabled” checkbox and it’s not clear why that is. Seems only the masking from the first paragraph is ever used?

Yes, only the masking of the first paragraph is used. This is the same set of masking settings the inspector displays (unless you check “force default inspector”).
The system previously allowed to set different masking settings and multiple instances of the same type of paragraphs (I.E: multiple instructive paragraphs) per page, but over time we realized that we wanted to simplify the structure of tutorial pages so they have only one instance per type of paragraph, top.

@fherbst

6. It seems that adding criterions is broken in 2021.2, seems to work in 2020.2 (Case 1320883)

Thanks for flagging, we’ll investigate that.

@fherbst

7. When no scene asset is set for a tutorial, a new scene will be created instead of staying in the current user scene. Is this intended? Imagine a tutorial that explains you how to create specific types of objects for your scene, or uses some analysis tools in it.

Yes, this is intended. We could add an option to alter this behaviour, though.

@fherbst

8. Selection criteria are broken when the object reference is a child in the hierarchy; additionally, going “Next” then “Back” on a selection criterion breaks the tutorial (tutorial window is masked, user can’t progress anymore). (Case 1320889)

By going “Back” you can’t rerun the tutorial step, but you should be able to move forward again

@fherbst

9. It seems that VisualElements (renamed: UI Toolkit) isn’t supported for masking - is that an oversight? Will that come? Or am I missing something… This feels weird as Unity has been advocating for moving editor UIs to VisualElements for a few years now (and some of Unity’s UIs have already moved over, e.g. Settings and PackageManager). Doesn’t look like one can make tutorials for them now. Good test case: Make a tutorial page that asks the user to select a specific package in PackMan and install it.

I completely agree with you, and in fact we plan to support VisualElements, it’s one of the things we’re prioritizing. i don’t have an ETA, as the integration is not that obvious. The UIDebugger code might be really helpful in understanding how to do this.

@fherbst

10. In general, I think you should “dogfood” this more in actual usecases besides the beginner-friendly workflows in e.g. the Microgames. How about you make an In-Editor Tutorial for tutorial creators, that shows people how creating tutorials work?

IET is going to be included in all templates, so there’s plenty of room for dogfooding. Creating a tutorial is a two-clicks operation. Configuring them is the hard part, and we could definitely do some work on that. Thanks for the idea!

@fherbst

11. It would be great to have a slight delay between “criterion is completed” and “tutorial jumps to the next page”, otherwise it feels like the user has done something wrong (“huh - why did the page switch now? I haven’t clicked Done yet!”). Auto-advancing in itself is useful but having a slight delay of say 1s would make it less confusing.

Thanks for the feedback! As there might be different preferences concerning this, we might consider making this a tutorial setting that every tutorial Author can edit according to its preferences.

@fherbst

Looking forward to feedback to the above - hope that helps! I really look forward to more in-editor tutorials
Thanks,
Felix

This truly helps! Thank you so much for your time, we really appreciate your feedback.

[RikuTheFuffs-U]

@fherbst also, just to be sure, what version of IET is your post based on? 2.0.0-pre.4, or 1.2.X?

[fherbst]

I’m on 2.0.0-pre.4, I think I tried 1.2.X in the LEGO tutorials but for the tests above I used the latest.

2. For IMGUI there’s a separate debugger under Window > Analysis > IMGUI Debugger that can pick IMGUI elements:
   

3. Would be great if you move it to a less-obvious place - right now it’s the first button when someone opens a tutorial page; Unity-style candidate would be the ⋮ (three dots vertical) menu in the top right of each window:
   I understand it’s an emergency option and very useful, I just think it’s not good to have a “pre-release” or even “verified” version that needs the emergency button in such an obvious place.

4. Would be great if you change the UI to reflect that this only makes sense for the first (e.g. hide the masking toggle from the other elements when the default inspector is active) – otherwise it’s pretty confusing and far too easy to put stuff in wrong places and then wonder why it doesn’t work. I’m also wondering if I’m allowed to change that structure (3 paragraphs) or if everything will break then; right now it looks like I can?

5. Yes, please, as outlined - let me know if you need additional info as to what the usecase for that is. Right now it seems “tool tutorials” would always need a special scene and I can’t tell people how a tool works in their scene even with careful crafting of conditions etc.

6. I’m not sure I understand that. Users certainly have a need to go a step back if they feel they didn’t understand something or want to take another look. In the case I outlined (and reported as bug) it actually breaks the tutorial because the condition can’t be fulfilled anymore since the marking is incorrect and doesn’t allow me to click on the right object.

7. I think you should really prioritize this; otherwise this becomes another case of Unity releasing X New Things that only work with Old Things and not with each other, really hindering adoption (e.g. we try to use and give feedback to the combination of new things since that is relevant for the future, not how a New Thing works with Old Things individually).

8. Yes, I meant that you could try creating a tutorial for the tutorial creation. As mentioned I think that the “template tutorials” shipping with Unity now or in the future are all pretty low complexity (they are meant for beginners after all) and I see the value for end-user tutorial creation in much more complex flows. Thus, if you can make a tutorial for how to make tutorials that would certainly help to prove that complex workflows can be “tutorialized”.

Thanks for listening,
Felix

[RikuTheFuffs-U]

@fherbst

I’m on 2.0.0-pre.4, I think I tried 1.2.X in the LEGO tutorials but for the tests above I used the latest.

2. For IMGUI there’s a separate debugger under Window > Analysis > IMGUI Debugger that can pick IMGUI elements:

That’s great! I’ve never seen it, we’ll definitely have a look at that

@fherbst

4. Would be great if you move it to a less-obvious place - right now it’s the first button when someone opens a tutorial page; Unity-style candidate would be the ⋮ (three dots vertical) menu in the top right of each window:
   I understand it’s an emergency option and very useful, I just think it’s not good to have a “pre-release” or even “verified” version that needs the emergency button in such an obvious place.

True, we can definitely put it in another, less-obvious place

@fherbst

5. Would be great if you change the UI to reflect that this only makes sense for the first (e.g. hide the masking toggle from the other elements when the default inspector is active) – otherwise it’s pretty confusing and far too easy to put stuff in wrong places and then wonder why it doesn’t work. I’m also wondering if I’m allowed to change that structure (3 paragraphs) or if everything will break then; right now it looks like I can?

The whole point is that you should not use the default inspector, which bring us to the point of hiding the option, unless you truly know what you’re doing. In the final version of the package, you won’t be able to see it at all.
Changing the structure of the pages will very likely break everything. I’d recommend to stick to the simplified inspector.

@fherbst

7. Yes, please, as outlined - let me know if you need additional info as to what the usecase for that is. Right now it seems “tool tutorials” would always need a special scene and I can’t tell people how a tool works in their scene even with careful crafting of conditions etc.

Makes perfect sense

@fherbst

8. I’m not sure I understand that. Users certainly have a need to go a step back if they feel they didn’t understand something or want to take another look. In the case I outlined (and reported as bug) it actually breaks the tutorial because the condition can’t be fulfilled anymore since the marking is incorrect and doesn’t allow me to click on the right object.

I probably wasn’t clear enough in my previous answer, let me rephrase: As you said, that is a bug. The default behaviour would be “you can go back, the tutorial window is the only one highlighted, all conditions for going forward are completed and you don’t need to do anything else to be able to go forward again”

@fherbst

9. I think you should really prioritize this; otherwise this becomes another case of Unity releasing X New Things that only work with Old Things and not with each other, really hindering adoption (e.g. we try to use and give feedback to the combination of new things since that is relevant for the future, not how a New Thing works with Old Things individually).

As I said, I 200% agree with you, being a heavy user of VisualElements in my own projects (in which I had wanted to use IET for months). Unfortunately I’m not the dev in charge of IET (I only work on it), so I can’t make promises, but I already brought up the feature request to the team and the reply from my colleagues was encouraging. Getting more feedback like this from the community definitely helps in making clear that this is something there’s a use case for.

@fherbst

10. Yes, I meant that you could try creating a tutorial for the tutorial creation. As mentioned I think that the “template tutorials” shipping with Unity now or in the future are all pretty low complexity (they are meant for beginners after all) and I see the value for end-user tutorial creation in much more complex flows. Thus, if you can make a tutorial for how to make tutorials that would certainly help to prove that complex workflows can be “tutorialized”.

Yup, that’s a great idea, but it’ll take sometime. In the meanwhile Did you already had a look at the documentation embedded with the package and hosted online here? It explains how to use the authoring tools, and it’s very detailed.

Thank you again for your feedback, I wished more users were like you!

[giant_ian]

Hi Paolo,

We’re really excited to see the Tutorial Framework eke its way towards release, and are in the process of translating some teaching materials to it for testing purposes.

In one of your responses above you said:
Yes, only the masking of the first paragraph is used. This is the same set of masking settings the inspector displays (unless you check “force default inspector”).
The system previously allowed to set different masking settings and multiple instances of the same type of paragraphs (I.E: multiple instructive paragraphs) per page, but over time we realized that we wanted to simplify the structure of tutorial pages so they have only one instance per type of paragraph, top.

I feel that last part of this is a poor choice, or at least limits the flexibility of the Tutorial Framework. For example, I would like to make a really simple beginner’s tutorial that advises e.g.:

• Create a new empty GameObject

• Call the GameObject “MyFirstGameObject”

• Add a new Script called “FooScript” to the “MyFirstGameObject”

Each of these steps are simple enough, but I’d like to present them on one page, rather than end up with a tutorial that is 100s of pages long because steps like this must be split up if you want to be able to easily show the learner which part of the process they’ve got wrong.

Ideally for my use case, this kind of thing would have multiple instructions on a single tutorial page.

Any chance of allowing what appears to be the old/original functionality to be used?

[Ali-Unity3D]

@giant_ian

Hi Paolo,

I feel that last part of this is a poor choice, or at least limits the flexibility of the Tutorial Framework. For example, I would like to make a really simple beginner’s tutorial that advises e.g.:

• Create a new empty GameObject

• Call the GameObject “MyFirstGameObject”

• Add a new Script called “FooScript” to the “MyFirstGameObject”

Each of these steps are simple enough, but I’d like to present them on one page, rather than end up with a tutorial that is 100s of pages long because steps like this must be split up if you want to be able to easily show the learner which part of the process they’ve got wrong.

Ideally for my use case, this kind of thing would have multiple instructions on a single tutorial page.

Any chance of allowing what appears to be the old/original functionality to be used?

Hi, Ali, another IET dev here. Yup, due to this exact reason we’ve have been discussing of bringing the old functionality back, so there’a definitely a change, just don’t know if it’s for 2.0 or for 2.1 a bit later on.

[giant_ian]

Thanks for the response Ali I’ll hope for 2.0

[giant_ian]

[Apologies if this kind of post isn’t helpful – happy to delete]

We decided we couldn’t wait and implemented this ourselves (multiple instructions and multiple narrative intermingled however you like):

I have to say it really improves the flow of the kinds of tutorials we are trying to create. So it’d be great if your discussions do end up in the return of this functionality for everyone. Looks nifty too with your good clean UI design

Also, we implemented a couple of other features we think are pretty necessary for certain classes of tutorial:

Left: unlocking progression (i.e. tutorials with pre-requesites). It’s likely to be helpful to allow for a multi-part tutorial with each section requiring the previous to be completed. In the picture: section 1 & 2 are complete, section 3 is unlocked (as section 2 is complete), and section 4 is locked (as section 3 is not complete).

Right: code blocks as part of the rich-text support. Our implementation of this has them be selectable too, so things can be copied into the code editor. An even better version would also provide automatic syntax highlighting also. (Not pictured: inline code sections too, so we can mix normal text and highlight code parts).

Any chance that these are features you might implement in the future?

[RikuTheFuffs-U]

Hi @giant_ian , thank you so much for your feedback!

@giant_ian

We decided we couldn’t wait and implemented this ourselves (multiple instructions and multiple narrative intermingled however you like):

That’s great! Did you manage to do that by using the public APIs we provided, or did you have to make your own version of the package? I’m asking this so we understand which classes/APIs is best to expose so users like you can expand the framework.

@giant_ian

I have to say it really improves the flow of the kinds of tutorials we are trying to create. So it’d be great if your discussions do end up in the return of this functionality for everyone. Looks nifty too with your good clean UI design

Yeah, I understand. I think that we removed it around 1 year ago because we wanted to simplify the structure of tutorials to better refine the system’s foundation before building on top of it

@giant_ian

Also, we implemented a couple of other features we think are pretty necessary for certain classes of tutorial:
Any chance that these are features you might implement in the future?

They definitely seem useful! Would you be willing to share the code? That would speed up the whole process from our side and probably open the road to similar/related improvements

[giant_ian]

Hi Paolo,

Only a short answer here, more to follow when I have a bit more time:

• We used a bit of both, but mostly this relied on our own version of the package. It wasn’t really possible to allow for multiple “paragraphs” per tutorial page otherwise we don’t think. Would be super happy to chat about how we may have been able to accomplish more of what we did through the API directly (we were rushing, so may have missed some opportunities there).
• Very happy to share the code, would like to help the whole community in this endeavour What’s the best approach there?

Cheers,
Ian

[RikuTheFuffs-U]

Hi Ian,

Thanks for the collaborative approach!

@giant_ian

Hi Paolo,

• Very happy to share the code, would like to help the whole community in this endeavour What’s the best approach there?

The easiest thing that comes to my mind is that you share your version of the package + the snippets of code that are interacting with it (if any) to achieve the result. In this way we can see what differs from our version, and integrate the missing pieces. I created a private conversation between you and me where you can share the material (either as a zip or as a repo), please let me know if that works

[fherbst]

@giant_ian you could make a fork of GitHub - needle-mirror/com.unity.learn.iet-framework.authoring: Authoring tools for In-Editor Tutorials. Required for authoring In-Editor Tutorial assets. 📦 [Mirrored from UPM, not affiliated with Unity Technologies.] and add your changes in the fork, then paste the link here - this way it’s clear what version of the package it’s based on, there’s a clear update path etc.

[giant_ian]

@PaoloAbela I’ve sent you a ZIP of all our stuff and the example tutorials we’ve made with it
@fherbst Alas, I am git-phobic (in the minority I know, but we use SVN for all our projects)

[giant_ian]

We implemented some other things before trialing IET with our students. Informal feedback so far is they thought IET was far superior to text tutorials with screenshots or even a video version of the same content (we provided all three).

Left: We added hyperlinks the reference Assets (left image, with the mouse hovering over the “BasicSpawner” link)

Right: Also hyperlinks to GameObjects in the current open Scene (right image, with the mouse hovering over the “Spawner” link).

Selecting these hyperlinks highlights the asset/gameobject with PingObject.

Left: we added a new form of completion criteria “Completed on User Indication”. Users tick this box to mark the Instruction as complete. (Perhaps this should’ve been done with a custom Criteria rather than an addition to the Instruction Paragraph, but I’m not sure how we would’ve got the checkbox then).

(Also Left: in-line code blocks).

Right: Poll Paragraphs. Selecting any option marks the Paragraph as complete, and users are shown whether the answer they selected is correct (and if not, the correct answer is shown) – here I selected the wrong answer. These Polls are meant as quizzes, just to test the user’s understanding of the concepts.

[RikuTheFuffs-U]

Hey Ian, thank you again for your support. It’s always exciting to see how users like you are able to push forward and overcome the limits of our products!
I forwarded your feedback and code to the rest of the team, and we plan to have a deep look at it as soon as we get internal approval from Legal (they need to sort out some license details) and we have the chance to reserve some time.

[giant_ian]

Adding those features was a pretty exciting side-project.

The in-line links to assets and gameobjects take very little to support, and really helps with student comprehension of directions, and means you don’t have to write complicated directions to stuff deep in folders too. If they can’t find something themselves, or are confused about exactly which thing you are referring to, it’s just a click away

We’d love to have added similar links to things in the inspector too: components (e.g. SpriteRenderer) and fields (e.g. Transform’s Rotation) and sub-fields (e.g. Transform’s Rotation’s Z value). But alas the Highlighter class seems to have fallen into disrepair (perhaps connected to old UI versus new UI changes?). It’d be great if an update / replacement for that could be part of the IET package.

[fherbst]

@PaoloAbela got more feedback about the workflow, and possibly more bugs…

1. In some cases, it seems the strings are not editable, not sure why:
   

2. We’ve tried out our tutorials internally, and some people (not all) have reported that tutorial exit / closing doesn’t work and instead opens the Unity Hub - was able to get a recording:

This is a serious bug as the only way to get out of that loop is to kill Unity via Task Manager. Not sure how to reproduce though (on a different machine it worked), on the affected machines it happens every time (persists after restart).

EDIT: I was able to isolate the issue and reported as (Case 1332108) [IET] Tutorial workflow is broken, skips steps, can’t be closed anymore.

3. The same issue I reported as bug before “(Case 1320889) [IET] Masking objects does not mask correct regions after going backwards in the Tutorial steps” was also brought up by many as a bug, and in fact breaks the tutorial flow entirely in some cases. I can see that the issue was “closed” but I have no idea why, and I still believe this is a bug.

Here’s a simple flow that breaks with this:

• Tutorial asks to select Object A (Object A is masked)
• Tutorial asks to click a button in a window to export Object A; the button is masked (only the button can be clicked)
• Tutorial reaches last step. The user clicks around a bit (no masking), and selects Main Camera. Now, they want to repeat some of those steps, so they click Back, and Back again.
• Tutorial asks to select Object A (no Masking) but marks the step as done
• User clicks “Next”
• Tutorial asks to click a button to export Object A; only that button is available. Note that currently Main Camera is selected, not Object A, and the user has no way to correct this. Clicking Export will result in a wrong state and wrong result for the rest of the tutorial.

For users, these two actions are identical:

• restarting the tutorial
• clicking back until they arrive at the first step again

[RikuTheFuffs-U]

Hey @fherbst , thanks for the feedback!

@fherbst

@RikuTheFuffs-U got more feedback about the workflow, and possibly more bugs…

1. In some cases, it seems the strings are not editable, not sure why:

Unfortunately, this is a critical bug that we aim to fix before the next release. The bug is "Live editing of Tutorial Pages no longer works on pages that have a Criterion". Can you confirm that you had the tutorial running while it happened?

2. We’ve tried out our tutorials internally, and some people (not all) have reported that tutorial exit / closing doesn’t work and instead opens the Unity Hub - was able to get a recording:

This is a serious bug as the only way to get out of that loop is to kill Unity via Task Manager. Not sure how to reproduce though (on a different machine it worked), on the affected machines it happens every time (persists after restart).

EDIT: I was able to isolate the issue and reported as (Case 1332108) [IET] Tutorial workflow is broken, skips steps, can’t be closed anymore.

Nice catch. Do you know if the user encountering the problem was logged in with his Unity account, in the Editor? I suspect this happens when the user is not logged in, and probably relates to the way we report progress for tutorials (cc @Ali-Unity3D @mikaisomaa )

3. The same issue I reported as bug before “(Case 1320889) [IET] Masking objects does not mask correct regions after going backwards in the Tutorial steps” was also brought up by many as a bug, and in fact breaks the tutorial flow entirely in some cases. I can see that the issue was “closed” but I have no idea why, and I still believe this is a bug.

Unfortunately, IET currently does not support redoing steps by going back. IIRC, You can go back, but the masking will just unmask the Tutorial Window.
Right now, you can use the Tutorial Callbacks provided (I.E: OnBeforePageShown) to select the object relevant to the step you’re entering (I.E: to automatically select Object A in the export step)

[RikuTheFuffs-U]

Moreover, can you please point out on which Unity + IET Framework + IET Authoring version are you on?