FlowCanvas (inspired by Unreal Blueprints)

EDIT–This post is totaly re-written

Hey,

So, I am working on a new visual scripting NodeCanvas extension for some time now, and would like to share what I’ve been doing with you all :).
The initial idea started off inspired by the node editor in XSI (ICE), but soon enough I kind of realized that this would rather not be very usefull as-is in a game development situation, which of course we are all after I presume.

The reason I started with something similar to XSI’s node editor for a visual scripting extension, was the freedom of data manipulation that is possible, compared to other purely game oriented designs existing for Unity right now. Without going to technical, the main difference is that it doesn’t have the usual “In - Out” state like nodes, but instead all data are piped down to an Update call, very similar (visualy) to how you would create a shader in a visual editor.


Softimage ICE Node Editor

Long story short is that in a game dev environment, the above system:

  • It may be costly for games.
  • It’s hard to read when it comes to flow control, like loops, conditionals etc which a game obviously is full of.
  • Require more work to figure it out that it’s worth, where instead you could go in and code the same thing. That is considering that a visual scripting tool is really there to bridge the gab between coder and designer, rather than replace either of the two!

With that said, I love the ICE system and use it personaly, but I think something similar would not fit for visual scripting game logic.

So, the next closest thing to XSI ICE node editor actually for game development, was clearly Unreal 4 Blueprint, Bitsquid Engine flowgraph and at a lesser degree CryEngine’s flowgraph, which combine a visualy forward execution state-like behaviour together with backward data fetching.

Blueprint, like ICE, also provides some very low level control, but there are some pros and cons by having this control or not.

Putting this to test, I came up with this. (Of course color coding things is now mostly missing)


In this first case, we can directly pass data around and there is no need to store it somewhere temporary, although we could if need be. Furthermore there is both great flow and data manipulation control. The downside is that a real graph might easily become a spageti, where in pure code it could be just a 2 liner.


In this 2nd approach, there is hardly any low level control and we can’t directly pass data around, but instead need temporary storing, which also needs to take place “within” the flow design by executing rather than getting (eg ‘Get Property’). The obvious good thing is that its much cleaner and probably more focused by not exposing all funtion arguments and return value as ports

Then again, at first glance it seems that the initial purpose of data manipulation is defeated without exposed ports.

In a way, we could combine the 2 designs above to get some good points from both worlds, by using ‘compounds’. Or in other words, a handfull of nodes compounded into one with specific exposed ports.

(This image is a proof of concept in photoshop)

And then use and reuse the compound node to make the higher level graph cleaner and focused.

This seems like a good aproach, but on the other hand, this compound node could be made into a simple function in code and then use an Execute Function node like the rest. So this is a point where I am skeptical about.
In the end it comes down to how low or high level control to exist. In both cases I hope this to become a very versatile tool in the overal steps of blueprint :slight_smile:

So, this is what I am working on and my thoughts on the matter shared with you.
What do you think? Do you have any ideas that want to share? Would you prefer one aproach over another? Would such a tool be useful to you?

Implementation wise, this is already working more or less except the compounds, UI color coding and some other thingies I will post in time. If you have any question feel free to ask :slight_smile:

Cheers!

5 Likes

[space]

Well I for one (as you know nuvie) will be following this! :smile:

Thank you Steve for your support!

Hey everyone,
I’ve totaly re-written the initial post with more in depth information.
I’d be glad to hear your thoughts.

Cheers!

So, here is a feature on debugging the graph, which I presonaly like :slight_smile:
You can selectively debug view any ‘Value Port’ for it’s current or last value in runtime. This is shown on top of the connection within the editor window like bellow:

I think having a visual debuging info “in-place” like this, really helps things when it comes down to creating some bigger graphs.

More comming soon :slight_smile:
Cheers!

Excellent! Nice one

Is this will be sold as a separate package from NC?

Would love to see gif version of visual debugging

I just bought NodeCanvas, but a bit to soon I think. I am investigating several approaches for simulating complex aircraft systems. I had a quick play with NodeCanvas but it appears it is not really designed for what I want to do as the simulated systems look a bit awkward using the current BT or FSM approach. For example, I can’t even add a simple AND, OR, or NOT gate. But that is a BT/FSM limitation I suppose.

I want to make the visual representation as close as possible to the real (simplified) electric / logic schematics available and avoid using the sequencer / selector approach as these are not representative of real world hardware. Lines between nodes should be able to carry boolean or float values, so simulation of pneumatic, hydraulic, and electric systems is possible.

It looks like the new extension you are working might be the thing I am looking for although the removal of AND / OR gates in your latest example has me slightly worried. Also, the possibility to change the graphics of the current generic square node would be a plus. This way it would have a closer resemblance to schematic logic gates, hydraulic reservoirs, etc.

Does it make sense what I am trying to say and would do you think that the thing I am looking for is feasible?

Hello :slight_smile:

Yes, this will be a seperate package than NodeCanvas. Either an extension or a standalone. It still depends.
Here is a quite silly gif example for the shake of showing some animated values:
1873752--120418--FG_Debug.gif

This will of course be improved, but want to focus on the core first :slight_smile:

Hello,

You are definetely making sense, don’t worry :slight_smile:
The existing NodeCanvas package doesn’t include this system. NC (as you figured out) has a Behaviour Tree and HFSM implementation modules, which work and achieve things quite different than what you are after.
The WIP I am showing here is a separate module, build on NodeCanvas core framework.

This system, can indeed carry different (any) type of data across.
The AND was not removed in the last image. I was showcasing the concept of putting (nesting) a couple of nodes inside “compound nodes”. So in the last image of the first post, the AND is included in the “Can Open” compound node. Again, this is a concept, which I haven’t really done yet. So, AND, OR etc low level nodes, as of now, are included.

As far as changing the shape of the node, unfortunately I think that will be quite hard to do, but a node can have an icon instead, which is better than nothing :slight_smile: For example:
1873752--120420--Bool.png

As a final word, I would like to let everyone know, that the direction (purpose) of this tool might change in time and based on feedback or ideas provided. For the time being, I am focusing on creating a quite general data flow node editor, able to handle most situations which are common between any such node editors.

Cheers!

Hello,

What I would like to know is your estimation about this tools performance (Not the editor, its runtime performance)
Is it able to produce scripts or does it simply work at run-time?
If it produces scripts; Do you think it’ll be feasible to add a custom code node, so anyone can insert some code snippets into the flow.

Thanks

From your vision, what will this flow node main use in the future?

From what i know of nodecanvas:

  • BT - decision making, AI
  • HFSM - stateful process, simple decision making
  • Dialogue Tree - non linear dialogue data

Is flow node will be a general purpose visual programming helper? Maybe could become a nested node on BT/FSM. Almost removing need to open monodevelop?

@nuverian_1 ,
Sounds good! That icon will do the trick. It is all about “instant readability”, so that would be fine. Some more feature requests:

-Ability to stretch the icon to the background of the node and add the ability to slightly adjust the position of the line contact points to align int with the icon.
-Save a custom node for later use.
-Ability to change the icon with a simple API so that it can for example display an open or closed valve or switch.
-Simple animation on a node to visualize data such as a pie or bar, to be used for time, volume, etc.
-Nodes capable of running a script on the input and outputs instead of having pre determined behavior.
-Visual grouping of systems with each their own input and outputs. For example, create a bunch of nodes which represent an engine. Then group all these nodes together, call the large square “Engine”, and give it input “bool start” and “float friction”, and output “float RPM”.

@inas ,
Not sure what the supposed “main” usage would be but it would be perfect for mechanical/electrical system simulation. Remember that Unity is not only used to make games, as can be seen here:

Other then systems simulation, it is also just another way of doing things which in my opinion is more intuitive.

Hello,

No scripts are created from the system, no. It “simply” works in run-time, and very fast for that matter :slight_smile:
The reason of the fast performance is that when it comes to actually running the graph, the implementation behind is very close to how it would be in code.

What I mean, is that there is really not much between 2 ports execution. An Out call and an In method are directly binded and thus calling Out.Call() in the 1st node is essentialy calling method In() of the 2nd node directly, since ‘.Call’ is a System.Action which has been assigned the method ‘In’ of the 2nd node.
Node1.Out.Call = Node2.In

The same thing also happens for value ports as well, but instead this time it’s a System.Func instead. So for example when an AddFloats node ‘asks’ for FloatA.value, ‘.value’ returns directly the value of a method that is in the node which ‘FloatA’ is connected to.
Node2.FloatA.value = Node1.Result

Thus, there are also no runtime assignments. An output value is fetched when and only it is asked for.

When in Unity Editor only (#If UNITY_EDITOR), the connection is also added inbetween these getters to show the debug information as shown in previous post.

I hope I didn’t get very technical here :slight_smile:

Now, as far as adding in your own code, you don’t really have to create a custom node for that, since any function can be used as a node through reflection, which reflection is also turned into delegates through expression trees for performance instead of using Invoke which is slow.
So there is a single node which can wrap any function.

  • Each parameter of the function is turned into an input value port.
  • If the function has a return value, that value is turned into an output value port.
  • out parameters (like for example in Physics.Raycast hitInfo) are also turned into output value ports.

Here is what I mean. This is probably not how the UI will be in final, but for the shake of demonstration:

1874201--120448--Func.gif

This node can also wrap static functions, in which case the “instance” port will not exist.
Similarly, properties are also accesible.

Hello inas,

Yes, you are prety much spot on of the purposes of the current systems in NodeCanvas and this is a very interesting question :slight_smile:

This system started heavily inspired by XSI’s ICE as I state in the first post, which is purely a data flow editor. Later on ideas from unreal blueprint were introduced (basicaly event driven forward state-like excution. See In-Out ports ), for a more game development centric application.
As it currently stands, it will be a generic visual scripting editor, yes, but it might also change direction to become something more specific but certainly related to visual logic and/or behaviour.
For example if I see that it leads to creating fully spaggeti graphs to achieve something rather easy, then I might reconsider it’s purpose to be more meaningful, direct and usable.

No matter what direction it takes, it will certainly be fully compatible with the rest of NodeCanvas systems in one way or another. Having nested flow graphs is one way :slight_smile:

Now with that said, I never thought of any visual scripting tool as something to completely replace coding for a full project, but as something to bridge the gap of coders and desingers and organize thought into structure. At least to me, this is what a visual scripting editor is about :slight_smile:
This also prety much wraps the vision about this, as well as the other systems :slight_smile:

Hey,

I am prety much obsessed with instant readability myself :slight_smile:
Let me give you some answers regarding your requests:

  • Having the icon as a background could be something possible, but aligning will probably not be :slight_smile:
  • A node is a class, within where you define it’s functionality and looks. What do you mean by saving custom node?
  • I will take a look at switching icons. It might be easy.
  • A node can display anything (Unity GUI) inside it by overriding OnNodeGUI in the node class. So graphs, pies, bars can be possible.
  • By running a script you mean something like my first answer of this post where you can use any function as node?
  • Yep, this is prety much the “compounds” I’ve been talking about, showcased as a proof of concept in the first post. I really like this and I hope to manage achieving it :slight_smile:

Cheers everyone!

Looking cool. I like the debug values.
I personally would love something more like http://scratch.mit.edu. Where you’re mostly just assembling actual code (with real C# being written behind the scenes, which you can edit if you want) in a block format, which is unbreakable. You could also type and make it as fast as typed code.
You could then use the idea of ‘nodes’ and wires, more like functions. Where the execution path is a wire, but the individual blocks can just snap vertically like typed code execution path.
So you’d end up with ‘nodes’ of clusters of blocks, which are collapsible (just visibly, not separate), connected to each other and other components with wires.
I actually really hate Blueprints, even though I love modular environments for audio work. They’re way too messy without any real functionality to justify it. They all need more snapping and auto flow inference with the OPTION to split them apart into branching wires.

I meant that if you make a custom node, say an AND gate with 5 inputs, it would be nice to be able to post it somewhere so other people can use it as well.

Yes, but not just a Unity build in function, but a function I made myself.

Hey,

I am a heavy user of unreal blueprints so you have my attention.

Will we be able to make full games with this tool?

This is one of the MAJOR advantages of Unreal Blueprint Systems.

Integration with things such as Unity Networking/Unet and the like would also be appreciated :slight_smile:

Hello Vectrex,

Glad to see you here. You helped me quite a bit in NC :slight_smile:

What you are proposing here is a very different system altogether. It’s hardly a node based editor :slight_smile:
Furthermore I would like to let you know that there IS a Unity plugin like scratch. It’s called ‘plyBlox’. Haven’t used it but it looks cool.

I agree that Blueprints can become messy without justificaion, yes. I am trying to get that messyness away somehow, if at all possible. One way I am thinking, is to encapsulate code within nodes without much port exposition (2nd image in 1st post), thus simplifing the graph. But on the other hand port exposition allows for data flow in case scenarios like Elecman posted. In the end I think I am going to allow both ways. Low and High level.

Yes, of course. Once this is out, I will create a repository for nodes at the official site.
Also, yes, you can run any function including from your own code, not just Unity built in :wink:

Hello there,

This is a prety vague question, but as it stands now, you will be able to achieve what blueprint can. If you can make a full game wholy with blueprints, then this system will most probably allow the same.
Personaly though, I think that there is no such thing as creating a full game without writing a single line of code, unless it’s a very simple one :slight_smile:

Cheers :slight_smile:

nuverian,

I absolutely loves your idea of nodes repository! Is it open source? Maybe hosted on Github? Can the community contribute custom node or sample tutorial? NC will become strong platform if supported by many people :slight_smile:

Ps: my username is nascode on your NC forum :slight_smile:

Oh, hey nascode :wink: How the JRPG battle system is going?

I am still investigating the best approach of a nodes repository, so that it is definetely community driven.
One way is the per-se open source repository like you describe (github or bitbucket). More apealing to coders.
Another way is something from within a Unity editor window, possible to download/upload nodes. More apealing to non-coders.
(same thing for tasks as well)

The 2nd is more difficult for me, due to my limited networking knowledge :stuck_out_tongue:

Both have pros and cons, but the idea is the same :slight_smile:

Sounds like the thing you are working on is exactly what I need. Unfortunately it is not finished yet :wink:

I also had a look at uScript but it doesn’t support AND/OR/NAND/XOR/Switches either. Do you know something that is currently available which can do such a thing?

By the way, I hope you don’t go fully blueprint style. This is how NOT (no pun intended) to do it:

I am already seeing our engineers going WTF over this.