Neural net 2d representation.

m4l4 · January 15, 2021, 11:03am

Hi everyone, i know that this is probably the wrong forum to ask such a question, but i’m pretty sure that some of you had my same exact problem, and may lead me to a solution.

I want to click on my agent, and have his brain displayed onto a 2d panel in the upper right corner of the screen. Looked at some tutorials about making graphs and some UI, but they all seems too overcomplicated for such a simple result.

I find it both funny and weird, that i’m able to design, setup and train, complex 3d agents, but i can’t draw a bunch of colored circles and lines.

Have you ever made an UI to display NN? is it really such a complicated issue in unity?

mbaske · January 15, 2021, 2:50pm

My guess is you’ll need to read out Barracuda nodes. Haven’t tried that myself yet though.

github.com

Unity-Technologies/barracuda-release/blob/release/1.2.1/Documentation~/ModelOutput.md#introspecting-intermediate-nodes

# Working with outputs

After [executing a model](ModelExecution.md) you can then retrieve outputs or intermediate information.

## Fetching outputs
If the model has a single output, you can use  `worker.PeekOutput()`. Otherwise, provide output names:

```Csharp
var output = worker.PeekOutput(outputName);
// or
var output = worker.PeekOutput(); // warning: returns outputs[0]
```
**Note:** `worker.PeekOutput()` does not transfer ownership of the tensor to you: it is still owned by the `worker`. Calling `worker.PeekOutput()` is preferable and reduces memory allocations. However, if you expect to use the tensor for a long period, call `worker.Fetch()` . Otherwise, tensor values are lost after the next call to `worker.Execute()` or after the call to `worker.Dispose()`.


## Introspecting intermediate nodes

You can also analyse intermediate node values by passing a list of node names that you want to query when creating the `worker`.
```Csharp
// list of nodes to querry

This file has been truncated. show original

vincentpierre · January 15, 2021, 6:07pm

If you have a ONNX model, you can use https://netron.app/ to visualize the network and how all the pieces are connected together. I wonder what the use cases of visualizing the network in Unity would be, can you describe what you have in mind?

celion_unity · January 15, 2021, 10:49pm

Note that Netron also works for Barracuda (.nn) files.

m4l4 · January 16, 2021, 9:55am

thanks for your replies.
My actual projects involves a custom implementation of the NEAT algorithm. To make it work, i had to write a simple feedforward net, so i’m not using barrarcuda nor the unity ml functions.

Everything revolves around the concept of genome.
I have a list of node genes (neurons) and a list of connections genes (weighted connections), that describe the net topology. From there i generate the neural net, and i’d also like to generate a 2d depiction of it.

I’ve never worked with 2d nor UI, and i find graph tutorials to be way more complex than i expected.
Nothing impossible, but there must be an easy way to just draw 2d circles and lines on a panel.

Topic		Replies	Views
where should i start? Unity Engine UI-Toolkit	2	754	January 16, 2021
Simple way to visualize NN graph Unity Engine ML-Agents , Performance , com_unity_ml-agents	2	1113	October 19, 2020
Output of .nn model Unity Engine ML-Agents , com_unity_ml-agents	7	7114	April 20, 2020
Getting weights from model Unity Engine ML-Agents , com_unity_ml-agents	3	1152	August 13, 2021
ML Agengts Neural Network Architecture Unity Engine ML-Agents , Question , com_unity_ml-agents	2	1797	February 15, 2022

Neural net 2d representation.

Related topics