i have a very vague conception of the data flow graph, and how it ties in with animation and dsp graph, but can the package be used to build neural nets? will it be used as a DOTS implementation of ml agents?
visually / conceptually they look sort of similar.
so would it work if you used set.SetData( ) as input, have the data propagate through the graph in Set.Update(), and then have the ai spit out a result? the graph would train itself by varying what each GraphKernel Node does to the inputs / outputs based on rewards and what not.
will DOTS in future allow you more direct access to the gpu for training purposes? or is the data flow graph completely not meant for this?
I think you want to look at barracuda which is the low level ML inference package on top of which higher level ML agents builds. It supports both burst & compute.
I think you are confusing the common pictures used to depict neural nets and the way a lot of programmers use graphs to chain code together. The lines in the picture you posted represent multiplication of a feature by a set of weights (basically each arrow is a multiplication operation. Since every circle is multiplied by every weight in the next layer (or vertical stack of circles), this type of operation is typically implemented very efficiently using vector multiplication ā an operation that allows neural nets to take advantage of things like GPUs.
The types of graphs someone using Unity typically uses (ie: shader graph; Iām not sure about dsp graph) are more like ways to visually represent a bunch of programming functions, so a programmer can easily see what is going on, and also do these types of things without having to write code. A very familiar type of graph that is used by Unity users is the graph used to create finite state machines which represent different animation states. These types of graphs are for end-user convenience more than anything.
You can use Barracuda.ModelBuilder APIs to create your network at runtime, if you wish. Kinda like the code you wrote above. Though normally you probably just want to import your trained network from ONNX file.
ML Agents are powered by Barracuda and they run using Burst on CPU and Compute shaders on GPU.
However training is not currently supported by Barracuda.
