Run ISystem Source Generation on Custom Source Generation Code

Hello. This is my first post, so hopefully I’m doing this right.

I am currently writing some code that tries to time scheduled jobs by wrapping the job (pretty much all IJobEntitys) between two other jobs that start/stop a stopwatch timer. The issue is that I don’t want to have to go to every job in every system to set up profiling code and wrap each of them with jobs à la:

// This is what I don't want to do
// for every timed job in every system...
public partial struct MySystem : ISystem {

	// Profiler setup code
	private NativeText _profilerKey_LogicJob1;
	private NativeText.ReadOnly profilerKey_LogicJob1;
	private NativeText _profilerKey_LogicJob2;
	private NativeText.ReadOnly profilerKey_LogicJob2;
	// ...

	void OnCreate(ref SystemState state) {
		_profilerKey_LogicJob1 = new NativeText("Logic Job 1", Allocator.Persistent);
		profilerKey_LogicJob1 = _profilerKey_LogicJob1.AsReadOnly();
		_profilerKey_LogicJob2 = new NativeText("Logic Job 2", Allocator.Persistent);
		profilerKey_LogicJob2 = _profilerKey_LogicJob2.AsReadOnly();
		// ...
	}
	
	void OnDestroy(ref SystemState state) {
		_profilerKey_LogicJob1.Dispose();
		_profilerKey_LogicJob2.Dispose();
		// ...
	}

	void OnUpdate(ref SystemState state) {
		// Schedule and time LogicJob1
		JobHandle startTimingHandle1 = new StartTimingJob{ timerName = profilerKey_LogicJob1 }.Schedule(state.Dependency);
		JobHandle logicJobHandle1 = new LogicJob1 { ... }.ScheduleParallel(startTimingHandle1);
		JobHandle endTimingHandle1 = new StartTimingJob{ timerName = profilerKey_LogicJob1 }.Schedule(logicJobHandle1);
		// Schedule and time LogicJob2 (it depends on logicJobHandle1 finishing
		JobHandle startTimingHandle2 = new StartTimingJob{ timerName = profilerKey_LogicJob2 }.Schedule(logicJobHandle1);
		JobHandle logicJobHandle2 = new LogicJob2 { ... }.ScheduleParallel(startTimingHandle2);
		JobHandle endTimingHandle2 = new StartTimingJob{ timerName = profilerKey_LogicJob2 }.Schedule(logicJobHandle1);
		// ...
		
		state.Dependency = logicJobHandle2;
	}
}

I have tried coming up with solutions involving generic functions where the generic type is an unmanaged IJobEntity and moving code into a static helper class, but everything I try results in the entities source generator for the ISystem not picking up on the generic/static function being called.

The solution I then sought out was to perform source generation of my own. I followed some steps (mostly from here: https://www.youtube.com/watch?v=UiYQR8eQfgU&t=2460s) to set up and incorporate my own source generator into my Unity project. This allows me to set up job profiling in a system with much less code:

[JobProfiler(typeof(LogicJob1), "Logic Job 1")]
[JobProfiler(typeof(LogicJob2), "Logic Job 2")]
// ...
public partial struct MySystem : ISystem {

	// Profiler setup code
	// key fields auto-generated

	void OnCreate(ref SystemState state) {
		// Call auto-generated function SetupJobProfilers
		SetupJobProfilers(ref state);
	}
	
	void OnDestroy(ref SystemState state) {
		// Call auto-generated function DisposeJobProfilers
		DisposeJobProfilers();
	}

	void OnUpdate(ref SystemState state) {
		// Schedule and time LogicJob1 using auto-generated Profiler_ScheduleParallel(LogicJob1 job, ...); function
		LogicJob1 job1 = new LogicJob1 { ... };
		JobHandle logicJobHandle1 = Profiler_ScheduleParallel(job1, profilerKey_LogicJob1, ref state);
		// Schedule and time LogicJob2 using auto-generated Profiler_ScheduleParallel(LogicJob2 job, ...); function
		LogicJob2 job2 = new LogicJob2 { ... };
		JobHandle logicJobHandle2 = Profiler_ScheduleParallel(job2, profilerKey_LogicJob1, logicJobHandle1, ref state);
		// ...
		
		state.Dependency = logicJobHandle2;
	}
}

This way I have minimal setup and fewer changes to make to profile any arbitrary job in my ISystems’ OnUpdate methods. The problem I’m getting to is that this ends up not working with Unity.

I know the source gen at least partially works because my IDE (VSCode) provides options for source gen’d methods and Unity properly compiles when tabbing back in. The issue is that my custom source-gen’d functions are not further processed by Unity’s own source generation, resulting in Exception: This method should have been replaced by codegen during runtime calls to Profiler_ScheduleParallel. I can see the final source gen’d system output by Unity by adding the ‘DOTS_OUTPUT_SOURCEGEN_FILES’ define in the script compilation, but I cannot see the intermediate source gen’d system that should be output by my custom source generator. The final output also tells me that the custom Profiler_ScheduleParallel(…) function is not in Unity’s final generated version of the System as far as I can tell.

It seems like Unity needs to be able to find my custom source-generated code and run it’s own source generation on it to get it into a usable final format, but I am unsure about how to proceed with doing this.

My question is basically two-fold:

  1. Is there a way to get Unity to recognize my custom source generation output as something that should be further processed in its own ISystem source generation procedure? I.e. can I get Unity’s ISystem source generator to treat my custom source generated code as if it were original source code?
  2. Is there a way to see the intermediate source generation code produced by the custom source generation on my System?

Thanks, and any help is welcome.

Nope. Source generators run independent of each other and cannot depend on each other. Yes, it is really frustrating how Unity rewrites entire methods and doesn’t have an API for auto-populating an IJobEntity and then letting you use it as an IJobChunk type in your own generics with your own scheduling.

If you use incremental source generators, your IDE should be able to show it. Learn how to inspect the Unity’s generated code for either IJobEntity or IAspect in your IDE (ISystem doesn’t show in all IDEs), and then try the same workflow to inspect your own generated code.

1 Like

I see, so I won’t be able to do it that way.

I guess I could try to write a (very limited) source generator that produces code similar to that which Unity produces for this specific usecase. The key field generation still works anyways, so if the new approach fails, I’ll just have to manually edit the code in OnUpdate, I suppose.

Thanks for the pointers.

Do you actually need to manually profile jobs? The Unity Profiler already profiles them, if you want to see the default job profiling all you have to do is click on a point in time in the profiler, locate your system’s update on the main thread in that frame and then look under it to see the jobs running during it, including their name, duration and timing.

That’s a good point, and I do use Unity’s profiler to check the job runtime on each thread for my own debugging purposes. The real purpose which I didn’t touch on in the original post is that I am trying to make an in-game profiler for specific systems/jobs because I’m developing a game-maker of sorts. I want the user to be able to see if there are specific jobs or systems that slow down when they make a change.

If it is possible to get unity profiling data in game without causing too much bloat, I would be open to hearing suggestions for it, though.