Hey everyone, I’m building a web application for the first time and I’m struggling to enhance my performance.
I’ll recap the function of this application for your understanding:
Theres a big CSV in the resources folder that contains data to draw lines with. This CSV is converted to a list of linedatas (= custom struct) on Start and then repeatedly checked each frame to draw specific lines from the CSV.
So far I’ve gotten this to work on a local server (using python) on Microsoft Edge, but not on other browsers such as Chrome or Firefox. From a previous forum post I’ve learned that this is most likely because of the memory used to draw these lines, and so I did some digging with the Unity profiler.
In the profiler I have 2 snapshots, one of the scene I’m trying to build succesfully (A), and another empty one (only camera with black background) (B)
If I’m reading this correctly it means my application is using 1.9GB, of which 0.9GB is native (used on system operations and running Unity itself). Does that mean my application requires 1GB (,which I’m aware is a lot) memory to run on its own or am I reading this wrong?
If I am understanding it correctly and it does currently require 1GB to run, how come only Edge is able to run it? I’ve heard that Chrome supports up to 1.5GB memory for desktop and 0.5GB for mobile devices.
Note: I’m making a linePool at start, then updating lines when needed to improve performance, this pool should be 170 lines, but is only 10 lines in the build I’ve tested here.
Sidenote: If you have any tips to reduce memory usage further, which would it be? I’m practically only using an overlay Canvas (TMP) and lineRenderers in this scene.
How many lines respectively line renderer objects are we talking about here?
For example, if you were to pool 10,000+ objects then that in itself could amount to significant memory usage regardless of the renderer purely due to the GameObject instance overhead (plus any components attached to it).
Also check with a single line renderer to see if Chrome can render it. If it can’t try using a different material/shader.
I assume you are using URP?
Also be sure to analyze a release build (see Player Settings).
Hi!
What memory settings are you using in the Unity - Manual: Web Player settings? In Unity 6 you can configure up to 4 GB of memory(Maximum Memory Size settings).
Can you share how your CSV parsing code looks like? If you have a lot of temporary objects and copying data around you might run into Web Platform specific memory limitations: Unity - Manual: Memory in Unity Web
If this is the root cause it might make sense to refactor the code a bit to have less allocations and reuse temporary objects instead of always allocating them within a loop body.
Another way to work around the memory limitations on the web platform is to use native containers that are not handled by garbage collection: Collection types | Collections | 2.5.3
The memory of those containers is immediatly freed when Dispose() is called or the object leaves a using scope.
Hey there, thanks again for replying to another of my forum questions. The object pool for the lines is only 10 gameObjects in this example, it should be 170 in the final build though. In my previous question you also recommended the lineRenderer test and since then I’ve confirmed a simple script to move a lineRenderer works succesfully on Chrome.
private List<LineData> lineDataList = new List<LineData>();
private void LoadCSV()
{
TextAsset csvFile = Resources.Load<TextAsset>("MCLines");
if (csvFile != null)
{
int lineCounter = 0;
string[] lines = csvFile.text.Split('\n');
foreach (string line in lines)
{
string[] values = line.Split(',');
if (values.Length >= 8) // Ensure correct number of values
{
lineCounter++;
float alpha = float.Parse(values[0]);
float beta = float.Parse(values[1]);
Vector3 point1 = new Vector3(float.Parse(values[2]), float.Parse(values[3]),
float.Parse(values[4]));
Vector3 point2 = new Vector3(float.Parse(values[5]), float.Parse(values[6]),
float.Parse(values[7]));
lineDataList.Add(new LineData(lineCounter, alpha, beta, point1, point2));
}
}
//debugText.text = "CSV Loaded, lines = " + lineCounter;
debugText.text = "CSV Loaded, Datalist count = " + lineDataList.Count;
}
else
{
debugText.text = "CSV Not found";
//Debug.LogError("CSV file not found!");
}
}
Here’s my code for reading the CSV, the function is called on Start().
I’ve checked my max memory and I used to have it at default (2048 MB), but I turned it down to 1500 MB because I thought that was Chromes limit on desktop. Didn’t change anything though. For Edge it works on both.
How many bytes of CSV are we talking about?
Splitting the file duplicates its memory. Splitting each line also duplicates each line’s memory. That means a 1 MiB file will consume 4 MiB while being “parsed”. Pretty sure Parse methods also create garbabe. And the parsed values also consume 36 bytes per entry. Unless you can rule out that this will not contribute significantly to memory usage, profile it.
Note that Parse can throw exceptions and you’re not try catching these. Perhaps this is causing the failure to render? Use TryParse to silently use a default value or catch those exceptions.
Also, much could be said about string.Split() on CSV alone. Unless you’re perfectly in control of the CSV data and the only person to ever edit this CSV file, it is quite likely to fail for the next user. Heck, it may even fail running it on a machine with different locale (possibly browser because it may behave differently than the OS).
Consider that different spreadsheet programs, even saving the same file on a machine with different locale in the same program, can result in the separator no longer being a comma but a semicolon or tab. Not to mention floating point values changing from 1.234 to 1,234 due to locale - although Parse will take care of the conversion, it does force a change in delimiter.
The CSV is quite a big list, 32593KB, so around 32MB. It doesn’t need to be readjusted anywhere though, so I don’t think that’ll be causing any issues. I’ve tried optimising each part of the scripts as much as I can, but I fail to see where I can optimise it further. I’ve tried using the StringReader to optimise the LoadCSV function, but it seems thats not enough.
Below is my full code for the LineManager, if you have any more suggestions please do share them.
using UnityEngine;
using System.Collections.Generic;
using System.IO;
public class MLineManager : MonoBehaviour
{
public GameObject mLine;
public GameObject mLines;
public int poolCount;
public Vector3 lineOffset;
public float currentAlpha;
public float currentBeta;
public float closestAlpha;
public float closestBeta;
public float lineLengthMultiplier;
public Gradient alphaGradient;
private List<LineData> lineDataList = new List<LineData>();
private List<GameObject> mLinePool = new List<GameObject>();
private float lastAlpha = Mathf.Infinity;
public float tmr = 0f;
public float interval = 0.1f;
private void Start()
{
currentAlpha = 0f;
currentBeta = 0f;
closestAlpha = Mathf.Infinity;
closestBeta = Mathf.Infinity;
LoadCSV();
SetPool();
}
private void Update()
{
tmr += Time.deltaTime;
if (tmr > interval)
{
tmr = 0;
UpdateLastLine();
}
}
private void LoadCSV()
{
TextAsset csvFile = Resources.Load<TextAsset>("MCLines");
if (csvFile != null)
{
// Use a StringReader to read the file line by line
StringReader reader = new StringReader(csvFile.text);
string line;
while ((line = reader.ReadLine()) != null)
{
string[] values = line.Split(',');
if (values.Length >= 8)
{
try
{
float alpha = float.Parse(values[0]);
float beta = float.Parse(values[1]);
Vector3 point1 = new Vector3(float.Parse(values[2]), float.Parse(values[3]), float.Parse(values[4]));
Vector3 point2 = new Vector3(float.Parse(values[5]), float.Parse(values[6]), float.Parse(values[7]));
lineDataList.Add(new LineData(alpha, beta, point1, point2));
}
catch
{
Debug.Log("Skipped line, value issue");
}
}
}
}
else
{
Debug.Log("NO CSV FOUND");
}
}
private void SetPool()
{
for (int i = 0; i < poolCount; i++)
{
GameObject newLine = Instantiate(mLine);
newLine.transform.SetParent(mLines.transform);
mLinePool.Add(newLine);
MLine lineScript = newLine.GetComponent<MLine>();
if (lineScript != null)
{
lineScript.SetValues(0, 0, Vector3.zero, Vector3.zero, lineOffset);
}
}
}
private void UpdateLastLine()
{
if (mLinePool.Count == 0) return;
GameObject lastLine = mLinePool[0];
LineData closestLine = null;
float minDistance = Mathf.Infinity;
foreach (LineData line in lineDataList)
{
float alphaDiff = Mathf.Abs(currentAlpha - line.alpha);
float betaDiff = Mathf.Abs(currentBeta - line.beta);
float distance = alphaDiff + betaDiff;
if (distance < minDistance)
{
minDistance = distance;
closestLine = line;
}
}
if (closestLine != null)
{
if (closestLine.alpha != lastAlpha)
{
lastAlpha = closestLine.alpha;
}
float colorFloat = (lastAlpha / (2 * Mathf.PI));
Color lineColor = alphaGradient.Evaluate(colorFloat);
LineRenderer lineRenderer = lastLine.GetComponent<LineRenderer>();
lineRenderer.material.color = lineColor;
MLine lineScript = lastLine.GetComponent<MLine>();
if (lineScript != null)
{
lineScript.SetValues(closestLine.alpha, closestLine.beta, closestLine.startPoint * lineLengthMultiplier, closestLine.endPoint * lineLengthMultiplier, lineOffset);
lineScript.DrawLine();
}
mLinePool.RemoveAt(0);
mLinePool.Add(lastLine);
}
}
}
Try taking a snapshot with MemoryProfiler before and after loading the CSV to get the details of your code’s memory usage. A 32 MB file with string processing can generate a surprising amount of garbage if not done carefully.
Does it have to be CSV? You could perhaps convert it to Json and just load it with JsonUtility.
I’ve taken some comparison screenshots in the profiler, in this one A has the lineManager script turned off and B has it turned on.
In this next one I’ve commented out the UpdateLine function for capture B, while A remains normal.
I think the file could become a Json since it doesn’t need changing, would this make it easier to load in in a memory performant way? If so, I’ll take a look into converting the file.
I would also like to note that aside from this LineManager script I also have a simple script that makes the camera pivot turn continuously and a Slidermanager that updates the currentAlpha and currentBeta of LineManager continously.
Thanks to everyone for the advice I’ve received so far around how to fix this. Since my last reply I’ve tried many things to optimise memory, including using a materialPool to assign colors. My new LineManager looks as follows:
using UnityEngine;
using System.Collections.Generic;
using System.IO;
public class MLineManager : MonoBehaviour
{
public GameObject mLine;
public GameObject mLines;
public int poolCount;
public Vector3 lineOffset;
public float currentAlpha;
public float currentBeta;
public float closestAlpha;
public float closestBeta;
public float lineLengthMultiplier;
public Gradient alphaGradient;
private List<LineData> lineDataList = new List<LineData>();
private List<GameObject> mLinePool = new List<GameObject>();
public List<Material> materialList = new List<Material>();
private float lastCSVAlpha = Mathf.Infinity;
private float lastLineAlpha = Mathf.Infinity;
private LineData closestLine = null;
private GameObject lastLine = null;
private Material currentAlphaMaterial;
public float tmr = 0f;
public float interval = 0.1f;
private void Awake()
{
currentAlpha = 0f;
currentBeta = 0f;
closestAlpha = Mathf.Infinity;
closestBeta = Mathf.Infinity;
LoadCSV();
SetPool();
}
private void Update()
{
tmr += Time.deltaTime;
if (tmr > interval)
{
tmr = 0;
UpdateLastLine();
}
}
private void LoadCSV()
{
TextAsset csvFile = Resources.Load<TextAsset>("MCLines");
int alphaIndex = 0;
if (csvFile != null)
{
// Use a StringReader to read the file line by line
StringReader reader = new StringReader(csvFile.text);
string line;
while ((line = reader.ReadLine()) != null)
{
string[] values = line.Split(',');
if (values.Length >= 8)
{
try
{
float alpha = float.Parse(values[0]);
float beta = float.Parse(values[1]);
Vector3 point1 = new Vector3(float.Parse(values[2]), float.Parse(values[3]), float.Parse(values[4]));
Vector3 point2 = new Vector3(float.Parse(values[5]), float.Parse(values[6]), float.Parse(values[7]));
lineDataList.Add(new LineData(alphaIndex, alpha, beta, point1, point2));
if (alpha != lastCSVAlpha)
{
alphaIndex++;
lastCSVAlpha = alpha;
float colorFloat = (lastCSVAlpha / (2 * Mathf.PI));
Color materialColor = alphaGradient.Evaluate(colorFloat);
Material material = CreateUnlitMaterial(materialColor);
materialList.Add(material);
}
}
catch
{
Debug.Log("Skipped line, value issue");
}
}
line = null;
values = null;
}
reader = null;
}
else
{
Debug.Log("NO CSV FOUND");
}
csvFile = null;
}
private void SetPool()
{
for (int i = 0; i < poolCount; i++)
{
GameObject newLine = Instantiate(mLine);
LineRenderer line = newLine.GetComponent<LineRenderer>();
newLine.transform.SetParent(mLines.transform);
currentAlphaMaterial = materialList[0];
line.material = currentAlphaMaterial;
mLinePool.Add(newLine);
MLine lineScript = newLine.GetComponent<MLine>();
if (lineScript != null)
{
lineScript.SetValues(0, 0, Vector3.zero, Vector3.zero, lineOffset);
}
}
}
private void UpdateLastLine()
{
if (mLinePool.Count == 0) return;
lastLine = mLinePool[0];
float minDistance = Mathf.Infinity;
foreach (LineData line in lineDataList)
{
float alphaDiff = Mathf.Abs(currentAlpha - line.alpha);
float betaDiff = Mathf.Abs(currentBeta - line.beta);
float distance = alphaDiff + betaDiff;
if (distance < minDistance)
{
minDistance = distance;
closestLine = line;
}
}
if (closestLine != null)
{
if (closestLine.alpha != lastLineAlpha)
{
lastLineAlpha = closestLine.alpha;
//Debug.Log(closestLine.alphaIndex);
if (closestLine.alphaIndex >= 360)
{
currentAlphaMaterial = materialList[closestLine.alphaIndex-1];
}
else
{
currentAlphaMaterial = materialList[closestLine.alphaIndex];
}
}
LineRenderer lineRenderer = lastLine.GetComponent<LineRenderer>();
lineRenderer.material = null;
lineRenderer.material = currentAlphaMaterial;
currentAlphaMaterial = null;
MLine lineScript = lastLine.GetComponent<MLine>();
if (lineScript != null)
{
lineScript.SetValues(closestLine.alpha, closestLine.beta, closestLine.startPoint * lineLengthMultiplier, closestLine.endPoint * lineLengthMultiplier, lineOffset);
lineScript.DrawLine();
}
mLinePool.RemoveAt(0);
mLinePool.Add(lastLine);
}
closestLine = null;
lastLine = null;
}
Material CreateUnlitMaterial(Color color)
{
Material newMaterial = new Material(Shader.Find("Universal Render Pipeline/Unlit"));
newMaterial.SetColor("_BaseColor", color); // "_BaseColor" is the default property for unlit shaders
return newMaterial;
}
}
With this program, I’m still running into memory leaks, more specifically pertaining to UnityObject.Material. I’ve ran the program and captured a snapshot at the start and a little after running. When comparing these two snapshots, nothing seems to appear, yet when typing “leaked” into the profiler it shows the following.
I’m very certain this script is solely responsable for my leaks, but struggle to find what I can change for the better.
Extra Info: There should be 360 unique alphas, so 360 materials, which I do see appear in the public materialList.
