Limits of arrays or variables per script ? JS

Unity Engine
1

Hello, i’ve found something weird.

I have a script that is containing arrays to store variables about planets and stars in my game. There’s a lot of variables for each planet and the game can support 1000 planets running smoothly at 70FPS, so most of those arrays have 1000 slots, those for planets and a few, for stars have 100 slots. (the game generate up to 100 solar systems with up to 10 planets per system) I was adding more arrays to the script, for buildings count on planets and suddenly, without any error message, the scenes wich have this script could not be played anymore in editor, making unity close when hitting play, without any error message, but i could build and run the game. It happens when i have more than 105 arrays. If i stop at 100 and add others arrays in another script, i have no problem and the scene can be played in editor. I could not find anything on the subject, so the question is Is there indeed a limit for the number of arrays that can be used in a script, as it seems, or a limit for variables, as my 100 arrays are, in fact a bit less than 100000 variables ? The weird thing is that there’s no error message.

2

I don’t know of any limit, but I would strongly suggest not using so many arrays anyway, just because that seems like it would be a nightmare to maintain. I expect you want to use a better method of organization, such as a single multi-dimensional array instead of a ton of separate arrays.

–Eric

3

I can’t imagine it being a hardcoded limit, the size of those arrays isn’t anything ridiculous. Without seeing the code, I can only guess, maybe you’ve got a loop that’s becoming an infinite loop for some reason when you have more than 100, thus generating arrays infinitely until memory is full and the editor crashes. Go over your array initializations and make sure the loops can’t become infinite in some way you didn’t anticipate. Odds are good that there’s some problem with your initialization of your arrays.

I wouldn’t recommend multidimensional arrays if you can avoid them; they’re high on overhead, especially when they get big. You might want to consider using lists rather than arrays, if you haven’t looked into other ways of managing your objects.

4

It can’t be an infinite loop problem as there is no loop in the script. In this script i’m just declaring arrays (wich are static arrays, modified via, for exemple the script that is on planets). And the new arrays i was adding after 100 were not linked yet to any other script, so no loop trouble there also, it’s just that if i’m declaring more than 100/105 arrays, the script is bugged. And no problem with the declaration in itself as if i paste those new arrays in a new script and adding this new script to the scene, wich i’ve done, It’s working fine. It’s really weird, Unity script seems really capped to 100 arrays or 100000 slots. I find it quite easy to have one array for each stats, thing are clear in that way, i just have to make sure i refer to the good array. The good news is that it’s working if i use multiple scripts to declare my arrays, so i can continue this way, the bad news is that it’s a bit more complicated to write code refering to planets stats as i have to refer to different script for variables that i should be able, in my opinion, to store in a single script.But i’ll have a look to Generic lists, it seems that i could use them instead of arrays, the trouble is that it would require quite a bit of work to change this now.

5

This code is working for me:

	public int[,] arrays = new int[1000,100000];

	void Start () {
		for (int i = 0; i < arrays.GetLength(0) ; i++) {
			for (int x = 0; x < arrays.GetLength (1); x++) {
				arrays[i,x] = 100000;
			}
		}
	}

This demonstrates a multidimensional array in C#, effectively creating a multidimensional array of 1000 arrays with 100000 values assigned to 100000, or something like that. It takes a few seconds to get running if this script is on something, but it does work. I’m guessing that what you’re getting is a problem with running out of memory. For example, if we take this array and expand it to a third dimension, like this:

	public int[,,] arrays = new int[1000,100000,1000];

	void Start () {
		cameraInUse.transform.Translate (0,0,-distance);
		currentZoom = -distance;
		for (int i = 0; i < arrays.GetLength(0) ; i++) {
			for (int x = 0; x < arrays.GetLength (1); x++) {
				for (int y = 0; y < arrays.GetLength (2); y++) {
					arrays[i,x,y] = 100000;
				}
			}
		}
	}

You’ll probably get an out of memory exception, unless you have some beast of a machine. I’d guess that what you’re doing is causing an out of memory exception that Unity isn’t able to anticipate, thus making it crash when you try to run it.

Try to consider how you’re managing your arrays so that they’re not all loaded into memory at once (putting them in different scripts on different objects is one way), or just get rid of them and find a more memory efficient way to do what you want.

6

So to get the technical limits out of the way…

A 1-dimensional array’s length has a limit, and it’s 2^32 - 1… that’s because an array stores its length as a 32-bit integer.

A class (or script, which is a class) technically has a limit around like 2^16 (~65k) members. And the intermediate language, and how it works, allows for something like 2^24 members per module (which all types get packed into… usually 1 module per assembly… I don’t know that well, not my area of expertise).

No matter what though, these numbers are extremely high. Just a single integer array at its maximum size (no need for multi-dimensional array here), you’re at 16 gigs of memory used.

Memory limits tend to get in the way long before you get in the range of the max values of the things. 1-dimensional arrays on a high memory rig being the one exception (though we’re talking my rig at 32 gigs of memory could only really handle 1 full array… the second could only fill partially, since I also need memory for my OS and applications and whatnot).

But those memory limits are under the expectation that you’d even be allowed access to all that memory. Which you’re not. The .net/mono runtime doesn’t allow you access to all of it, and depending on the system creates you a much lower roof. This roof can be as low as 2gigs for a 32-bit application.

Then there comes in technical limitations of arrays themselves. And array in memory is a “contiguous” block of memory. This means that when allocating for an array it needs to find as many bytes as the array needs available IN A ROW. An array of integers only 1000 in length needs to find 4000 bytes (+ 8 or so more bytes for id and length) all sequentially available in memory. You have 100+ arrays like this (thankfully those 100 arras don’t have to be available sequentially).

This is where the technical limits of classes becomes fuzzy. Sure the limits allow for an array of 2^32 -1, sure a class can support 2^16 members. But when we go to allocate these objects… will a block of memory be available to fit everything appropriately. The .Net/Mono memory management system has it’s ways of doing so (and I barely know everything about it, let alone its nuances, let alone the difference in how Mono does it compared to .Net). For example, members of a class, depending on how declared, can sometimes be restricted to with in a contiguous block allowed for the class, because that the way the memory manager decided to deal with that chunk of memory and can’t find a chunk of memory to work with. This can lead to slow down, or straight up locking up until something can be found to fit all this.

And lets not get into the garbage collection of that memory. Like in your code, do you create new arrays to replace old ones when you change stuff? That starts taking up more and more memory until GC can clean up the old ones… which can be rather late. Try recyclying/reusing your arrays.

One of the things I do usually… use classes to organize that memory. You have 100+ arrays of 1000 items each in ONE CLASS. That’s a nightmare… why is it all in one class?

7

here is how i declare my arrays, with the example of one ( wich keeps the numbers of Spaceyards that are builded on each planet for the race Human, Imperium), there are all on the same model, with a 0 initial value in each slot but for the array of the names of the planets that contain 1000 strings, the names of the planets. I’ve just been copying and pasting this line, changing the name of the array :

# pragma strict
static var BuildPlanetsImperiumSpaceYard : Array = new Array(1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0); //More than 1000 slots

So there’s not even a specific class, but the script, i’m just declaring those arrays at the beginning of the script (wich don’t even have a start function, nor an update function. Arrays are not replaced by others when changing values, i’m just replacing the values in it. It just take around 2 to 3 second to load the scene where this script is, wich is the main scene of my game. Then it’s running smoothly with still lots of memory available at 70FPS, there is no update function used to acces those script (my game is turn based so in fact there is some update functions that acces the script, but only when you press the Next turn button, wich take then a few seconds to process before you can play your next turn). But well i’m quite new to Unity and JS, so the way that i declare my arrays may be an heresy. But if you’re right, i only need 400kB of memory to declare 100 arrays, wich is not really much why would there be a memory exception for like 105 arrays, wich is like 420kB ? I have 8 GB memory, therre should be no problem…

8

I need to clarify for you… a script is a class. Specifically, a script is a class that inherits from MonoBehaviour.

I also went over memory issues. It’s not just an upper limit, but also finding a contiguous block of memory as well.

Do note though, that may not actually be your problem. You could be having numerous other problems. We don’t know what your code is, and you’ve admitted you’re very new to this, and you’re using practices that are very naive. This leads me to believe you probably have some very shaky code under the hood…

9

Rather than keeping all this in a central tracking system with arrays, you’d want to have the game keep track on each planet of whether you’ve built a starport there. Essentially, just a boolean on each planet object that will be true if a starport has been built. Then, you would have your management script query if a relevant planet has a starport. It’ll be a lot lighter on overhead, and should avoid the problems you’re having. The approach you’re taking is heavy on work from your end, and super inefficient. JavaScript isn’t my forte, but in C# it’d look something like this:

public class Planet : MonoBehaviour
{
    public bool hasStarPort = false;
    public bool hasCapitol = true;
    public bool hasParkingLot = false;
}

With this script attached to each planet in the game, you can easily find out if a specific planet has a starport, without any need to resort to arrays. You don’t even need to keep an array with all your planets in it; you can do a sort of FindPlanetByName method, which will allow you to only keep track of planets as you need to. And, big bonus, each of those booleans takes up a miniscule amount of memory compared to storing the states of the planet in integers, which is what I assume JavaScript translates all those 1s and 0s to.

10

Well so admitting it’s a memory issue, wich i think it is, what is the size of max contiguous block of memory ? Maybe that I go over that limit declaring more than 100 1000 slots arrays in a class. And is this limit the same on all systems or this limit can be different depending on the system ? (meaning that my game can run fine on my computer but may not work on other computers ) And how can you explain the fact that the game was not working in the editor with 105 arrays in the script but was working if i builded and run the exe?

11

The editor creates a lot more overhead, so if you run it in the editor, you may run into memory problems like this that you won’t in the built version. Just because it will run doesn’t mean it should run, however; your 8 gigs of RAM may be fine for creating this thing, but is 8 gigs of ram really the minimum requirement you want?

Trust the experts, your methodology needs a new approach.

12

I’ll try to work around the problem and to take another path, maybe with variables on the planets as you said. But it can’t be boolean as i want to be possible to build multiple instances of buildings on each planets, like 8 research centers or 220 farms. But i’ve done some testing and it appears that my computer use around 35% of its memory for windows, Unity and other stuffs and that i’ve a peak at around 40% usage when the scene with this script is loaded with the exe, but just a peak, then it’s returning back to 35 to 37%. So the scene is taking no more than 5% of 8GB, and just when loading, then it’s 1% to 2%.

13

The point isn’t really the data type, integers will just do fine, the point is that the approach I presented is the accepted, object oriented approach. Rather than keeping all your data in a single ginormous object stored in a necessarily ginormous contiguous space, you split it into a bunch of small objects that are stored wherever they fit. It takes advantage of multi-thread processing and a whole bunch of other nifty things; you’re working with an object-oriented language (and engine, for that matter), so you need an object-oriented approach.

I know it sucks to scrap a bunch of code you’ve worked hard on, but part of learning sometimes involves learning a new concept and throwing a bunch of code out the window rather than trying to shoehorn your code in until you make it work through a bunch of shaky workarounds. If you build a building on a bad foundation, you tear it down and make a new foundation, you don’t try to fix it by building more stuff on the bad foundation.