To Linq or not to Linq

I’ve been exploring Linq recently and it is pretty cool how compact I can make my code with it.

That being said I read something about it being potentially dangerous if you don’t understand it fully, and I certainly don’t.

I’m curious whether I should continue down the Linq path and use it wherever I can, or if it is better to just use the standard iteration techniques I am more familiar with.

As far as I know, Unity deploy in .NET 2.0, which does not have LINQ.

Unity doesn’t actually use .NET at all. Instead, it uses Mono - which is an open source reimplementation of the .NET framework. Because of that, it’s not possible to directly compare Mono versions with .NET versions. However, I believe the closest comparison (assuming Unity 4.3) would be to .NET 3.5, with some features of .NET 4.0 thrown in.

Regarding your usage of LINQ… I wouldn’t consider it dangerous per-say, though it’s likely slower than other mechanisms that can accomplish the same thing. So, I’d recommend that you be conscious of where and how you use it - and only do so in places where it makes sense. I certainly wouldn’t use it in performance-critical portions of code.

Jeff

LightStriker I am already using Linq in Unity so it definitely is supported, although I have read some features are not fully implemented on all platforms, like the sorting options on iOS.

Ok makes sense, I will have to reevaluate my usage of it somewhat as I was initially using it to pull items from very large lists.

LightStriker, .NET 3.5 is a superset of .NET 2.0. It is still considered base .NET 2.0 but it adds features on top of it. The version of Mono that Unity 3.5+ uses is equivalent to .NET 3.5, but with a few things stripped out that Unity doesn’t support (such as System.Data).

That being said, LINQ works just fine. However, if you’re planning on targeting iOS which uses Full AOT, you need to be aware that there are some limitations that will cause LINQ to break with AOT errors. This is because of the way LINQ does some things dynamically and that it uses an expression builder for.

1 Like

Linq is awesome, will make you more productive, reduce maintenance costs, and increase your productivity over time as you start seeing problems as list comprehensions rather than loops and solve more problems using less code that directly expresses intent instead of having intent hidden by layers of imperative bookkeeping.

However, Linq allocates like crazy, which is pretty much the worst thing you can do in Unity due to the non-generational-and-generally-horrible garbage collector. This is the reason people avoid it, not because of any lack of understanding.

On that note, I’ve just released Smooth.Slinq, which provides a faster-than-Linq-and-allocation-free enumeration / comprehension API for Unity.

Asset Store Link

Asset Forums Announcement

1 Like

I use Linq all of the time in editor scripts and this works fine on both Windows and OS X.

Like Smooth P mentioned, Linq does allocate like crazy… this isn’t always a bad thing, but I would certainly avoid usage of Linq within your “Update” functions.

I do not know how true this next point is, but a while back I was led to believe that not all Linq functionality worked properly on certain mobile devices. I am uncertain on this, but it might be worth looking out for.

I’m still too old school. I’ve never had need for Linq in games, but I would like to know more why I would need Linq? what’s the use cases?

1 Like

It just makes your code cleaner, that’s all really:

// The following array contains a collection of numbers:
var someNumbers = GetRandomCollectionOfNumbers();

// Now, using Linq you can get all unique even numbers and sort ascending with the following:
var uniqueEvenSorted = someNumbers
    .Where(n => n % 2 == 0)
    .Distinct()
    .OrderBy(n => n)
    .ToArray();

You do not need to use ToArray, you could use ToList, or even just accept the IEnumerable result!

Linq doesn’t work in UnityScript though, you need to be using C# for this!

1 Like

Increased productivity, reduced maintenance, faster time to market.

Basically, get more done in less time with less code that declaratively expresses intent rather than being mired in imperative logic.

2 Likes

I’m assuming that by “allocates like crazy”, you mean creates objects on the heap quite a bit. Which it does… relatively speaking.

But compare some of those actions to the amount of memory that would have to be allocated in a none linq setting. And you’ll have comparable amount of memory usage. Sometimes lower for super simple linq statement, often times larger for things like sorting.

For example:

non-linq

foreach (var obj in someList)
{
	if(obj.SomeBooleanTest) //do stuff
}

linq

foreach(var obj in from o in someList where o.SomeBooleanTest select o)
{
	//do stuff
}

Sure the non-linq will be less, there’s no extra anonymous enumerable being created in memory.

But lets say that list needed to be sorted, made distinct, like in numberkruncher’s example.

// The following array contains a collection of numbers:
var someNumbers = GetRandomCollectionOfNumbers();

// Now, using Linq you can get all unique even numbers and sort ascending with the following:
var uniqueEvenSorted = someNumbers
    .Where(n => n % 2 == 0)
    .Distinct()
    .OrderBy(n => n);
//note I removed the ToArray because that processes the entire list immediately, I often don't need it as an array
//actually I use arrays very rarely in my code... when I do they're mostly static in size and don't change much if at all after construction... such as field members for a component that the inspector needs.

(I don’t even want to post the alternative non-linq code, it’d be so damn messy)

Well, that’ll require a bit more objects in created on the heap. Both in the linq and non-linq methods. Thing is… how efficient will the average programmers sort technique be? They’ll probably end up using the built in sorting algorithms supplied by .Net (just like linq will use), but done with arrays most likely. And will probably need copies of the arrays made as well and so on. You’ll be creating a lot of objects, taking up lots of memory, and having to manage all those objects.

Allocating like crazy…

And you have to do all the work yourself.

And then comes the last thing, which is the main reason I like Linq so much. If you do this… I bet you’ll probably write the code that does this all up front. Processing the list, sorting it, etc, all BEFORE you go and actually loop over it. Requiring a bunch of up front processing time.

The linq alternative, it processes as you need it. Say I loop through some linq statement, and I stop mid-way. It stops… it doesn’t process anymore, it doesn’t allocate anymore, it stops with me. I sort, and process half way through the list, it has no need to sort the back end of the list… so it doesn’t.

Sure all this functionality could be done, by hand, but will require a whole bunch of leg work, and is hardly readable. Causing you to create your own linq like library… like Smooth P did (which may or may not over come of the flaws in linq, and may or may not introduce its own flaws).

That last bit though, the processing of the list as I need it. That’s the part that sells me the most.

I also use linq in Update methods all over the place and barely see a performance hit relative to other options… but my linq statement also are in the realm of things that other options would be just as costly. And if the performance hit is so bad, but can be pre calculated on say load, I would do so instead.

2 Likes

Interesting, I didn’t realize that, thanks!

Note it hinges on the sort algorithm, which means where you stop it may have sorted further past it.

Watch this visualization of quicksort (the algorithm used by linq OrderBy):

https://www.youtube.com/watch?v=m1PS8IR6Td0

You’ll notice how it sorts over the collection and partitions it into groups, and then sorts each partition. When you break into a new partition, is when more sorting will be done. Say you loop up to where it’s at at 10 seconds, all that unsorted stuff afterward won’t ever be touched again. It’s been touched once, in a cursory way to partition out the list, but it won’t do the individual heavy lifting necessary to get it in perfect order.

Thanks for the explanations. It’s still kind of not typical in games and seems to be lower performance. I can see it being useful for editor scripts, or perhaps an RPG though. For action games, or games in general, I’m just not seeing these database style queries being a super workflow improvement over simple code. But that’s just my inexperience with Linq talking, no doubt. Granted, sorting the list seems easy. But sorting in realtime is the sort of thing you want optimised.

Oh another thing I really like with linq is the ‘from from from’ statements.

2 froms translates to non-linq relatively simple. But 3 froms translates to non-linq really messy. I had to write this one recently… so much nicer in linq:

var slips = (from s in _allSlips from bId in customer.BoatsOwnedByCustomer from b in s.BoatsInSlip where bId == b.Id select s).Distinct();

I had a collection of the boat ids that a customer owns, I had a collection of all the slips in the marina, and those slips have a collection of the boats in it (the slip could be linear and contain multiple boats). I had to find the slips that contain a boat that is owned by the customer who can own multiple boats.

Rewrite that as a for loop…

List<SlipInfo> slips = new List<SlipInfo>();

foreach(var slp in _allSlips)
{
	foreach(var bId in customer.BoatsOwnedByCustomer)
	{
		bool bAdded = false;
		foreach(var b in slp.BoatsInSlip)
		{
			if(b.Id == bId)
			{
				slips.Add(slp);
				bAdded = true;
				break;
			}
		}
		
		//if we added this slip, break so we go to next slip and remain distinct
		//note this assumes allSlips is distinct already
		if(bAdded) break;
	}
}

More readable in my opinion. Which has been brought up.

Note my code isn’t efficient, it could be made more so with out linq… but becomes even more unreadable.

This though is an example where linq is going to allocate more memory and take more processing to enumerate the entire list. But it’s a situation where I don’t do it frequently as well.

2 Likes

I would very much disagree!

Here’s some examples of my linq in my current game, linq I’d hate to write not in it, and operates very efficiently:

        private void OnEarlyUpdate(object sender, System.EventArgs e)
        {
            foreach (var go in from m in ApocMovementMotor.LiveObjects where m.gameObject.activeInHierarchy  m.IsEnabled()  !m.HasTag(Constants.TAG_PLAYER) select m.gameObject)
            {
                if (go.transform.position.y < TERMINAL_DEPTH) go.Kill();
				...
			...
            var prior = GameObject.FindObjectsOfType<GameObject>();
            yield return asyncOp;

            var objects = new GameObjectCollection((from o in Object.FindObjectsOfType<GameObject>().Except(prior) select o.transform.root.gameObject).Distinct());
			...
    void IMovementStyle.Activate()
    {
        _objectToLayerCache = new Dictionary<GameObject, int>();

		//GetAllChildrenAndSelf is an extension method with some deeper linq going on
        foreach (var go in from t in this.gameObject.GetAllChildrenAndSelf() select t.gameObject)
        {
            if (go.IntersectsLayerMask(Constants.MASK_MOBILELAYERS))
            {
                _objectToLayerCache[go] = go.layer;
                go.ChangeLayer(Constants.LAYER_PROJECTILE, false);
            }
            else if (go.IntersectsLayerMask(Constants.MASK_MOBILEHITBOXLAYERS))
            {
                _objectToLayerCache[go] = go.layer;
                go.ChangeLayer(Constants.LAYER_PROJECTILEHIT, false);
            }
        }

    }

This is in a platformer.

I suspect you’re right. For an RPG where the player might have a large stash of items, you could implement the meat of a “show me all of my shoulder armor pieces sorted by level, then quality” feature with one line of code. That sort of thing is less applicable to simpler games of course.

My game is a networked FPS, and believe me when I say nearly every class uses Slinq. Thinking of something like LINQ / Slinq as database queries hides what it’s real power is, and that’s for clean, declarative, high productivity / low maintenance collection enumerations and comprehensions. In general, if you’re doing stuff with collections and understand lambdas (and generics, though generic parameters can nearly always be inferred), using LINQ / Slinq will save you development time and reduce maintenance costs.

Part of the reason LINQ is often overlooked and/or misunderstood is because of Microsoft’s decision to equate LINQ with SQL / “queries” and use SQL inspired names for the operations, eg: Select vs Map, and the inclusion of the “query syntax” which is less powerful than chained method syntax.

Nearly all live, modern languages are moving towards a functional style of programming, including (non-ancient versions of) C# and Java. A good collection enumeration / comprehension API isn’t something you use once in a while, it’s fundamental to nearly all code.

Some common things that are loops or nested loops in imperative code, but become clean one-liners in Linq / Slinq:

  • Finding a specific element in a collection (First, Last, Min, Max, etc… and in Slinq these type of operations return an Option, which can be considered an enumerable of size 0 or 1, and provides its own chainable enumeration API)
  • Performing an action on each element of a collection (ForEach)
  • Folding / Aggregating a collection (Sum, Average, Aggregate, etc)
  • Doing any of these things to a filtered subset of a collection (Where, While, Take, TakeRight, Skip)
  • Doing any of these things to mapped collection (Select)
  • Doing any of these things to a collection of collections (SelectMany, Flatten)
  • Removing particular element(s) from a collection (Remove, Slinq only)

Sorting is a complex issue that is best left to another post, but it is important to point out that you may want to sort filtered and or mapped subset(s) of the original collection, that the total cost of sorting the subsets of a partitioned collection is cheaper than sorting the entire collection, and that the OrderBy() operations in LINQ / Slinq aren’t sloppy bubble sort code thrown together by a “how to make game” forum poster…