Benchmark Group vs Array of Units vs List of Units vs always use a Filter

weaaddar

New Member
Reaction score
6
Here is a rather silly benchmark. However, assuming you have a group of units you are always keeping track of, you may want to know (for some inexplicable reason) what is the most optimal structure to hold these units. [Here's a hint as its been the winner in all my benchmarks thus far].


I'm not including Table as I've demonstrated that, it will be about twice as slow as the array variant. Nor am I including Set, as Set isn't an approved resource, and since it's enumeration pattern is that of a circular list, it'll probably perform exactly the same as linked list for enumeration.

The benchmark is two tests.
1. Create the collection of units you care about [N/A for the filter only pattern].
2. Then enumerate through the collection and set all unit's HP to 23.

To make the code simple, I've decided that I want to test enumeration of all units on the map, which just so happen to be 740 footmen I've created.

in the stopwatched section, each action is run 10 times, then the result is multiplied by 100. As the time of this excercise is microscopic due to the small amount of units available. I could've used a much larger set, but to make things easy, I've decided I didn't want to care about memory on the linked list, so I just create a new one, and say screw the old one.

Anyway the first benchmarking::
Comparison: Building up the set
[LJASS]group[/LJASS] vs [LJASS]array[/LJASS] vs [Ljass]List[/ljass]​
Each of these is using the native [ljass]GroupEnumUnitsInRect[/ljass],
With the following filter functions::
  • group used [ljass]null[/ljass]
  • array used
    JASS:
        function A()->boolean
    {
        U<s> = GetFilterUnit();
        S+=1;
        return false;
    }
    </s>

    [*] list used
    JASS:
        function B()-&gt;boolean
    {
         L.Add(GetFilterUnit());
         return false;
    }

To be extra safe, and silly, I decided that it might be worthwhile to avoid the call to Filter(function A/B), so I put these as variables.

Since if you only use a filter you never have to worry about construction, it is not included in this construction trial. Here is the result, everything is being compared against using a group as it won this trial.
Results (winner = [ljass] group [/ljass])
  • [ljass]group[/ljass]: 3.700 -- 100%
  • [ljass]array[/ljass]: 5.969 -- 161%
  • [ljass]list[/ljass]: 13.988 -- 378%

Okay now on to benchmark 2::
Comparison: Enumeration and applying a native.
[LJASS]group[/LJASS] vs [LJass]array (nes way)[/ljass]
vs [LJASS]array (normal way)[/LJASS] vs [Ljass]List[/ljass] vs [ljass]Filter[/ljass]​
You'll note the added contestants of array (nes way) and filter.
Here for each collection we went through each unit and set their health to 23. This is a somewhat plausible scenario.
The enumeration was done slightly different for each contestant.
  • [ljass]group[/ljass] used -- [ljass]ForGroup[/ljass] with a 1-liner callback of [ljass] SetWidgetLife[/ljass]
  • [ljass]array (nes way)[/ljass] used -- [ljass]for(j = S-1; j != -1; j-=1) SetWidgetLife(U[j],23);[/ljass]
  • [ljass] array (normal way) [/ljass] used -- [ljass]for(j = 0; j != S; j+=1) SetWidgetLife(U[j],23);[/ljass]
  • [ljass] list [/ljass] used -- [ljass]for(n = L.First; n != L; n = n.Next) SetWidgetLife(n.Object,23);[/ljass]
  • [ljass] Filter [/ljass] used -- [ljass] GroupEnumUnitsInRect[/ljass] with a similar 1 liner callback of SetWidgetLife as group did.

Results (winner = [ljass]array (nes way)[/ljass])
Here the results are displayed as percentages of the winner.
  • [ljass]group[/ljass] 6.508 -- 378%
  • [ljass]array (nes way)[/ljass] 1.538 -- 100%
  • [ljass] array (normal way) [/ljass] 1.587 -- 103%
  • [ljass] list [/ljass] 1.973 -- 128%
  • [ljass] Filter [/ljass] 6.673 -- 434%

Comments & Personal Criticism:
  • First an interesting result of the combination is that using the array approach of building then iterating is ~7.507. Which is only 12% slower then doing just a filter alone.
  • One can by looking at these results conclude that if you are only doing the enumeration against a collection of units once, its best to do it with a filter function. If you ever find yourself needing to do another enumeration against the same collection and you are a speedfreak, you should use this array approach.
  • Another thing to appreciate is that there is almost no noticeable difference testing in a loop against a constant or a variable. Linked list once again are slower then arrays.
  • I'm using zinc. The guidelines say use Vjass, but Zinc does make it a bit terser, and potentially harder to read then vJass. There are some shortcuts I'm exploiting that means that inadvertently Zinc might be doing silly, as Zinc is a higher level language then vJass which is more to the metal. Since zinc does translation of things like [ljass]for(;;)[/ljass] and [ljass]j+=1[/ljass] inefficiencies may be introduced. I checked the compiled jass and the exitwhen's did not include anything crazy. That said I'm using oneliner's which some might argue are only appropriate when coming out of the mouth of Schwarzenegger :p Again, though I find zinc much easier to work with and they both are mostly equivalent.
  • I'm using [ljass]TriggerSleepAction[/ljass] some say you should use exec function. I don't really like that approach. I feel sleep cleans the thread limit, and i don't have to worry about losing local variables. There is stack to worry about, but since I'm not using any stack variables in the function [ljass]OnEsc[/ljass] its not a big deal.
  • Linked list may have an unfair disadvantage. Since I used one letter variables to give the [ljass]array[/ljass],[ljass]group[/ljass] and [ljass]Filter[/ljass] the least worry about variable names, I didn't bother cleaning up the linked list library. As a result, linked list had the longest parse time of all the code, which may have had a very minor affect on the results. However, I still firmly believe that arrays will always be faster then linked lists.

Finally here is the library used to do this::
JASS:
    ///////////////////////////////////////////////
    // Native declarations for stopwatch natives //
    //  - Requires no modified common.j import   //
    ///////////////////////////////////////////////
    native StopWatchCreate  takes nothing returns integer
    native StopWatchMark    takes integer stopwatch returns real
    native StopWatchDestroy takes integer stopwatch returns nothing
// Why this is in Zinc
//! textmacro TestPattern takes name,code
    TriggerSleepAction(0);
    BJDebugMsg(&quot;Test Name::$name$&quot;);
    sw = StopWatchCreate();
        $code$
        $code$ 
        $code$
        $code$
        $code$
        $code$
        $code$ 
        $code$
        $code$
        $code$
    result = StopWatchMark(sw);
    StopWatchDestroy(sw);
    BJDebugMsg(R2S(result * 100.0));
//! endtextmacro
//! zinc
library Tester requires LinkedList
{
    group G = CreateGroup();
    unit U[];
    List L;
    integer S = 0; 
    
    function A()-&gt;boolean
    {
        U<s> = GetFilterUnit();
        S+=1;
        return false;
    }
    function B()-&gt;boolean
    {
        L.Add(GetFilterUnit());
        return false;
    }
    function C()
    {
        SetWidgetLife(GetEnumUnit(),23);
    }
    function D()-&gt;boolean
    {
        SetWidgetLife(GetFilterUnit(),23);
        return false;
    }
    
    public function OnEsc()
    {
        integer sw = StopWatchCreate();
        real result;
        group g = CreateGroup();
        integer i;
        integer j;
        Node n;
        filterfunc a = Filter(function A);
        filterfunc b = Filter(function B);
        code c = function C;
        filterfunc d = Filter(function D);
        BJDebugMsg(&quot;Test building&quot;);
        //! runtextmacro TestPattern(&quot;Use a Group&quot;,&quot;GroupEnumUnitsInRect( g,bj_mapInitialPlayableArea,null);&quot;)
        //! runtextmacro TestPattern(&quot;Use an Array&quot;,&quot;GroupEnumUnitsInRect(G,bj_mapInitialPlayableArea,a);&quot;)
        //! runtextmacro TestPattern(&quot;Use a List&quot;,&quot;GroupEnumUnitsInRect(G,bj_mapInitialPlayableArea,b);&quot;)
        BJDebugMsg(&quot;Use Filter&quot;);
        BJDebugMsg(&quot;N/A&quot;);
        //Okay now we actually need to create the real stuff we care about, since the first example generated junk. 
        // We don&#039;t need to worry about the group, as Enumerating through a group always clears it out.
        // I don&#039;t care about memory for this example. 
        //We should be okay for nodes with the list. And the array is fine provided we always stay with in [0,S)
        S = 0;
        GroupEnumUnitsInRect(G,bj_mapInitialPlayableArea,a);
        L = List.create(); // who cares about that old one. Destroying it might take a while...
        GroupEnumUnitsInRect(G,bj_mapInitialPlayableArea,b);
        BJDebugMsg(&quot;Test enumeration by setting life to 23&quot;);
        //! runtextmacro TestPattern(&quot;Use a group&quot;,&quot;ForGroup(g,c);&quot;)
        //! runtextmacro TestPattern(&quot;Use an Array (nes way)&quot;,&quot;for(j = S-1; j != -1; j-=1) SetWidgetLife(U[j],23);&quot;) 
        //! runtextmacro TestPattern(&quot;Use an Array (normal way)&quot;,&quot;for(j = 0; j != S; j+=1) SetWidgetLife(U[j],23);&quot;) 
        //! runtextmacro TestPattern(&quot;Use a List&quot;,&quot;for(n = L.First; n != L; n = n.Next) SetWidgetLife(n.Object,23);&quot;)
        //! runtextmacro TestPattern(&quot;Use Filter&quot;,&quot;GroupEnumUnitsInRect(G,bj_mapInitialPlayableArea,d);&quot;)
        
    }
    private function onInit()
    {

        integer i;
        trigger t = CreateTrigger();
        TriggerRegisterPlayerEvent(t,Player(0),EVENT_PLAYER_END_CINEMATIC);
        TriggerAddAction(t,function OnEsc);
        //BJDebugMsg(&quot;I&#039;m being called&quot;);
        L = List.create();
        for(i = 0; i &lt; 740; i+=1)
        {
            CreateUnit(Player(0),&#039;hfoo&#039;,0,0,0);
        }
        
    }
 
}
//! endzinc
</s>


LinkedList is the same library used in my collection shoot out, except Object is now a unit, it is about as optimal as a circular linked list can get in Jass (I'm welcome for suggestions of improvements).

Edit:
Zinc seems to break the spoiler tag. This is annoying. Sorry.

Edit again:
Per J4L's suggestion I've moved the LinkedList library here:

Here is the code, it is as optimal as the module just not a module. The only slight improvement is removing the count variable. [I didn't feel like making operators for it, and zinc lacks the [ljass]readonly[/ljass] keyword :/
JASS:
//! zinc
library LinkedList
{
    public struct Node
    {
        thistype Next;
        thistype Prev;
        unit Object;
        static method create(unit object)-&gt;thistype
        {
            thistype n = thistype.allocate();
            n.Object = object;
            return n;
        }
    }
    public struct List extends Node
    {
        integer Count;
        
        method operator First()-&gt; Node
        {
            return Next;
        }
        method Add(unit object) // not checking because it&#039;s faster.
        {
            Node node = Node.create(object);
            Prev.Next = node;
            node.Next = this;
            Prev = node;
            Count+=1;
        }
        
        method Fetch(unit object) -&gt; Node
        {
            Node node;
            for(node = Next; node != this; node=node.Next)
            {
                if(node.Object == object)
                {
                    return node;
                }
            }
            return 0;
        }
        
        method Contains(unit object)-&gt; boolean
        {
            return Fetch(object) &gt; 0;
        }
        
        method RemoveNode(Node node) -&gt; unit
        {
            unit retVal = node.Object;
            if(node != 0)
            {
                node.Prev.Next = node.Next;
                node.Next.Prev = node.Prev;
                node.destroy();
                Count -=1;
            }
            return retVal;
        }
        method Remove(unit object)-&gt;boolean
        {
            Node node = Fetch(object);
            if(node != 0)
            {
                node.Prev.Next = node.Next;
                node.Next.Prev = node.Prev;
                node.destroy();
                Count-=1;
                return true;
            }
            return false;
        }
        static method create()-&gt;thistype
        {
            thistype n = thistype.allocate(null);
            n.Next = n;
            n.Prev = n;
            return n;
        }
    }
}
//! endzinc
 

Vexorian

Why no custom sig?
Reaction score
187
why do you compare against a filter? I mean, array, reverse array and group all need to use a filter. So wouldn't you have to test Enum+filter vs Enum+filter+ForGroup vs Enum+Filter+List + Enum+filter+array ?

for should be fine for this. for(i=0; i<2; i+=1) translate to exitwhen i>=2
 

weaaddar

New Member
Reaction score
6
Yes, I know that Vexorian, because I think Zinc is great, and you won't go and do something like make while( x == 7) translate to exitwhen not x==7 like some other C language abstraction for Jass...

As for the test, the whole never using a group seems to be a popular paradigm for simple spells, because then you don't have to really worry about clean-up or memory leaks [you'll always suffer the cost of the group whatever it is].

I kind of already did that, each of them used Enumeration to build there initial set, the complexity of building was recorded and then i did enumeration through their respective means [i.e. for(;;) or ForGroup].

If you really want combined analysis you can always add the value they got in benchmark 1.

The more important point though as I suggested is that obviously Enum Filtering to do your dirty work is going to be great for 1 time things. For multiple hits its probably best to translate to an array. Or don't as this still completes really really fast. The Filtering enumeration for 740 units completed in like 7 ms (again results * 100, done 10 times so ~7 ms). If thats too slow your map, you are probably doing something really really wrong, and I doubt the gain of 5ms of performance will be significant.

Admittedly, This benchmark is disingenuous as I would never suggest doing anything but using a ForGroup or EnumUnits.... Just like iterating backwards through an array is faster then doing it a human readable fashion, the difference is minuscule, and may effect readability or usability of your map. If you are keeping a collection of hero units, I guess its probably okay to switch to something like AutoIndex with a UnitListModule and a unit module filter, but I'd be reaching if I'd say it'd be easier,faster or any more useful then AutoIndex and a group.

What was interesting for a short while I had the testPattern like this and it actually ran much worse. I'm guessing the introduction of the loop and endloop degraded the results, but I can't understand why. It would only run once. I guess Vex if you can somehow detect that the code is doing something retarded like do{}while(false) you can maybe optimize out the loop/endloop and exitwhen.
JASS:
do
{
    $code$
    /* 8 more code statements here */
    $code$
}while(false);
 

Jesus4Lyf

Good Idea™
Reaction score
397
LinkedList is the same library used in my collection shoot out
That code should be available somewhere, via link.

Interesting stuff. But you mentioned in it that an array of units is faster than a list of units to iterate through. In iterating through a set of integers (not units or handles), I find a list is actually faster. I've said that before, despite that it seems you keep saying it's slower - and I think it'd because you're not letting the value be the struct id. This is a downside of using a linked list resource (that isn't this).
 

weaaddar

New Member
Reaction score
6
My linkedlist is basically my dictionary submission sans hashtable actions.Since you wrote the tweaks, it's pretty much as fast as they get. Jesus4lyf can you actually do a benchmark to prove this point? As I stated the only benchmark I saw was from Grim001 over on wc3c that tested this::
[ljass]for(n = L.First;n != L;n = n.Next){}[/ljass] vs [ljass]for(i = 0;i != S; i+=1){}[/ljass]

edit: In fact I never saw the results. Grim just said hey 88% of linked list performance when done with an array. But he never posted the code or whatever.

You'll note that every single statement on the Linked list here inlines. This doesn't have anything to do with the array at all, all were testing is local set+array read faster then local set and comparing against a global/local. So the fact that the local set + array read was slower is balogne, it won but by a fraction of a percent.

In fact the jass may shed some light [but very little] as to why the linked list performed slightly poorer. It may just be that the library is too terse in naming. One letter library names may indeed perform better, via Vex's optimizer::
JASS:
//for(n = L.First; n != L; n = n.Next) SetWidgetLife(n.Object,23);
        set n=(s__Node_Next[(Tester___L)]) // INLINED!!
        loop
        exitwhen n == Tester___L
            call SetWidgetLife(s__Node_Object[n], 23)
        set n=s__Node_Next[n]
        endloop

Where as the for loop normal style translates to this::
JASS:
//for(j = 0; j != S; j+=1) SetWidgetLife(U[j],23);
        set j=0
        loop
        exitwhen j == Tester___S
            call SetWidgetLife(Tester___U[j], 23)
        set j=j + 1
        endloop

Since we can assume that the initial set isn't too costly [and if you'd like we can set the set to L.first to n = 2 so we're only doing a constant set], and the exitwhens are equivalent. We are now trying to make the claim that either [ljass]j+1[/ljass] is slower or faster than an [ljass]array[/ljass] read. I think integer addition is plenty fast myself. If we were to maybe renaming Next to N, object to O,and the struct itself to N maybe we'd seen tables turn, maybe.

But even if I were to concede that linked list iteration was faster, the population cost cannot just be wizzed away, but like I've said before, there are different collections for different jobs.
 

Jesus4Lyf

Good Idea™
Reaction score
397
Your LinkedList library should be in the first post.
One letter library names may indeed perform better, via Vex's optimizer
Based on my study of bytecode I do not believe it to be true for a moment. My understanding is that all var names are replaced by integer pointers, or so my findings seemed to display. You may follow the instructions there to study it some more yourself.

I optimised T32 by 6% by changing from an array to a linked list. I may look into the matter further some other time. :)

PS. Aside from that:
JASS:
set j=j + 1

I don't believe in incrementing while looping through an array - Cohadar once taught me to decrement instead and [LJASS]exitwhen x==0[/LJASS] and I don't see any reason to do it any other way (compare to literal ([LJASS]0[/LJASS]) instead of var in the exitwhen - of course this doesn't apply if the order is relevant, in which case use a list).
 

Vexorian

Why no custom sig?
Reaction score
187
The more important point though as I suggested is that obviously Enum Filtering to do your dirty work is going to be great for 1 time things. For multiple hits its probably best to translate to an array. Or don't as this still completes really really fast. The Filtering enumeration for 740 units completed in like 7 ms (again results * 100, done 10 times so ~7 ms). If thats too slow your map, you are probably doing something really really wrong, and I doubt the gain of 5ms of performance will be significant.
Well, just about every time I deal with enums I either need to do it just once. Or the contents of the rect/units in range of the point change too frequently to allow saving in an array or list so I have to use multiple enums anyway.
Admittedly, This benchmark is disingenuous as I would never suggest doing anything but using a ForGroup or EnumUnits.... Just like iterating backwards through an array is faster then doing it a human readable fashion, the difference is minuscule, and may effect readability or usability of your map. If you are keeping a collection of hero units, I guess its probably okay to switch to something like AutoIndex with a UnitListModule and a unit module filter, but I'd be reaching if I'd say it'd be easier,faster or any more useful then AutoIndex and a group.

I think the minimal speed increase of doing it in reverse might be because of allocation. Arrays' memory is allocated by powers of two.

The first time you do arr[747] you are allocating all the necessary memory, if done upwards, you are first allocating at arr[2], then at arr[4] , then arr[8] , etc...

Well, comparing to constant might also be the reason but somehow I don't think it might be that important.

I think that doing an arr[747 ] = 0 before the upwards array test would change the results a little.

JASS:
exitwhen == 0

A reason not to do it is that using 1-indexed arrays makes me want to kill myself...

Based on my study of bytecode I do not believe it to be true for a moment. My understanding is that all var names are replaced by integer pointers, or so my findings seemed to display. You may follow the instructions there to study it some more yourself.
I dunno. When Pipe was around it was already established that shorter names increase speed and he never seemed to suspect anything about it. I also remember he help grim show that locals are much faster than globals because of the hashing . It wouldn't make sense if he mentioned hashing if the identifiers are replaced with numbers...

What I know is that the speed difference has been tested beyond error. I also know that a map with 1000-long variable names gets very slow not only at compile time. there are always chances blizz fixed it in one of many patches, so feel free to test.
 

weaaddar

New Member
Reaction score
6
Now this thread is interesting. I demonstrated earlier that the [ljass]exitwhen j == s[/ljass] and [ljass]exitwhen j == -1[/ljass] are roughly equivalent (the decrement version won but like by 3%, both were roughly 1.5 ms for the whole operation). And I vaguely remember reading the same thread from PipeDream that said that long var names are slower.

Again though either way, J4L, array normal way or array decrementing way, both outperformed the linked list.

Here is the compiled decrementing version which is always doing a local var to a constant comparison vs a local var to a global comparison:
JASS:
        set j=Tester___S - 1
        loop
        exitwhen j == - 1
            call SetWidgetLife(Tester___U[j], 23)
        set j=j - 1
        endloop

Either way, we still are trying to say integer subtraction is slower then an array read. This sounds totally absurd to me. I guess the next benchmark someone should do is just like 10000 [ljass]j=j-1[/ljass] vs [ljass]n = array[n][/ljass].

And J4L, I followed your suggestion and have moved the linked list library to the first post.

I'm also with Vex, you should do it [ljass]exitwhen j == -1[/ljass] or [ljass]exitwhen j < 0[/ljass], because one based arrays are horrible. Lua's table starting at 1 really annoys me.

As for my tests I can try next time to do [ljass] U[740] = null[/ljass] and through an onInit for the node struct that initializes all variables of [ljass]thistype(8190)[/ljass] first to see if it makes a difference. (I don't think it will make a huge one).
 

Jesus4Lyf

Good Idea™
Reaction score
397
JASS:
exitwhen == 0

A reason not to do it is that using 1-indexed arrays makes me want to kill myself...
Put the exitwhen after actions.

>When Pipe was around it was already established that shorter names increase speed and he never seemed to suspect anything about it.
Odd. I might have to check that again, I remember seeing arrays were incremental integers or something in bytecode... perhaps that's a wc3 string pointer. o.o
Either way, we still are trying to say integer subtraction is slower then an array read.
No, when the data is the node, the list simply removes a single "-1".
JASS:
local node n=first
loop
    exitwhen n==0 // exitwhen int==literal
    set n=n.next // array read, var set
    call DoThings(n) // local read
endloop
// versus
local integer i=max
loop
    exitwhen i==0 // exitwhen int==literal
    set i=i-1 // -1, var set     &lt;-- only difference, has an extra -1.
    call DoThings(node<i>) // local read, array read
endloop</i>

It is not possible (logically) for it to be slower, and in my tests it was 6% faster (no code right now, speed is relative to what code is in DoThings, ofc).

Hope this sheds light on the matter.
 

weaaddar

New Member
Reaction score
6
That is rather constructed scenerio. Assuming you have data that needs to be grouped and you have an array of structs, you should use an array struct so you don't suffer the dereferencing cost.

Either way, I think its easy enough to say that an array is going to be faster. If you have an array of units, and a linked list of units, because of the way the moving forward is done, the array is just going to be faster. If you have a bunch of structs that you need to have a collection of, and you have a fixed amount of them you can probably get away with an array struct.

And that code you posted only works for 1-based arrays. Change it to exitwhen i==-1 or move it to the end.
 

Jesus4Lyf

Good Idea™
Reaction score
397
>That is rather constructed scenerio.
I actually consider it the rule, not the exception.

>Assuming you have data that needs to be grouped and you have an array of structs, you should use an array struct so you don't suffer the dereferencing cost.

Huh?

>If you have a bunch of structs that you need to have a collection of, and you have a fixed amount of them you can probably get away with an array struct.

An array struct does not help. When a struct with a lower index is destroyed, you will have a break in the iteration if you try to iterate through them. Else you must move all members, and re-attach it to everything it's attached to. A linked-list is truly actually faster.

>And that code you posted only works for 1-based arrays. Change it to exitwhen i==-1 or move it to the end.
I don't feel that's directly relevant to this discussion.
 

weaaddar

New Member
Reaction score
6
Destruction isn't always a common occurence. But yes in your scenario it's faster. In the described case above an array is faster (i.e. array of units vs list of units). Sometimes arrays make sense, other times linked-list make sense. let's agree to say they're both fast enough and really using either is probably good enough.
 
General chit-chat
Help Users
  • No one is chatting at the moment.

      The Helper Discord

      Members online

      No members online now.

      Affiliates

      Hive Workshop NUON Dome World Editor Tutorials

      Network Sponsors

      Apex Steel Pipe - Buys and sells Steel Pipe.
      Top