Tutorial How to use Groups without leaking

Jesus4Lyf

Good Idea™
This tutorial is short and straight to the point. Everything I say has been tested by me and I put my word to it.

Groups are a dilemma due to leaking RAM (I say RAM specifically because it does not leak handle ids).

What leaks?

Enumerating with a null boolexpr leaks. (Example in spoiler.)
JASS:
function FTRUE takes nothing returns boolean
    return true
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local group g=CreateGroup()
    local integer i=1000
    loop
    set i=i-1
    exitwhen i==0
        call GroupEnumUnitsInRange(g,0,0,50000,null /*"null" Leaks*/)
    endloop
    call DestroyGroup(g)
    set g=null
endfunction
Destroying a group that has had an enumeration called for it leaks RAM. (Example in spoiler.)
JASS:
function FTRUE takes nothing returns boolean
    return true
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local group g
    local integer i=1000
    loop
    set i=i-1
    exitwhen i==0
        set g=CreateGroup()
        call GroupEnumUnitsInRange(g,0,0,50000,Filter(function FTRUE))
        call GroupClear(g) // irrelevant, really
        call DestroyGroup(g /*Leaks because "g" has had an "Enum" called on it*/)
        set g=null
    endloop
endfunction

What should I do?

This is a magic snippet:
JASS:
globals
    group GROUP=CreateGroup()
endglobals

I use this in all my maps. Inside any enum function, we may place a filter. We can use this to execute code, and return false so no units are ever added. Let's say we want to heal every unit on the map for 200 health.

This is how you may be used to doing it:
JASS:
globals
    // values to carry to the DoThings function
    real AmountToHeal
endglobals
function DoThings takes nothing returns nothing
    call SetWidgetLife(GetEnumUnit(), GetWidgetLife(GetEnumUnit()) + AmountToHeal
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local group g = CreateGroup()
    call GroupEnumUnitsInRect(g, bj_mapInitialPlayableArea, null /*leak*/)
    set AmountToHeal = 200.0
    call ForGroup(g, function DoThings)
    call DestroyGroup(g /*leak*/)
    set g=null /*at least this doesn't leak*/
endfunction


Implement the magic snippet. Now, examine this code:
JASS:
globals
    // values to carry to the DoThings function
    real AmountToHeal
endglobals
function DoThings takes nothing returns boolean
                                      /*nothing becomes boolean*/

    call SetWidgetLife(GetFilterUnit(), GetWidgetLife(GetFilterUnit()) + AmountToHeal
                     /*GetEnumUnit() becomes GetFilterUnit()*/
    
    return false
  /*return false appears here*/
endfunction

function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    // Store values before the enum instead
    set AmountToHeal = 200.0
    // Note the next line.
    call GroupEnumUnitsInRect(GROUP, bj_mapInitialPlayableArea, Filter(function DoThings))
    // The end.
endfunction

Explanation for those who don't understand:
I will explain the last line. We must put something in the filter. We could put in the following if we like:
JASS:
function ReturnTrue takes nothing returns boolean
    return true
endfunction
//...
call GroupEnumUnitsInRect(GROUP, bj_mapInitialPlayableArea, Filter(function ReturnTrue))

But then we need to clear our group later, or use a FirstOfGroup loop (largely deprecated). This way we never even actually add the units to the group. This method is only useful when you do not need to store the group with units in it for any period of time, ie. you are not storing a group of units, but rather just want to do things for a bunch of units, like in the example.

The rest of the comments explain the process of changing the callback that usually goes into ForGroup into a filter for the Enum instead. Be sure not to return nothing. It must return boolean or bad things will happen (untested personally, but apparently it can cause desynchs or something).

If you need to store a group of units, for example, all units that a spell has hit so far, you can use dynamic groups, but not with [LJASS]CreateGroup[/LJASS] and [LJASS]DestroyGroup[/LJASS]...

To use dynamic groups, groups should be recycled instead of destroyed. I recommend Recycle to recycle groups. Download this snippet and install it into your map. Then replace:
[LJASS]CreateGroup()[/LJASS] with [LJASS]Group.get()[/LJASS]
and
[LJASS]call DestroyGroup(g)[/LJASS] with [LJASS]call Group.release(g)[/LJASS]​

This will reuse the groups and store them in a list until they are reused. The groups are cleared before they are stored, so there is no difference in use to destroying the group, except you should be careful not to store a reference to the group and do something to it after releasing it (that would be hard to debug).

Thus we solve the leak that occurs when we destroy a group that has had an enum called for it.

What else should I know aside from RAM leaks?

Any Enum call clears all units from a group before it adds units to the group. To enum units in a group and keep the previous units, instead use your global GROUP variable, and in the filter, add the units to the group you wish to add to.

Adding a unit to a group does not leak its handle id if it is removed from the game.

Removing a unit from the game does not remove its reference from the group! This is called a "shadow reference". Now, from my understanding, it is generally accepted that a shadow reference is never included a ForGroup, but I think I found that it can be if you call the ForGroup immediately after removing the unit from the game. Shadow references take up a little RAM, so you should clean out a group occasionally. This can be done with this simple snippet written by CaptainGriffen:
JASS:
library GroupRefresh
    
    globals
        private boolean clear
        private group enumed
    endglobals
    
    private function AddEx takes nothing returns nothing
        if clear then
            call GroupClear(enumed)
            set clear = false
        endif
        call GroupAddUnit(enumed, GetEnumUnit())
    endfunction
    
    function GroupRefresh takes group g returns nothing
        set clear = true
        set enumed = g
        call ForGroup(enumed, function AddEx)
        if clear then
             call GroupClear(g)
        endif
    endfunction
    
endlibrary

Simply call GroupRefresh on a group to clear out the shadow references. This is only needed when you have a permenant group in a map that keeps track of units, and only needed to free up some RAM. It is O(n) complexity, so don't spam its use if possible. (I never use it personally, unit attachment solves this in O(1) complexity, but that doesn't belong to this tutorial.)

There is a deprecated thing called a FirstOfGroup loop. It works like this:
JASS:
globals
    GROUP group = CreateGroup()
endglobals
function ReturnTrue takes nothing returns boolean
    return true
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local unit u
    call GroupEnumUnitsInRect(GROUP,bj_mapInitialPlayableArea,Filter(function ReturnTrue))
    loop
        set u=FirstOfGroup(GROUP)
        exitwhen u==null
        call GroupRemoveUnit(g,u)
        // Do things to u
        
    endloop
endfunction

This is not as efficient as using a filter for your actions, as a general rule. It is also vulnerable to shadow references if the units have been stored in the group for any period of time (in other words, is not done instantly after an Enum generally). It also requires you to make some sort of filter, even if you don't need it, otherwise you will have a leak due to the null filter (or boolexpr). In short, it is pointless to use this method, except it gives you access to your local variables so you don't need to carry things over. I recommend you do not use it, but you are welcome to all the same.

GroupClear clears all references out of a group, including shadow references.

You should never need to use GroupClear on your global GROUP, because Enum calls clear it anyway, and you should never need to actually add units to it.

Summary, Please!

Use
JASS:
globals
    group GROUP=CreateGroup()
endglobals

for doing things to a bunch of units, and put your actions in the Filter, returning false at the end. (Example: heal or damage an AoE of units.)

Use Recycle for tracking a bunch of units for a time. (Example: record what units have been hit by a spell.) Release your group at the end using Group.release(groupVariable), and I'd recommend to null your references to it unless you trust yourself not to accidentally use them (or else you can end up with issues which are hard to debug).

If you have a permanent global group for tracking units, use GroupRefresh on it occasionally to free some RAM up. :thup:

You can now use groups without leaks, in efficient, sensible ways. :)
 

Frozenhelfir

set Gwypaas = Guhveepaws
GroupUtils appears to do the same thing as recycle and refresh (recommended to me by Viikuna). Also, how do we use the dynamic groups to store something hit by the spell? You mentioned them, and the why, but how? :( This is the problem I had with my spell, so I had to use the FirstOfGroup loop instead of putting it in the filter.

Didn't know groups leaked with enums on destroy... Time to go [del]recode[/del] find/replace my map again...
 

Viikuna

No Marlo no game.
Perfect.

You should probably add link to GroupUtils.

I know you probably prefer some system like Recycle instead, but most of this information originates from that thread and Griffens GroupRefresh thread, so link would be nice.

edit. To Guy with Polar Bear avatar. FirstOfGroup loop is just fine, if you do it like Jesus4Lyf shows in his tutorial. ( Right after GroupEnum, so theres no possible shadow references )
 

Jesus4Lyf

Good Idea™
If you want to track which units have been hit by a spell, add them to the group. Then you use IsUnitInGroup to see if it has been hit. Didn't think it was general enough to include. Can't code every way groups can be used here.

So let's say you write carrion swarm, you keep using GROUP to enum units in range of projectile, in the filter you check if they're in the dynamic group (you probably attach it to a struct) and add it to the group if it is not, as well as performing checks for ally/enemy, dealing damage, etc.

Here's why I hate GroupUtils:
JASS:
function ReleaseGroup takes group g returns boolean
    local integer stat = Status[GetHandleId(g)-MIN_HANDLE_ID]
    if g == null then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Null groups cannot be released")
        return false
    elseif stat == 0 then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Group not part of stack")
        return false
    elseif stat == 2 then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Groups cannot be multiply released")
        return false
    elseif Count == 8191 then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Max groups achieved, destroying group")
        call DestroyGroup(g)
        return false
    endif
    call GroupClear(g)
    set Groups[Count]                        = g
    set Count                                = Count + 1
    set Status[GetHandleId(g)-MIN_HANDLE_ID] = 2
    return true
endfunction

You gotta admit, that's kind of lame. It's terrible for efficiency.

I honestly didn't read that thread, so I don't know if the info came from there or not. But since the first two posts link to that thread, I consider that enough - otherwise I don't see how I could relevantly mention that link - "Here's something I recommend you don't use, and tells you everything you now already know"...?

But hey, for people who care, good, we have the link in the first couple of posts...

Edit: By the way, I'm pretty sure there's information in this thread that isn't even in that one. There's stuff I researched just for this tutorial, like do shadow ref's leak handle ids, and finding that shadow refs can have ForGroup calls if you call it immediately after removing the units (pretty sure on that).
 

Viikuna

No Marlo no game.
I agree, its lame. All that should be covered with debug prefixes, really.

But yea, at least I learned all this stuff from wc3c. Those guys did quite lot research about this stuff as far as I know.


Shadow references leaking handle ids makes sense and is logical and everything, as well as they being there during ForGroup calls too.

It might be that they never did any research about this stuff, or then they just didnt mentioned it, because neither of those is not really any problem at all.
Its FirstOfGroup failure that got people interested about shadow references.


Anyways, its a good tutorial. Now I dont have to explain this stuff over and over again, and I can just link all those guys here. Good works.
 

Troll-Brain

You can change this now in User CP.
Jesus4Lyf said:
and finding that shadow refs can have ForGroup calls if you call it immediately after removing the units (pretty sure on that)
Maybe i've misunderstood, but when you use RemoveUnit(), the unit isn't removed instantly, you still can work with it, like get this HP, or this typeid, and so one.

Also i really prefer this way for GroupUtils :

JASS:
library GroupUtils initializer init
//******************************************************************************
//* BY: Rising_Dusk
//* 
//* This library is a simple implementation of a stack for groups that need to
//* be in the user's control for greater than an instant of time. Additionally,
//* this library provides a single, global group variable for use with user-end
//* enumerations. It is important to note that users should not be calling
//* DestroyGroup() on the global group, since then it may not exist for when it
//* it is next needed.
//*
//* The group stack removes the need for destroying groups and replaces it with
//* a recycling method.
//*     function NewGroup takes nothing returns group
//*     function ReleaseGroup takes group g returns boolean
//*     function GroupRefresh takes group g returns nothing
//* 
//* NewGroup grabs a currently unused group from the stack or creates one if the
//* stack is empty. You can use this group however you'd like, but always
//* remember to call ReleaseGroup on it when you are done with it. If you don't
//* release it, it will 'leak' and your stack may eventually overflow if you
//* keep doing that.
//* 
//* GroupRefresh cleans a group of any shadow references which may be clogging
//* its hash table. If you remove a unit from the game who is a member of a unit
//* group, it will 'effectively' remove the unit from the group, but leave a
//* shadow in its place. Calling GroupRefresh on a group will clean up any
//* shadow references that may exist within it.
//* 
globals
    //* Group for use with all instant enumerations
    group ENUM_GROUP = CreateGroup()
    
    //* Temporary references for GroupRefresh
    private boolean Flag                                              = false
    private group Refr                                                = null
    
    //* Arrays and counter for the group stack
    private group array Groups
    private integer Count = 0
    
    // Dummy unit for released groups
    private unit Dummy
endglobals

private function AddEx takes nothing returns nothing
    if Flag then
        call GroupClear(Refr)
        set Flag = false
    endif
    call GroupAddUnit(Refr, GetEnumUnit())
endfunction
    
function GroupRefresh takes group g returns nothing
    set Flag = true
    set Refr = g
    call ForGroup(Refr, function AddEx)
    if Flag then
        call GroupClear(g)
    endif
endfunction

function NewGroup takes nothing returns group
    if Count == 0 then
        set Groups[0] = CreateGroup()
        return Groups[0]
    else
        set Count = Count - 1
        call GroupClear(Groups[Count])
    endif
    return Groups[Count]
endfunction

function ReleaseGroup takes group g returns boolean
    debug if g == null then
    debug   call BJDebugMsg(SCOPE_PREFIX+" Error: Null groups cannot be released")
    debug   return false
    debug endif
    
    debug if IsUnitInGroup(Dummy,g) then
    debug   call BJDebugMsg(SCOPE_PREFIX+" Error: Groups cannot be multiply released")
    debug   return false
    debug endif
    
    debug if Count == 8191 then
    debug   call BJDebugMsg(SCOPE_PREFIX+" Error: Max groups achieved, destroying group")
    debug   call DestroyGroup(g)
    debug   return false
    debug endif

    call GroupClear(g)
    debug call GroupAddUnit(g,Dummy)
    set Groups[Count] = g
    set Count = Count + 1

    return true
endfunction

private function init takes nothing returns nothing
    debug set Dummy = CreateUnit(Player(13),'hfoo',0.,0.,0.)
    debug call ShowUnit(Dummy,false)
endfunction

endlibrary
 

GoGo-Boy

You can change this now in User CP
Wait... GroupClear() does nothing at all and weather I do it before a new GroupEnum doesn't matter?
 

Romek

Super Moderator
Staff member
GroupEnum.. Clears the group automatically.
If you want to add units without clearing the group, use GroupAddGroup.
 

Troll-Brain

You can change this now in User CP.
JASS:
function NewGroup takes nothing returns group
    if Count > 0 then
        set Count = Count - 1
        return Groups[Count + 1]
    endif
    return CreateGroup()
endfunction

function ReleaseGroup takes group g returns boolean
    call GroupClear(g)
    set Count = Count + 1
    set Groups[Count] = g
endfunction


This is faster. lol
Ofc but there is no way to prevent double free here.

EDIT :

And you fail >.<
It doesn't work when you release a group ...
 

Jesus4Lyf

Good Idea™
I was told you need to clear global groups if you haven't removed all of the units inside of it.

Apparently if a unit gets removed while in the group it causes a leak.
 

Darthfett

Super Mod
While you are trying to avoid the use of Create/Destroy group functions, I would still like to see a mention of the handle id reference leak if you don't set a local group variable to null at the end of a function.

It should just be as simple as stating that if Create/Destroy group functions are used, that the normal handle nulling rules still apply.

Aside from that, it's very well written, very informative, and provides a great resource for those who are confused about all the different ways groups can leak.

Approved!
 

Prozix

New Member
Question

Could somebody explain to me why looping trough a group with FirstOfGroup(group) is deprecated?

And how would I optimize this piece of code?

JASS:
function CopyGroup takes group g returns group //This function was NOT written by me, I've got it from a tutorial on wc3c.net
    set bj_groupAddGroupDest = CreateGroup()
    call ForGroup(g, function GroupAddGroupEnum)
    return bj_groupAddGroupDest
endfunction

function CountLivingHeroesOfForce takes force team returns integer
    local integer count = 0
    local unit u
    local group g
    set g = CopyGroup(Heroes)
    loop
        set u = FirstOfGroup(g)
        exitwhen u == null 
        if IsPlayerInForce(GetOwningPlayer(u), team) and GetWidgetLife(u) &gt; 0.045 then
            set count = count + 1
        endif
        call GroupRemoveUnit(g, u)
    endloop
    call DestroyGroup(g)
    return count
endfunction


If this has been discussed somewhere before I'm sorry but I couldn't find a thread.
 

Darthfett

Super Mod
Could somebody explain to me why looping trough a group with FirstOfGroup(group) is deprecated?

If this has been discussed somewhere before I'm sorry but I couldn't find a thread.
The reason it is not useful in your function is because you're already going through every unit in the group with the ForGroup function. It's doing twice the amount of operations to loop through one group, when you could simply use a ForGroup, or a GroupEnum's filter.

FirstOfGroup loops aren't necessarily a bad thing, but when you have to GroupEnum these units through a filter, just to get them into the group, why not simply perform the operations in the filter, and avoid having to release/destroy a group at all.

This is especially useful as GroupEnum will cause a group to leak even if it is destroyed, so simply using one global group will create only one leak (am I correct in that the leak is only one per group, Jesus4Lyf?).
 
General chit-chat
Help Users
  • No one is chatting at the moment.
  • Ghan Ghan:
    I really do not want to move to California otherwise I might consider it.
  • The Helper The Helper:
    yeah California is not anywhere you really want to live
  • The Helper The Helper:
    That is why I did not take the job Blizzard offered me back in the day, there is no way I could have moved my family there on what they were offering, not even close and that was like 20 years ago
  • The Helper The Helper:
    yeah they wanted me on the tech support team when they did not get me they got one of the next MVPs in Dinobot
  • The Helper The Helper:
    Dinobot was one of the youngest of the MVPs tkron probably could have worked for Blizzard but he had a good job in Chicago doing business programming already
  • The Helper The Helper:
    Dinobot probably still works for Blizzard would love to reconnect with that guy
  • The Helper The Helper:
    I wonder what ever happened to Wargasm?
  • The Helper The Helper:
    This new version of Xenforo really is awesome
  • Ghan Ghan:
    Wargasm is still around. He works for the domain registrar where thehelper.net is kept.
  • Varine Varine:
    Is sqrage still around?
  • The Helper The Helper:
    I have not seen him on lately the forum says he was last on 2 years ago
  • The Helper The Helper:
    How are you doing Varine have not seen you around in a minute
  • thewrongvine thewrongvine:
    lol I live in CA
  • thewrongvine thewrongvine:
    I've got some friends in animation department who have been applying for Blizzard, doing interviews and such. They said the workplace seems nice, though now it'd be all remote work I suppose
  • The Helper The Helper:
    good for them I live close to Austin and am trying to get my kids interested in getting into Games but not having much luck. I would never let them move to Cali.
  • The Helper The Helper:
    unless it was huge money
  • The Helper The Helper:
    and even then with the taxes I really could never let them move there
  • thewrongvine thewrongvine:
    Haha fair enough, I have no interest in CA at all as a living place either. Really only here just for the work in LA, but hopefully once I get settled enough in my work, I can move out
    +1
  • Ghan Ghan:
    +1
  • jonas jonas:
    " a traditionally-sized California family of one person ":cool:
    +1

    Members online

    No members online now.

    Affiliates

    Hive Workshop
    Top