Tutorial How to use Groups without leaking

Jesus4Lyf

Good Idea™
This tutorial is short and straight to the point. Everything I say has been tested by me and I put my word to it.

Groups are a dilemma due to leaking RAM (I say RAM specifically because it does not leak handle ids).

What leaks?

Enumerating with a null boolexpr leaks. (Example in spoiler.)
JASS:
function FTRUE takes nothing returns boolean
    return true
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local group g=CreateGroup()
    local integer i=1000
    loop
    set i=i-1
    exitwhen i==0
        call GroupEnumUnitsInRange(g,0,0,50000,null /*"null" Leaks*/)
    endloop
    call DestroyGroup(g)
    set g=null
endfunction
Destroying a group that has had an enumeration called for it leaks RAM. (Example in spoiler.)
JASS:
function FTRUE takes nothing returns boolean
    return true
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local group g
    local integer i=1000
    loop
    set i=i-1
    exitwhen i==0
        set g=CreateGroup()
        call GroupEnumUnitsInRange(g,0,0,50000,Filter(function FTRUE))
        call GroupClear(g) // irrelevant, really
        call DestroyGroup(g /*Leaks because "g" has had an "Enum" called on it*/)
        set g=null
    endloop
endfunction

What should I do?

This is a magic snippet:
JASS:
globals
    group GROUP=CreateGroup()
endglobals

I use this in all my maps. Inside any enum function, we may place a filter. We can use this to execute code, and return false so no units are ever added. Let's say we want to heal every unit on the map for 200 health.

This is how you may be used to doing it:
JASS:
globals
    // values to carry to the DoThings function
    real AmountToHeal
endglobals
function DoThings takes nothing returns nothing
    call SetWidgetLife(GetEnumUnit(), GetWidgetLife(GetEnumUnit()) + AmountToHeal
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local group g = CreateGroup()
    call GroupEnumUnitsInRect(g, bj_mapInitialPlayableArea, null /*leak*/)
    set AmountToHeal = 200.0
    call ForGroup(g, function DoThings)
    call DestroyGroup(g /*leak*/)
    set g=null /*at least this doesn't leak*/
endfunction


Implement the magic snippet. Now, examine this code:
JASS:
globals
    // values to carry to the DoThings function
    real AmountToHeal
endglobals
function DoThings takes nothing returns boolean
                                      /*nothing becomes boolean*/

    call SetWidgetLife(GetFilterUnit(), GetWidgetLife(GetFilterUnit()) + AmountToHeal
                     /*GetEnumUnit() becomes GetFilterUnit()*/
    
    return false
  /*return false appears here*/
endfunction

function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    // Store values before the enum instead
    set AmountToHeal = 200.0
    // Note the next line.
    call GroupEnumUnitsInRect(GROUP, bj_mapInitialPlayableArea, Filter(function DoThings))
    // The end.
endfunction

Explanation for those who don't understand:
I will explain the last line. We must put something in the filter. We could put in the following if we like:
JASS:
function ReturnTrue takes nothing returns boolean
    return true
endfunction
//...
call GroupEnumUnitsInRect(GROUP, bj_mapInitialPlayableArea, Filter(function ReturnTrue))

But then we need to clear our group later, or use a FirstOfGroup loop (largely deprecated). This way we never even actually add the units to the group. This method is only useful when you do not need to store the group with units in it for any period of time, ie. you are not storing a group of units, but rather just want to do things for a bunch of units, like in the example.

The rest of the comments explain the process of changing the callback that usually goes into ForGroup into a filter for the Enum instead. Be sure not to return nothing. It must return boolean or bad things will happen (untested personally, but apparently it can cause desynchs or something).

If you need to store a group of units, for example, all units that a spell has hit so far, you can use dynamic groups, but not with [LJASS]CreateGroup[/LJASS] and [LJASS]DestroyGroup[/LJASS]...

To use dynamic groups, groups should be recycled instead of destroyed. I recommend Recycle to recycle groups. Download this snippet and install it into your map. Then replace:
[LJASS]CreateGroup()[/LJASS] with [LJASS]Group.get()[/LJASS]
and
[LJASS]call DestroyGroup(g)[/LJASS] with [LJASS]call Group.release(g)[/LJASS]​

This will reuse the groups and store them in a list until they are reused. The groups are cleared before they are stored, so there is no difference in use to destroying the group, except you should be careful not to store a reference to the group and do something to it after releasing it (that would be hard to debug).

Thus we solve the leak that occurs when we destroy a group that has had an enum called for it.

What else should I know aside from RAM leaks?

Any Enum call clears all units from a group before it adds units to the group. To enum units in a group and keep the previous units, instead use your global GROUP variable, and in the filter, add the units to the group you wish to add to.

Adding a unit to a group does not leak its handle id if it is removed from the game.

Removing a unit from the game does not remove its reference from the group! This is called a "shadow reference". Now, from my understanding, it is generally accepted that a shadow reference is never included a ForGroup, but I think I found that it can be if you call the ForGroup immediately after removing the unit from the game. Shadow references take up a little RAM, so you should clean out a group occasionally. This can be done with this simple snippet written by CaptainGriffen:
JASS:
library GroupRefresh
    
    globals
        private boolean clear
        private group enumed
    endglobals
    
    private function AddEx takes nothing returns nothing
        if clear then
            call GroupClear(enumed)
            set clear = false
        endif
        call GroupAddUnit(enumed, GetEnumUnit())
    endfunction
    
    function GroupRefresh takes group g returns nothing
        set clear = true
        set enumed = g
        call ForGroup(enumed, function AddEx)
        if clear then
             call GroupClear(g)
        endif
    endfunction
    
endlibrary

Simply call GroupRefresh on a group to clear out the shadow references. This is only needed when you have a permenant group in a map that keeps track of units, and only needed to free up some RAM. It is O(n) complexity, so don't spam its use if possible. (I never use it personally, unit attachment solves this in O(1) complexity, but that doesn't belong to this tutorial.)

There is a deprecated thing called a FirstOfGroup loop. It works like this:
JASS:
globals
    GROUP group = CreateGroup()
endglobals
function ReturnTrue takes nothing returns boolean
    return true
endfunction
function Trig_Untitled_Trigger_001_Actions takes nothing returns nothing
    local unit u
    call GroupEnumUnitsInRect(GROUP,bj_mapInitialPlayableArea,Filter(function ReturnTrue))
    loop
        set u=FirstOfGroup(GROUP)
        exitwhen u==null
        call GroupRemoveUnit(g,u)
        // Do things to u
        
    endloop
endfunction

This is not as efficient as using a filter for your actions, as a general rule. It is also vulnerable to shadow references if the units have been stored in the group for any period of time (in other words, is not done instantly after an Enum generally). It also requires you to make some sort of filter, even if you don't need it, otherwise you will have a leak due to the null filter (or boolexpr). In short, it is pointless to use this method, except it gives you access to your local variables so you don't need to carry things over. I recommend you do not use it, but you are welcome to all the same.

GroupClear clears all references out of a group, including shadow references.

You should never need to use GroupClear on your global GROUP, because Enum calls clear it anyway, and you should never need to actually add units to it.

Summary, Please!

Use
JASS:
globals
    group GROUP=CreateGroup()
endglobals

for doing things to a bunch of units, and put your actions in the Filter, returning false at the end. (Example: heal or damage an AoE of units.)

Use Recycle for tracking a bunch of units for a time. (Example: record what units have been hit by a spell.) Release your group at the end using Group.release(groupVariable), and I'd recommend to null your references to it unless you trust yourself not to accidentally use them (or else you can end up with issues which are hard to debug).

If you have a permanent global group for tracking units, use GroupRefresh on it occasionally to free some RAM up. :thup:

You can now use groups without leaks, in efficient, sensible ways. :)
 

Frozenhelfir

set Gwypaas = Guhveepaws
GroupUtils appears to do the same thing as recycle and refresh (recommended to me by Viikuna). Also, how do we use the dynamic groups to store something hit by the spell? You mentioned them, and the why, but how? :( This is the problem I had with my spell, so I had to use the FirstOfGroup loop instead of putting it in the filter.

Didn't know groups leaked with enums on destroy... Time to go [del]recode[/del] find/replace my map again...
 

Viikuna

No Marlo no game.
Perfect.

You should probably add link to GroupUtils.

I know you probably prefer some system like Recycle instead, but most of this information originates from that thread and Griffens GroupRefresh thread, so link would be nice.

edit. To Guy with Polar Bear avatar. FirstOfGroup loop is just fine, if you do it like Jesus4Lyf shows in his tutorial. ( Right after GroupEnum, so theres no possible shadow references )
 

Jesus4Lyf

Good Idea™
If you want to track which units have been hit by a spell, add them to the group. Then you use IsUnitInGroup to see if it has been hit. Didn't think it was general enough to include. Can't code every way groups can be used here.

So let's say you write carrion swarm, you keep using GROUP to enum units in range of projectile, in the filter you check if they're in the dynamic group (you probably attach it to a struct) and add it to the group if it is not, as well as performing checks for ally/enemy, dealing damage, etc.

Here's why I hate GroupUtils:
JASS:
function ReleaseGroup takes group g returns boolean
    local integer stat = Status[GetHandleId(g)-MIN_HANDLE_ID]
    if g == null then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Null groups cannot be released")
        return false
    elseif stat == 0 then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Group not part of stack")
        return false
    elseif stat == 2 then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Groups cannot be multiply released")
        return false
    elseif Count == 8191 then
        debug call BJDebugMsg(SCOPE_PREFIX+"Error: Max groups achieved, destroying group")
        call DestroyGroup(g)
        return false
    endif
    call GroupClear(g)
    set Groups[Count]                        = g
    set Count                                = Count + 1
    set Status[GetHandleId(g)-MIN_HANDLE_ID] = 2
    return true
endfunction

You gotta admit, that's kind of lame. It's terrible for efficiency.

I honestly didn't read that thread, so I don't know if the info came from there or not. But since the first two posts link to that thread, I consider that enough - otherwise I don't see how I could relevantly mention that link - "Here's something I recommend you don't use, and tells you everything you now already know"...?

But hey, for people who care, good, we have the link in the first couple of posts...

Edit: By the way, I'm pretty sure there's information in this thread that isn't even in that one. There's stuff I researched just for this tutorial, like do shadow ref's leak handle ids, and finding that shadow refs can have ForGroup calls if you call it immediately after removing the units (pretty sure on that).
 

Viikuna

No Marlo no game.
I agree, its lame. All that should be covered with debug prefixes, really.

But yea, at least I learned all this stuff from wc3c. Those guys did quite lot research about this stuff as far as I know.


Shadow references leaking handle ids makes sense and is logical and everything, as well as they being there during ForGroup calls too.

It might be that they never did any research about this stuff, or then they just didnt mentioned it, because neither of those is not really any problem at all.
Its FirstOfGroup failure that got people interested about shadow references.


Anyways, its a good tutorial. Now I dont have to explain this stuff over and over again, and I can just link all those guys here. Good works.
 

Troll-Brain

You can change this now in User CP.
Jesus4Lyf said:
and finding that shadow refs can have ForGroup calls if you call it immediately after removing the units (pretty sure on that)
Maybe i've misunderstood, but when you use RemoveUnit(), the unit isn't removed instantly, you still can work with it, like get this HP, or this typeid, and so one.

Also i really prefer this way for GroupUtils :

JASS:
library GroupUtils initializer init
//******************************************************************************
//* BY: Rising_Dusk
//* 
//* This library is a simple implementation of a stack for groups that need to
//* be in the user's control for greater than an instant of time. Additionally,
//* this library provides a single, global group variable for use with user-end
//* enumerations. It is important to note that users should not be calling
//* DestroyGroup() on the global group, since then it may not exist for when it
//* it is next needed.
//*
//* The group stack removes the need for destroying groups and replaces it with
//* a recycling method.
//*     function NewGroup takes nothing returns group
//*     function ReleaseGroup takes group g returns boolean
//*     function GroupRefresh takes group g returns nothing
//* 
//* NewGroup grabs a currently unused group from the stack or creates one if the
//* stack is empty. You can use this group however you'd like, but always
//* remember to call ReleaseGroup on it when you are done with it. If you don't
//* release it, it will 'leak' and your stack may eventually overflow if you
//* keep doing that.
//* 
//* GroupRefresh cleans a group of any shadow references which may be clogging
//* its hash table. If you remove a unit from the game who is a member of a unit
//* group, it will 'effectively' remove the unit from the group, but leave a
//* shadow in its place. Calling GroupRefresh on a group will clean up any
//* shadow references that may exist within it.
//* 
globals
    //* Group for use with all instant enumerations
    group ENUM_GROUP = CreateGroup()
    
    //* Temporary references for GroupRefresh
    private boolean Flag                                              = false
    private group Refr                                                = null
    
    //* Arrays and counter for the group stack
    private group array Groups
    private integer Count = 0
    
    // Dummy unit for released groups
    private unit Dummy
endglobals

private function AddEx takes nothing returns nothing
    if Flag then
        call GroupClear(Refr)
        set Flag = false
    endif
    call GroupAddUnit(Refr, GetEnumUnit())
endfunction
    
function GroupRefresh takes group g returns nothing
    set Flag = true
    set Refr = g
    call ForGroup(Refr, function AddEx)
    if Flag then
        call GroupClear(g)
    endif
endfunction

function NewGroup takes nothing returns group
    if Count == 0 then
        set Groups[0] = CreateGroup()
        return Groups[0]
    else
        set Count = Count - 1
        call GroupClear(Groups[Count])
    endif
    return Groups[Count]
endfunction

function ReleaseGroup takes group g returns boolean
    debug if g == null then
    debug   call BJDebugMsg(SCOPE_PREFIX+" Error: Null groups cannot be released")
    debug   return false
    debug endif
    
    debug if IsUnitInGroup(Dummy,g) then
    debug   call BJDebugMsg(SCOPE_PREFIX+" Error: Groups cannot be multiply released")
    debug   return false
    debug endif
    
    debug if Count == 8191 then
    debug   call BJDebugMsg(SCOPE_PREFIX+" Error: Max groups achieved, destroying group")
    debug   call DestroyGroup(g)
    debug   return false
    debug endif

    call GroupClear(g)
    debug call GroupAddUnit(g,Dummy)
    set Groups[Count] = g
    set Count = Count + 1

    return true
endfunction

private function init takes nothing returns nothing
    debug set Dummy = CreateUnit(Player(13),'hfoo',0.,0.,0.)
    debug call ShowUnit(Dummy,false)
endfunction

endlibrary
 

GoGo-Boy

You can change this now in User CP
Wait... GroupClear() does nothing at all and weather I do it before a new GroupEnum doesn't matter?
 

Romek

Super Moderator
Staff member
GroupEnum.. Clears the group automatically.
If you want to add units without clearing the group, use GroupAddGroup.
 

Troll-Brain

You can change this now in User CP.
JASS:
function NewGroup takes nothing returns group
    if Count > 0 then
        set Count = Count - 1
        return Groups[Count + 1]
    endif
    return CreateGroup()
endfunction

function ReleaseGroup takes group g returns boolean
    call GroupClear(g)
    set Count = Count + 1
    set Groups[Count] = g
endfunction


This is faster. lol
Ofc but there is no way to prevent double free here.

EDIT :

And you fail >.<
It doesn't work when you release a group ...
 

Jesus4Lyf

Good Idea™
I was told you need to clear global groups if you haven't removed all of the units inside of it.

Apparently if a unit gets removed while in the group it causes a leak.
 

Darthfett

Super Mod
While you are trying to avoid the use of Create/Destroy group functions, I would still like to see a mention of the handle id reference leak if you don't set a local group variable to null at the end of a function.

It should just be as simple as stating that if Create/Destroy group functions are used, that the normal handle nulling rules still apply.

Aside from that, it's very well written, very informative, and provides a great resource for those who are confused about all the different ways groups can leak.

Approved!
 

Prozix

New Member
Question

Could somebody explain to me why looping trough a group with FirstOfGroup(group) is deprecated?

And how would I optimize this piece of code?

JASS:
function CopyGroup takes group g returns group //This function was NOT written by me, I've got it from a tutorial on wc3c.net
    set bj_groupAddGroupDest = CreateGroup()
    call ForGroup(g, function GroupAddGroupEnum)
    return bj_groupAddGroupDest
endfunction

function CountLivingHeroesOfForce takes force team returns integer
    local integer count = 0
    local unit u
    local group g
    set g = CopyGroup(Heroes)
    loop
        set u = FirstOfGroup(g)
        exitwhen u == null 
        if IsPlayerInForce(GetOwningPlayer(u), team) and GetWidgetLife(u) &gt; 0.045 then
            set count = count + 1
        endif
        call GroupRemoveUnit(g, u)
    endloop
    call DestroyGroup(g)
    return count
endfunction


If this has been discussed somewhere before I'm sorry but I couldn't find a thread.
 

Darthfett

Super Mod
Could somebody explain to me why looping trough a group with FirstOfGroup(group) is deprecated?

If this has been discussed somewhere before I'm sorry but I couldn't find a thread.
The reason it is not useful in your function is because you're already going through every unit in the group with the ForGroup function. It's doing twice the amount of operations to loop through one group, when you could simply use a ForGroup, or a GroupEnum's filter.

FirstOfGroup loops aren't necessarily a bad thing, but when you have to GroupEnum these units through a filter, just to get them into the group, why not simply perform the operations in the filter, and avoid having to release/destroy a group at all.

This is especially useful as GroupEnum will cause a group to leak even if it is destroyed, so simply using one global group will create only one leak (am I correct in that the leak is only one per group, Jesus4Lyf?).
 
General chit-chat
Help Users
  • No one is chatting at the moment.
  • The Helper The Helper:
    I read in the PC Gamer review that it was not much of an update to the game like just a re-release
  • Ghan Ghan:
    It's a good remaster. They did not want to change the game much because there are still many die-hard players out there that wouldn't like it to be a new game.
  • Ghan Ghan:
    They've said they are open to adding new content in the future, but wanted to get what's there correct first. As it is, for what you get the $40 asking price is a bit steep.
  • The Helper The Helper:
    Yeah that is a high price think I am going to wait a minute and see if it goes down some.
  • The Helper The Helper:
    Looking forward to doing some dungeon grinding again though.
  • The Helper The Helper:
    still 101 bots on the site constantly
  • midnight8 midnight8:
    my bots got out of their cage? oh no
  • The Helper The Helper:
    these bots are from russia, china and asia they are less than they were but more than they need to be I have no idea why they hover here
  • tom_mai78101 tom_mai78101:
    Probably because of my Headline News
  • The Helper The Helper:
    Facebook is getting fucked right now lol
  • The Helper The Helper:
    I learned one thing from the Facebook outage - BGP - I am sure Ghan is an expert in it but I did not know what the name of it was I knew there was something like that there though
  • The Helper The Helper:
    What is up Tom?
  • jonas jonas:
    We once considered doing a project on BGP looking for problems in the implementations. But we didn't get sufficient interest and Tony Li was mean to us so we said whatever
  • Varine Varine:
    hello, peeps. I got most of my computer back up to Idaho so I have passwords again after that nightmare of like 4 months
  • Varine Varine:
    We're all trying to get time off now, but holy shit. I did not expect it to be that busy. Every day was a historical record, and every week was a total sales record for like all of summer. My pay is going to almost double next summer, which makes cooking finally a viable career, I might actually afford to change industries soon!
  • Varine Varine:
    We had people ask if there was anything we could do about the fucking sun. There's like 30 inside tables and every outside table has a sunbrella thing.
  • Varine Varine:
    No, I'm sorry, I cannot move the sun and the umbrella doesn't go 90 degrees sideways. They unfortunately had to deal with a sunset across mountains and a lake, we can only imagine the horror they had to undergo by visiting our restaurant
  • Varine Varine:
    Well, they do, but they're big and blow away and a 10 foot runaway umbrella is a much bigger problem than some cunt with the sun sorta in her eyes
  • The Helper The Helper:
    What is up Varine? Good to see you buddy! Too bad you are not in Houston I have a friend that needs Cooking staff like desperately remember if you ever want to get a different landscape, not sunsets over mountains and a lake Houston has opportunity!
  • tom_mai78101 tom_mai78101:
    Nice to see Varine back in action.
  • The Helper The Helper:
    Welcome back Varine!
  • The Helper The Helper:
    can someone answer a question in the World Editor Help forum?
  • jonas jonas:
    ok
  • The Helper The Helper:
    thank you jonas!
  • Varine Varine:
    I'll probably be here through next summer at least, I get paid decent at the moment and I expect it to go up significantly next year. Plus I'm on contract through the end of the year
    +1

    Members online

    Affiliates

    Hive Workshop NUON Dome
    Top