System Advanced Indexing & Data Storage

[kingkingyyk3]

Optimization :
Use linked list on periodic recycler. Link the index(In use). Unlink it after the index is released.

 

[Jesus4Lyf]

Optimization :
Use linked list on periodic recycler. Link the index(In use). Unlink it after the index is released.

To make that faster, I'd have to add an extra array for previous node. The difference is a practical 0 in a part of the system which is deprecated and by default doesn't exist. Furthermore, the practical 0 speed gain comes at the cost of an extra array. Hence, nothing will be changed.

 

[kingkingyyk3]

Hence, nothing will be changed.
lol

 

[Darthfett]

http://www.thehelper.net/forums/showthread.php?t=145766

Planning on doing anything like what you mentioned here with AIDS? I do enjoy using modules much more than I do textmacros.

 

[Jesus4Lyf]

Intentions are cruel.

I don't have time to do this yet to be honest - it requires me to take a serious look at how AIDS initialises. Changing the macro to a module changes the init order which in turn breaks structs - I still intend to make the change when I have time to sit down and test it thoroughly, but this probably won't be until late July at the earliest..

Believe me, I actually took a shot at it... but gave up when I realised it was not a trivial matter to swap over...

 

[SanKakU]

what would you recommend for me if i run into the following problem : http://www.thehelper.net/forums/showthread.php?t=149411 ?

 

[13lade619]

J4L,

What does the periodic recycler do? (pros/cons)

 

[tooltiperror]

This is going to sound crazy, but a wicked extreme but helpful tutorial would be outlining the though process and coding of AIDS.

If you know what I mean.

 

[Jesus4Lyf]

what would you recommend for me if i run into the following problem : http://www.thehelper.net/forums/showthread.php?t=149411 ?

You should be able to use the latest version of AIDS, delete UnitIndexingUtils, and use:

library UnitIndexingUtils uses AIDS
endlibrary

without problems. So long as you delete UIU.

>What does the periodic recycler do? (pros/cons)
Instead of removing the indexes when the unit is removed from it game, it loops through the indexes checking 32 per second to see if the unit has been removed, and frees the index then.

The only pro is we know it works, even if Blizzard fraks something up.

The cons, ofc, are that it is (debatably) less efficient, and gives a delay between the unit being removed from the game and the AIDS structs being destroyed, etc.

It's only really there in case something goes horribly wrong in your map to do with when AIDS structs are destroyed and unit indexes being recycled, you can check if it's the way AIDS frees indexes (has never happened to me). So it helps with debugging, perhaps.

>a wicked extreme but helpful tutorial would be outlining the thought process and coding of AIDS
Hm. I don't have time to do anything like this in a hurry, I think..

 

[SanKakU]

hmm. there must be something wrong with how i imported the spell. aids seems to work just fine in place of uiu. something else is afoot. i'll have to sort it out myself.

 

[Weep]

I've a problem with AIDS and TimerUtils (hashtable variety). The following code doesn't work for preplaced units, only ones which enter the map later:

Spoiler

library test uses AIDS, TimerUtils
        struct Test extends array
                private timer t
               
                static method periodic takes nothing returns nothing
                        call BJDebugMsg("debug")
                endmethod
                private method AIDS_onCreate takes nothing returns nothing
                        set t = NewTimer()
                        call TimerStart(t,3.,true,function thistype.periodic)
                endmethod
                private method AIDS_onDestroy takes nothing returns nothing
                        call ReleaseTimer(t)
                        set t = null
                endmethod
               
                //! runtextmacro AIDS()
        endstruct
endlibrary

I hear that AIDS runs its init on preplaced units before TU inits. What can I do, or am I doing something wrong?

 

[kingkingyyk3]

No... TimerUtils is bugged in AIDS.(Vex failed? ) Even I used my recycle library, it is bugged too. However, you can go with KT.

 

[Chaos_Knight]

I know this is Damn useful, but i still i dont know how to use... Sorry

 

[Weep]

However, you can go with KT.

What a pain... [edit] And I'm having trouble figuring out how to stop a KT timer that's running a method in an AIDS struct from AIDS_onDestroy.

 

[Jesus4Lyf]

How to unfail Vex's fail (until he fixes it).

library TimerUtils
//*********************************************************************
//* Edited by Jesus4Lyf to fix init issues.
//*********************************************************************
//* TimerUtils (red+blue+orange flavors for 1.24b+)
//* ----------
//*
//*  To implement it , create a custom text trigger called TimerUtils
//* and paste the contents of this script there.
//*
//*  To copy from a map to another, copy the trigger holding this
//* library to your map.
//*
//* (requires vJass)   More scripts: htt://www.wc3c.net
//*
//* For your timer needs:
//*  * Attaching
//*  * Recycling (with double-free protection)
//*
//* set t=NewTimer()      : Get a timer (alternative to CreateTimer)
//* ReleaseTimer(t)       : Relese a timer (alt to DestroyTimer)
//* SetTimerData(t,2)     : Attach value 2 to timer
//* GetTimerData(t)       : Get the timer's value.
//*                         You can assume a timer's value is 0
//*                         after NewTimer.
//*
//* Multi-flavor:
//*    Set USE_HASH_TABLE to true if you don't want to complicate your life.
//*
//* If you like speed and giberish try learning about the other flavors.
//*
//********************************************************************

//================================================================
    globals
        //How to tweak timer utils:
        // USE_HASH_TABLE = true  (new blue)
        //  * SAFEST
        //  * SLOWEST (though hash tables are kind of fast)
        //
        // USE_HASH_TABLE = false, USE_FLEXIBLE_OFFSET = true  (orange)
        //  * kinda safe (except there is a limit in the number of timers)
        //  * ALMOST FAST
        //
        // USE_HASH_TABLE = false, USE_FLEXIBLE_OFFSET = false (red)
        //  * THE FASTEST (though is only  faster than the previous method
        //                  after using the optimizer on the map)
        //  * THE LEAST SAFE ( you may have to tweak OFSSET manually for it to
        //                     work)
        //
        private constant boolean USE_HASH_TABLE      = true
        private constant boolean USE_FLEXIBLE_OFFSET = false

        private constant integer OFFSET     = 0x100000
        private          integer VOFFSET    = OFFSET
              
        //Timers to preload at map init:
        private constant integer QUANTITY   = 256
        
        //Changing this  to something big will allow you to keep recycling
        // timers even when there are already AN INCREDIBLE AMOUNT of timers in
        // the stack. But it will make things far slower so that's probably a bad idea...
        private constant integer ARRAY_SIZE = 8190

    endglobals

    //==================================================================================================
    globals
        private integer array data[ARRAY_SIZE]
        private hashtable     ht
    endglobals

    //It is dependent on jasshelper's recent inlining optimization in order to perform correctly.
    function SetTimerData takes timer t, integer value returns nothing
        static if(USE_HASH_TABLE) then
            // new blue
            call SaveInteger(ht,0,GetHandleId(t), value)
            
        elseif (USE_FLEXIBLE_OFFSET) then
            // orange
            static if (DEBUG_MODE) then
                if(GetHandleId(t)-VOFFSET<0) then
                    call BJDebugMsg("SetTimerData: Wrong handle id, only use SetTimerData on timers created by NewTimer")
                endif
            endif
            set data[GetHandleId(t)-VOFFSET]=value
        else
            // new red
            static if (DEBUG_MODE) then
                if(GetHandleId(t)-OFFSET<0) then
                    call BJDebugMsg("SetTimerData: Wrong handle id, only use SetTimerData on timers created by NewTimer")
                endif
            endif
            set data[GetHandleId(t)-OFFSET]=value
        endif        
    endfunction

    function GetTimerData takes timer t returns integer
        static if(USE_HASH_TABLE) then
            // new blue
            return LoadInteger(ht,0,GetHandleId(t) )
            
        elseif (USE_FLEXIBLE_OFFSET) then
            // orange
            static if (DEBUG_MODE) then
                if(GetHandleId(t)-VOFFSET<0) then
                    call BJDebugMsg("SetTimerData: Wrong handle id, only use SetTimerData on timers created by NewTimer")
                endif
            endif
            return data[GetHandleId(t)-VOFFSET]
        else
            // new red
            static if (DEBUG_MODE) then
                if(GetHandleId(t)-OFFSET<0) then
                    call BJDebugMsg("SetTimerData: Wrong handle id, only use SetTimerData on timers created by NewTimer")
                endif
            endif
            return data[GetHandleId(t)-OFFSET]
        endif        
    endfunction

    //==========================================================================================
    globals
        private timer array tT[ARRAY_SIZE]
        private integer tN = 0
        private constant integer HELD=0x28829022
        //use a totally random number here, the more improbable someone uses it, the better.
    endglobals

    //==========================================================================================
    function NewTimer takes nothing returns timer
        if (tN==0) then
            //If this happens then the QUANTITY rule has already been broken, try to fix the
            // issue, else fail.
            debug call BJDebugMsg("NewTimer: Warning, Exceeding TimerUtils_QUANTITY, make sure all timers are getting recycled correctly")
            static if( not USE_HASH_TABLE) then
                debug call BJDebugMsg("In case of errors, please increase it accordingly, or set TimerUtils_USE_HASH_TABLE to true")
                set tT[0]=CreateTimer()
                static if( USE_FLEXIBLE_OFFSET) then
                    if (GetHandleId(tT[0])-VOFFSET<0) or (GetHandleId(tT[0])-VOFFSET>=ARRAY_SIZE) then
                        //all right, couldn't fix it
                        call BJDebugMsg("NewTimer: Unable to allocate a timer, you should probably set TimerUtils_USE_HASH_TABLE to true or fix timer leaks.")
                        return null
                    endif
                else
                    if (GetHandleId(tT[0])-OFFSET<0) or (GetHandleId(tT[0])-OFFSET>=ARRAY_SIZE) then
                        //all right, couldn't fix it
                        call BJDebugMsg("NewTimer: Unable to allocate a timer, you should probably set TimerUtils_USE_HASH_TABLE to true or fix timer leaks.")
                        return null
                    endif
                endif
            endif
        else
            set tN=tN-1
        endif
        call SetTimerData(tT[tN],0)
     return tT[tN]
    endfunction

    //==========================================================================================
    function ReleaseTimer takes timer t returns nothing
        if(t==null) then
            debug call BJDebugMsg("Warning: attempt to release a null timer")
            return
        endif
        if (tN==ARRAY_SIZE) then
            debug call BJDebugMsg("Warning: Timer stack is full, destroying timer!!")

            //stack is full, the map already has much more troubles than the chance of bug
            call DestroyTimer(t)
        else
            call PauseTimer(t)
            if(GetTimerData(t)==HELD) then
                debug call BJDebugMsg("Warning: ReleaseTimer: Double free!")
                return
            endif
            call SetTimerData(t,HELD)
            set tT[tN]=t
            set tN=tN+1
        endif    
    endfunction

    private function init takes nothing returns nothing
     local integer i=0
     local integer o=-1
     local boolean oops = false
     
        static if( USE_HASH_TABLE ) then
            set ht = InitHashtable()
            loop
                exitwhen(i==QUANTITY)
                set tT<i>=CreateTimer()
                call SetTimerData(tT<i>, HELD)
                set i=i+1
            endloop
            set tN = QUANTITY
        else
            loop
                set i=0
                loop
                    exitwhen (i==QUANTITY)
                    set tT<i> = CreateTimer()
                    if(i==0) then
                        set VOFFSET = GetHandleId(tT<i>)
                        static if(USE_FLEXIBLE_OFFSET) then
                            set o=VOFFSET
                        else
                            set o=OFFSET
                        endif
                    endif
                    if (GetHandleId(tT<i>)-o&gt;=ARRAY_SIZE) then
                        exitwhen true
                    endif
                    if (GetHandleId(tT<i>)-o&gt;=0)  then
                        set i=i+1
                    endif
                endloop
                set tN = i
                exitwhen(tN == QUANTITY)
                set oops = true
                exitwhen not USE_FLEXIBLE_OFFSET
                debug call BJDebugMsg(&quot;TimerUtils_init: Failed a initialization attempt, will try again&quot;)               
            endloop
            
            if(oops) then
                static if ( USE_FLEXIBLE_OFFSET) then
                    debug call BJDebugMsg(&quot;The problem has been fixed.&quot;)
                    //If this message doesn&#039;t appear then there is so much
                    //handle id fragmentation that it was impossible to preload
                    //so many timers and the thread crashed! Therefore this
                    //debug message is useful.
                elseif(DEBUG_MODE) then
                    call BJDebugMsg(&quot;There were problems and the new timer limit is &quot;+I2S(i))
                    call BJDebugMsg(&quot;This is a rare ocurrence, if the timer limit is too low:&quot;)
                    call BJDebugMsg(&quot;a) Change USE_FLEXIBLE_OFFSET to true (reduces performance a little)&quot;)
                    call BJDebugMsg(&quot;b) or try changing OFFSET to &quot;+I2S(VOFFSET) )
                endif
            endif
        endif

    endfunction
    
    private module InitMod
        private static method onInit takes nothing returns nothing
            call init()
        endmethod
    endmodule
    private struct InitStruct extends array
        implement InitMod
    endstruct
endlibrary</i></i></i></i></i></i>

 

[Weep]

I would have hoped for a solution or alternate approach using unmodified approved resources. :banghead:

I've a feeling I'll be going back to GUI after my project.

 

[Jesus4Lyf]

That fix for TU I posted is very valid. It is not strictly an AIDS issue - TU is bugged due to new things which have been implemented in vJass to break the system.

There is a way to do it with KT2, but fixing TU is better.

 

[Weep]

There is a way to do it with KT2, but fixing TU is better.

Well, could you explain the KT approach? Requiring a modification of an accepted resource isn't a great idea for something that's meant to be a system I'm hoping to submit at some point.

[edit] All my static ifs, and I hadn't even checked whether TU has the same problem when working with AutoIndex; it does. I just assumed WC3C resources would play nice with each other.

 

[Jesus4Lyf]

I do it here.
You need to account for the rare occasion that an index is paused and restarted before the timer next expires. The unfortunate thing about KT2 was it was not designed for remote stopping.

If I were you I'd consider hashtable attachment right about now.
Or releasing a fix to TU as well, and requiring it. I'd happily approve until the real TU is repaired, if you require it.

 

[Weep]

I do it here.

I'll take a look, thanks.

Or releasing a fix to TU as well, and requiring it. I'd happily approve until the real TU is repaired, if you require it.

Last resort.

[edit] Lulz, workaround:

Spoiler

library test uses AIDS, TimerUtils
    globals
        private boolean SAFE = false
    endglobals
    
    private struct InitStruct
        private static method safeStartEnum takes nothing returns boolean
            call GetFilterUnit():Test.AIDS_onCreate()
            return false
        endmethod
        private static method safeStart takes nothing returns nothing
            local region r=CreateRegion()
            local group g=CreateGroup()
            local integer i=15
            set SAFE = true
            call RegionAddRect(r,GetWorldBounds())
            set r=null
            loop
                call GroupEnumUnitsOfPlayer(UnitHistory_GROUP,Player(i),Filter(function thistype.safeStartEnum))
                exitwhen i == 0
                set i = i - 1
            endloop
            call DestroyGroup(g)
            set g=null
            call DestroyTimer(GetExpiredTimer())
        endmethod
        private static method onInit takes nothing returns nothing
            call TimerStart(CreateTimer(), 0, false, function thistype.safeStart)
        endmethod
    endstruct
    
    struct Test extends array
        private timer t
        
        static method periodic takes nothing returns nothing
            call BJDebugMsg("debug")
        endmethod
        method AIDS_onCreate takes nothing returns nothing
            if SAFE then
                set t = NewTimer()
                call TimerStart(t,3.,true,function thistype.periodic)
            endif
        endmethod
        private method AIDS_onDestroy takes nothing returns nothing
            call ReleaseTimer(t)
            set t = null
        endmethod
        
        //! runtextmacro AIDS()
    endstruct
endlibrary