System Faster Table

Jesus4Lyf

Good Idea™
Reaction score
397
Faster Table​

Requirements:
- Jass NewGen

Description:
Hey look, it's faster!

JASS:
library Table initializer init
//***************************************************************
//* Table object
//* ------------
//*
//*   set t=Table.create() - instanceates a new table object
//*   call t.destroy()     - destroys it
//*   t[1234567]           - Get value for key 1234567
//*                          (zero if not assigned previously)
//*   set t[12341]=32      - Assigning it.
//*   call t.flush(12341)  - Flushes the stored value, so it
//*                          doesn't use any more memory
//*   t.exists(32)         - Was key 32 assigned? Notice
//*                          that flush() unassigns values.
//*   call t.reset()       - Flushes the whole contents of the
//*                          Table.
//*
//*   call t.destroy()     - Does reset() and also recycles the id.
//*
//*   If you use HandleTable instead of Table, it is the same
//* but it uses handles as keys, the same with StringTable.
//*
//***************************************************************
 
//=============================================================
    globals
        private constant integer MAX_INSTANCES=8190 //400000
 
    //=========================================================
        private gamecache gc
    endglobals
 
    //Hey: Don't instanciate other people's textmacros that you are not supposed to, thanks.
    //! textmacro Table__make takes name, type, key
    struct $name$[MAX_INSTANCES]
        private string base
        
        static method create takes nothing returns $name$
            local $name$ d=$name$.allocate()
            set d.base=I2S(d)
            return d
        endmethod
        
        method reset takes nothing returns nothing
            call FlushStoredMission(gc,this.base)
        endmethod
 
        private method onDestroy takes nothing returns nothing
            call FlushStoredMission(gc,this.base)
        endmethod
 
        method operator [] takes $type$ key returns integer
            return GetStoredInteger(gc,this.base,$key$)
        endmethod
 
        method operator []= takes $type$ key, integer value returns nothing
            call StoreInteger(gc,this.base,$key$, value)
        endmethod
 
        method flush takes $type$ key returns nothing
            call FlushStoredInteger(gc,this.base,$key$)
        endmethod
 
        method exists takes $type$ key returns boolean
            return HaveStoredInteger(gc,this.base,$key$)
        endmethod
 
    endstruct
    //! endtextmacro
 
    private function H2I takes handle h returns integer
        return h
        return 0
    endfunction
    
    //! runtextmacro Table__make("Table","integer","I2S(key)" )
    //! runtextmacro Table__make("StringTable","string","key" )
    //! runtextmacro Table__make("HandleTable","handle","I2S(H2I(key))" )
 
    //=============================================================
    // initialize it all.
    //
    private function init takes nothing returns nothing
        call FlushGameCache(InitGameCache("libtable.gc"))
        set gc=InitGameCache("libtable.gc")
    endfunction
endlibrary

Table-X, my successful attempt at not using gamecache, for those who want to see some interesting code. It virtually simulates gamecache, with the disadvantage being it doesn't save/load correctly, apparently. This might even be fixable.
JASS:
library Table
//***************************************************************
//***    Jesus4Lyf's    **    TABLE-X   **    Version 1.02    ***
//***************************************************************
//* Table object
//* ------------
//*
//*   set t=Table.create() - instanceates a new table object
//*   call t.destroy()     - destroys it
//*   t[1234567]           - Get value for key 1234567
//*                          (zero if not assigned previously)
//*   set t[12341]=32      - Assigning it.
//*   call t.flush(12341)  - Flushes the stored value, so it
//*                          doesn't use any more memory
//*   t.exists(32)         - Was key 32 assigned? Notice
//*                          that flush() unassigns values.
//*   call t.reset()       - Flushes the whole contents of the
//*                          Table.
//*
//*   call t.destroy()     - Does reset() and also recycles the id.
//*
//*   If you use HandleTable instead of Table, it is the same
//* but it uses handles as keys, the same with StringTable.
//*
//***************************************************************
//* This version of Table does not use GameCache.
//* Please note that all data attaching is messed up if you load
//*  a saved game. Use normal Table if you need to load games.

    ///////////////
    // Constants //
//===============================================================
    globals
        private constant integer MAX_INSTANCES=8100 //400000
        private constant integer MAX_ATTACHMENTS=8190
        // MAX_ATTACHMENTS is the maximum number of things being
        // attached to in total + the number of tables in
        // existance + 1, at any one time.
        // If it is any higher than 8190, it is slower than normal Table.

    ///////////////////////////
    // System Implementation //
//===============================================================
        private constant integer MAX_STRINGID=MAX_ATTACHMENTS-1
        private integer RandomData=MAX_ATTACHMENTS
    endglobals
    
    private keyword GTable
    
    //////////////////////////////////////////////////////
    // 2D DOUBLY LINKED-LIST HASHING TABLE SYNTAX CANDY //
    /////////////////////////////////////////////////////////////
    // Sounds fun? Think again. Use (for my own reference):
    // d.delete()
    // d.chainDelete()
    // Data.getPointer() --> Data
    // Data.getForRead(stringid) --> Data or 0
    // Data.getForWrite(trackingPointer, stringid) --> Data
    private struct Data[MAX_ATTACHMENTS] // Not really a struct, just syntax candy.
        integer stringid
        integer value
        Data prev
        Data next
        Data drill // down
        Data roll // counter-down (for unlinking)
        /////////////
        // GENERAL //
        /////////////////////////////////////////////////////////
        method wipe takes nothing returns nothing
            set this.stringid=0
            set this.value=0
            set this.prev=0
            set this.next=0
            set this.drill=0
            set this.roll=0
        endmethod
        method move takes Data targ returns Data
            // Copy
            set targ.stringid=this.stringid
            set targ.value=this.value
            set targ.prev=this.prev
            set targ.next=this.next
            set targ.drill=this.drill
            set targ.roll=this.roll
            // Fix pointers
            set this.next.prev=targ
            set this.prev.next=targ
            set this.drill.roll=targ
            set this.roll.drill=targ
            // Clean up
            call this.wipe()
            // Done
            return targ // Probably unneeded.
        endmethod
        method init takes Data prev, integer stringid returns nothing
            set this.stringid=stringid
            set prev.next.prev=this
            set this.next=prev.next
            set prev.next=this
            set this.prev=prev
        endmethod
        static method getRandom takes nothing returns Data
            local integer oldrandom=RandomData
            set RandomData=RandomData-1
            loop
                if RandomData<1 then
                    set RandomData=MAX_ATTACHMENTS-1
                endif
                exitwhen Data(RandomData).stringid==0
                if RandomData==oldrandom then
                    call BJDebugMsg("Table-X out of space. Increase MAX_ATTACHMENTS constant.")
                endif
                set RandomData=RandomData-1
            endloop
            return RandomData
        endmethod
        method delete takes nothing returns nothing
            set this.next.prev=this.prev
            set this.prev.next=this.next
            set this.drill.roll=this.roll
            set this.roll.drill=this.drill
            if this.roll==0 and this.drill!=0 then
                call this.drill.move(this)
            else
                call this.wipe()
            endif
        endmethod
        method chainDelete takes nothing returns nothing
            if this==0 then
                return
            endif
            call this.next.chainDelete()
            if this.roll==0 and this.drill!=0 then
                call this.drill.move(this)
            else
                call this.wipe()
            endif
        endmethod
        static method getPointer takes nothing returns Data
            local Data d=Data.getRandom()
            set d.stringid=-1
            return d
        endmethod
        
        //////////////////////////
        // STACKING ABSTRACTION //
        /////////////////////////////////////////////////////////
        method stack takes Data nextDown returns Data
            set this.drill.roll=nextDown
            set nextDown.drill=this.drill
            set this.drill=nextDown
            set nextDown.roll=this
            return nextDown
        endmethod
        static method get takes integer stringid returns Data
            local Data d=stringid-(stringid/MAX_STRINGID)*MAX_STRINGID+1
            loop
                if d.stringid==stringid then
                    return d // d.stringid=stringid
                endif
                exitwhen d.drill==0
                set d=d.drill
            endloop
            return d // may be empty or not, but is not the stringid's data, so stack or init here.
        endmethod
        static method getForRead takes integer stringid returns Data
            local Data d=stringid-(stringid/MAX_STRINGID)*MAX_STRINGID+1
            loop
                if d.stringid==stringid then
                    return d // d.stringid=stringid
                endif
                exitwhen d.drill==0
                set d=d.drill
            endloop
            return 0
        endmethod
        static method getForWrite takes Data flushList, integer stringid returns Data
            local Data d=stringid-(stringid/MAX_STRINGID)*MAX_STRINGID+1
            loop
                if d.stringid==stringid then
                    return d // d.stringid=stringid
                endif
                exitwhen d.drill==0
                set d=d.drill
            endloop
            if d.stringid!=0 then
                set d=d.stack(Data.getRandom()) // Get one stacked onto returned one.
                // d has become the result of getRandom here.
            endif
            call d.init(flushList,stringid)
            return d
        endmethod
    endstruct
    
    private function StringID takes string s returns integer
        return s
        return 0
    endfunction
    private function H2I takes handle h returns integer
        return h
        return 0
    endfunction
    
    private struct GTable[MAX_INSTANCES]
        // A "protected" keyword would've been really nice here.
        // Don't play with "trackingPointer" or "base" in your map.
        // Looking for ideas on how to protect them, still.
        Data trackingPointer
        string base
        
        method reset takes nothing returns nothing
            call this.trackingPointer.chainDelete()
            set this.trackingPointer=Data.getPointer()
            //call FlushStoredMission(gc,I2S(this))
        endmethod
        
        private method onDestroy takes nothing returns nothing
            call this.trackingPointer.chainDelete()
            //call FlushStoredMission(gc,I2S(this))
        endmethod
    endstruct
    
    globals//locals
        private integer sid
        private Data dat
    endglobals
    
    //Hey: Don't instanciate other people's textmacros that you are not supposed to, thanks.
    //! textmacro Table__make takes name, type, key
    struct $name$ extends GTable
        static method create takes nothing returns $name$
            local $name$ d=$name$.allocate()
            set d.trackingPointer=Data.getPointer()
            set d.base=I2S(d)+"x"
            return d
        endmethod
        
        method operator [] takes $type$ key returns integer
            set sid=StringID(this.base+$key$)
            set dat=sid-(sid/MAX_STRINGID)*MAX_STRINGID+1
            loop
                if dat.stringid==sid then
                    return dat.value // d.stringid=stringid
                endif
                exitwhen dat.drill==0
                set dat=dat.drill
            endloop
            return 0
        endmethod
        
        method operator []= takes $type$ key, integer value returns nothing
            set sid=StringID(this.base+$key$)
            set dat=sid-(sid/MAX_STRINGID)*MAX_STRINGID+1
            loop
                exitwhen dat.stringid==sid
                if dat.drill==0 then
                    if dat.stringid!=0 then
                        set dat=dat.stack(Data.getRandom()) // Get one stacked onto returned one.
                        // d has become the result of getRandom here.
                    endif
                    call dat.init(this.trackingPointer,sid)
                    return
                endif
                set dat=dat.drill
            endloop
            set dat.value=value
        endmethod
        
        method flush takes $type$ key returns nothing
            local Data d=Data.getForRead(StringID(this.base+$key$))
            if d!=0 then
                call d.delete()
            endif
        endmethod
        
        method exists takes $type$ key returns boolean
            return Data.getForRead(StringID(this.base+$key$))!=0
        endmethod
    endstruct
    //! endtextmacro
    
    //! runtextmacro Table__make("Table","integer","I2S(key)")
    //! runtextmacro Table__make("StringTable","string","key")
    //! runtextmacro Table__make("HandleTable","handle","I2S(H2I(key))")
endlibrary
Vexorian made the original Table.

Updates:
- Version ??: Did an optimisation of Table and left it at that.
- Version 1.02: Did a hack-job optimisation of reading/writing. It's now faster than gamecache by about 7%. Meh.
- Version 1.01: Implemented hashing, and 2d linked lists for collisions and flushing.
- Version 1.00: BETA release/proof of concept.
 

Azlier

Old World Ghost
Reaction score
461
OH MY GOD, JESUS.

I was in bed, three minutes ago thinking over strings and H2I.

String indices start at 0, but H2I indices can become strings... Convert the H2I to a string, get the string ID, boom! Perfect H2I. I was rushing to the computer to test this new idea, but no. You have to get it first. I need to realize things faster.
 

Jesus4Lyf

Good Idea™
Reaction score
397
Lol, c'mon Azlier. I pretty much pointed this out to you in your Trackables2 thread. If I said any more, I may as well not have released the system. XD

But not only did I realise it, I coded the whole thing to a usable extent. :D

Don't feel bad. I'll do a good job of maintaining this system, hopefully. :)

I intend to do linked lists or something for flushing entire tables, for example. And hashing for reusing string ids. I'm sure you wouldn't want to write that. :p

PS. I still object to H2I and say it never needs to be used. HOWEVER, this system is GREAT for attaching to strings, not just handles! ;)

Get ready for what could become a long and/or interesting thread...
 

Azlier

Old World Ghost
Reaction score
461
Tell me when this is totally usable. I want to port Trackable2 to this.
 

Azlier

Old World Ghost
Reaction score
461
I did some experimentation. Instead of H2I, H2S to transform the handle directly into a string saves an I2S call. It actually works, I tested with this simple library and scope and some preplaced units.

Yes, ignore that I'm attaching directly to units. It's just a test, anyway.

EDIT: Nevermind. I2S(H2I()) is better.

JASS:
library HandleData

globals
    private integer array Data
endglobals

private constant function SID takes string s returns integer
    return s
    return 0
endfunction

private constant function H2I takes handle h returns integer
    return h
    return 0
endfunction

function SetHandleData takes handle h, integer d returns nothing
    set Data[SID(I2S(H2I(h)))] = d
endfunction

function GetHandleData takes handle h returns integer
    return Data[SID(I2S(H2I(h)))]
endfunction

endlibrary

scope HandleDataTest initializer Init

private struct Data
    string Str
endstruct

private function Actions takes nothing returns boolean
    local unit u = GetTriggerUnit()
    local Data d = GetHandleData(u)
    if d == 0 then
        set d = Data.create()
        set d.Str = GetUnitName(u)
        call SetHandleData(u, d)
    endif
    call DisplayTextToPlayer(GetLocalPlayer(), 0, 0, d.Str)
    return false
endfunction

private function Init takes nothing returns nothing
    local integer i = 11
    local trigger t = CreateTrigger()
    loop    
        exitwhen i < 0
        call TriggerRegisterPlayerUnitEvent(t, Player(i), EVENT_PLAYER_UNIT_SELECTED, null)
        set i = i - 1
    endloop
    call TriggerAddCondition(t, Condition(function Actions))
endfunction

endscope
 

wraithseeker

Tired.
Reaction score
122
Why on earth do you want to modify table anyway >.>, maybe create a new account in wc3c and see what feedback you have.

I think vexorian himself knows what he is doing though.
 

Azlier

Old World Ghost
Reaction score
461
Well, Table is a vJassified game cache. It's useful because it has no limits.

However, this SID(H2S(handle)) combo is a winner. Almost guaranteed to fit into an array, unless your map leaks insane amounts of strings. Resistant to handle leaks, as well. I like that.

I say resistant because handle leaks can still cause chaos if you try to get/set data into each new handle. However, if you clean and null your handles, new ones should overwrite the old positions and keep the same array slot with SID(H2S())

EDIT: Nevermind, BAD idea! Converting a handle directly into a string leaks a string every use (oddly)! Stick with SID(I2S(H2I())). In fact, I don't think H2S worked at all. I think that it just created a new struct every time and displayed that data. And, with testing, it was. H2S = BAD.
 

Romek

Super Moderator
Reaction score
963
I think this is a bit useless.
In situations where Table is needed, speed isn't.
For all other attaching and stuff, there are much, much better alternatives.
 

wraithseeker

Tired.
Reaction score
122
Rising_Dusk over at MSN said:
Table is a standard created by a cooperation of the greatest minds in JASS and output by Vexorian. It is a standard, which means your systems and maps will be more interfaceable with other systems if you use it and don't deviate from it.

Bad bad!
 

Jesus4Lyf

Good Idea™
Reaction score
397
Azlier, I think H2S would return an invalid string id which would be EXACTLY H2I. So this would not work too well.

>I think vexorian himself knows what he is doing though.

I think you missed the point.

>I think this is a bit useless.
>In situations where Table is needed, speed isn't.

I think you didn't miss the point so much. :(
But I would like to point out that this will use the exact same interface, and hopefully I can develop it to bring absolutely no penalties. Why would you not just copy it over for the speed bonus? :nuts:

>Why on earth do you want to modify table anyway >.>, maybe create a new account in wc3c and see what feedback you have.

You realise that TU Purple was not created by Vex? Someone took his interface and modified the implementation, like this. What I'm doing is perfectly reasonable. And as for what Rising_Dusk said, it still applies to Table-X, so long as I develop it correctly. That's the beautiful thing. It should make no difference if you copy in Table or Table-X, except Table-X will be faster. :thup:
 

Darius34

New Member
Reaction score
30
Would string leaks break this? For example, if it's used in a map that constantly has random strings generated.
 

Azlier

Old World Ghost
Reaction score
461
Well, yes. Eventually. Other methods need to worry about handle leaks, this needs to worry about string leaks. A silly idea I had was to run an init function declaring a bunch of possble handle id strings, to kind of preload the string table.
 

Jesus4Lyf

Good Idea™
Reaction score
397
Would string leaks break this? For example, if it's used in a map that constantly has random strings generated.
I will be updating this and removing this problem, fear not.

A little hashing never hurt anyone. :D

Unless I can get this to be invisibly close to perfectly copying over the top, this is useless. And making mappers beware of string leaks is one such thing that would make this useless.

What is worth understanding: Randomly generated strings are not attached to. This leaves gaps in the string ids that you're attaching to. Therefore, hashing absolutely fixes this problem, as long as I do it right.
And I've done it many times, now. :cool:
 

Cohadar

master of fugue
Reaction score
209
"What on earth is this retard doing now?" I hear you ask... ;)
Nah, that question is getting outdated.

PS: Try not to change Table's interface, anything else you do to it is acceptable. (althou I cannot guarantee there will be no price to pay for it in the lower parts of hell)
 

Flare

Stops copies me!
Reaction score
662
Hmmm, what gave you the idea to simulate gamecache? :D

Might play around with this later, just to test it out to see what I can do with it and to see if there's anything remarkable happening if I disable this system and enable Table

Why on earth do you want to modify table anyway >.>, maybe create a new account in wc3c and see what feedback you have.

I think vexorian himself knows what he is doing though.
Timer systems, knockback systems, damage detection systems (there's probably more) - who knows how many of those are in existence, but people still make them. There's nothing wrong with giving people an alternative.

In situations where Table is needed, speed isn't.
there are much, much better alternatives.
Name the alternatives, explain why they are better? Unless a system is a broken piece of crap, it's not specifically better or worse than the others. Each system will differ in how it's used (e.g Set/Get/Clear for ABC, Set/Get for CSData), and in it's implementation. As long as it works correctly, there's no need for it to be better than anything else, that's the end-user's decision, and you shouldn't be trying to force them away from something that could be perfectly good to use
 

Viikuna

No Marlo no game.
Reaction score
265
Limit of 32000 different strings sounds pretty good, since there isnt really anything that could incerease string id count so much, but I2S(H2I()) and maybe some save&load codes.

Faster version of Table sounds pretty good too. Im waiting to see what emerges form this.
 

Jesus4Lyf

Good Idea™
Reaction score
397
Ah, good to hear some more interest in the system! :)

>Hmmm, what gave you the idea to simulate gamecache?
It occurred to me that there are legitimate reasons to attach to strings, such as text commands. Furthermore, what if you want to attach entire arrays of 2000 things to units? This system handles it without gamecache. :D

>Faster version of Table sounds pretty good too. Im waiting to see what emerges form this.
Right, so people are interested. Good to hear. I'll get to work on this maybe after I play with KT2 a little more. I'm still trying to put the final touches on the implementation there, to remove the last of the cons I'd like to see disappear.

Although the limit so far is obviously 32000 strings (which -may- be reached) I'd like to implement hashing. This would blow it out further than the actual limit you specify, and along with flush/destroy methods this could actually "recycle" string ids (not literally, but as far as the system is concerned).

As said, I'll get on it soon enough. ;)
 

Viikuna

No Marlo no game.
Reaction score
265
I dont know much about hashing, but hashes can have collinsions right? What kind of method are you going to use to avoid possible problems caused by them?
 
General chit-chat
Help Users
  • No one is chatting at the moment.

      The Helper Discord

      Members online

      No members online now.

      Affiliates

      Hive Workshop NUON Dome World Editor Tutorials

      Network Sponsors

      Apex Steel Pipe - Buys and sells Steel Pipe.
      Top