Memory problems again ...

[ghost]

Hey Guys, i think memory management should be refactored again... if i am unsetting a DB_ORM_MODEL with unset() method or =NULL then the memory is not given free... i think __desctruct() have to be implemented on every object which have nested objects...

i am importing a 26MB file and have to create ove 50.000 ORM Models to save the complete file to my database. this fails because i get an memory error, max allowed size exhausted etc. i optimized my code to unset every object if it is not needed anymore, but php gives the memory not free... could you please refactor this?

[ghost]

heres an example code... if you execute this you can see in task manager how the memory is appended and appended and appended.... unset has no result. i tried to extend various classes with a destruct method, but at this moment i could not get a working result which cleans up the memory ....

edit:: if the ->save() lines are commented out, everything works fine! therefore it has something to do with the save process...

 for ($i = 0; $i < 500; $i++) {
            $test = new Model_Leap_Model1();
            $test->M1_KEY = -1;
            $test->ITEM_KEY = $i;
            $test->save();

            unset($test);

            for ($j = 500; $j < 1000; $j++) {
                $test2 = new Model_Leap_Model2();
                $test2->M2_KEY = -1;
                $test2->ITEM_KEY = $i+$j;
                $test2->save();

                unset($test2);

                for ($k = 1000; $k < 1500; $k++) {
                    $test3 = new Model_Leap_Model3();
                    $test3->M3_KEY = -1;
                    $test3->ITEM_KEY = $i+$j+$k;
                    $test3->save();

                    unset($test3);
                }
            }
        }

[bluesnowman]

I believe unset does not actually destroy the object, but rather marks it for garbage collections. PHP does have a way to force garbage collection using gc_collect_cycles(). See the following links for more information on garbage collection in PHP:

• http://de.php.net/manual/en/function.gc-collect-cycles.php
• http://de.php.net/manual/en/features.gc.php
• http://php.net/manual/en/features.gc.collecting-cycles.php

[bluesnowman]

@ViperNeo It looks like the problem might be caused by some circular references that are not getting freed properly. Short term solution: add some code to free them properly. Long-term solution: eliminate all circular references. Although I will look into finding a solution, if you have any immediate solutions I am totally open to implementing them.

[bluesnowman]

@ViperNeo Here is a good article that explains why your __destruct() did not work:

• http://www.alexatnet.com/articles/optimize-php-memory-usage-eliminate-circular-references

Try calling __destruct() explicitly before calling unset() as the article in my last post suggests, and also try calling gc_collect_cycles(). If this works for you, please let me know (and if you don't mind post your solution) and I will add it to the documentation until we can find a more permanent solution. I have some ideas on how to reduce some of these circular references so I am going to play with some of them over the next few days and see what I can come up with.

[ghost]

i will give the gc_collect_cycles method a try...

in the meantime i added this function to the Core_Object:

public function __destruct() {
    foreach (get_object_vars($this) as $obj) {
        $this->destroy_var($obj);
    }
}

private function destroy_var(&$obj) {

    if (is_object($obj)) {
        $obj = null;
        unset($obj);
    }
    else if (is_array($obj)) {
        foreach ($obj as $o) {
            $this->destroy_var($o);
        }
    }
    else {
        $obj = null;
        unset($obj);
    }
}

but that has no effect... if i have results, i will let you know!

[ghost]

so far gc_collect_cycles has no effect too. memory usage is still increasing.

[bluesnowman]

@ViperNeo I added some additional methods to help manage resources on the 3.3/develop branch that you may want to try playing around with.

Try calling the dispose() method after saving and before you unset. Like so:

$test = new Model_Leap_Model1();
$test->M1_KEY = -1;
$test->ITEM_KEY = $i;
$test->save();
$test->dispose();
unset($test);

[ghost]

i have tested it... the memory usage is now a little bit better, but the execution time is doubled.

[bluesnowman]

@ViperNeo Did you try calling $test->dispose(FALSE); to see if performance improved?

[ghost]

with dispose FALSE the performance improves, but big memory leaks again. i have implemented now a database abstraction layer at my own and do not using leap any more.

import of my big 26MB file now takes 1 minute with a consistent memory usage of max. 8MB