Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

easier to read memory usage #3987

Open
columbian-chris opened this issue Oct 18, 2024 · 2 comments · May be fixed by #3990
Open

easier to read memory usage #3987

columbian-chris opened this issue Oct 18, 2024 · 2 comments · May be fixed by #3990
Milestone

Comments

@columbian-chris
Copy link
Contributor

columbian-chris commented Oct 18, 2024

Just a minor improvement, but I'm doing a large sync via WP-CLI and once the size gets into the tens of thousands of megabytes, it becomes harder to read:

...
Processed posts 173250 - 173400 of 570420. Last Object ID: 1853974
Processed posts 173400 - 173550 of 570420. Last Object ID: 1853405
Processed posts 173550 - 173700 of 570420. Last Object ID: 1852902
Processed posts 173700 - 173850 of 570420. Last Object ID: 1852292
Processed posts 173850 - 174000 of 570420. Last Object ID: 1851732
Time elapsed: 00:00:03.000000 (+0)
Memory Usage: 34589.75mb (Peak: 34593.84mb)
Processed posts 174000 - 174150 of 570420. Last Object ID: 1851110
Processed posts 174150 - 174300 of 570420. Last Object ID: 1850562
Processed posts 174300 - 174450 of 570420. Last Object ID: 1849972
Processed posts 174450 - 174600 of 570420. Last Object ID: 1849393
Processed posts 174600 - 174750 of 570420. Last Object ID: 1848665
Processed posts 174750 - 174900 of 570420. Last Object ID: 1848056
Processed posts 174900 - 175050 of 570420. Last Object ID: 1847495
Processed posts 175050 - 175200 of 570420. Last Object ID: 1846947
Processed posts 175200 - 175350 of 570420. Last Object ID: 1846300
Processed posts 175350 - 175500 of 570420. Last Object ID: 1845582
Time elapsed: 00:00:03.000000 (+0)
Memory Usage: 34872.27mb (Peak: 34876.36mb)
Processed posts 175500 - 175650 of 570420. Last Object ID: 1845002
Processed posts 175650 - 175800 of 570420. Last Object ID: 1844436
...

Could we instate gigabytes instead of thousands of megabytes when applicable?

Maybe something like (untested):

/includes/classes/Command.php line ~1342:

- $current_memory = round( memory_get_usage() / 1024 / 1024, 2 ) . 'mb';
- $peak_memory    = ' (Peak: ' . round( memory_get_peak_usage() / 1024 / 1024, 2 ) . 'mb)';
- WP_CLI::log( WP_CLI::colorize( '%Y' . esc_html__( 'Memory Usage: ', 'elasticpress' ) . '%N' . $current_memory . $peak_memory ) );
+ $current_memory = memory_get_usage() / 1024 / 1024;
+ $current_memory = ( $current_memory > 10000 ) ? round( $current_memory, 2 ) / 1024 . 'gb' : round( $current_memory, 2 ) . 'mb';
+ $peak_memory    = memory_get_peak_usage() / 1024 / 1024;
+ $peak_memory    = ( $peak_memory > 10000 ) ? round( $peak_memory, 2 ) / 1024 . 'gb' : round( $peak_memory, 2 ) . 'mb';
+ WP_CLI::log( WP_CLI::colorize( '%Y' . esc_html__( 'Memory Usage: ', 'elasticpress' ) . '%N' . $current_memory . ' (Peak: ' . $peak_memory . ')' ) );

Or maybe throw in a new method to handle this and include more than just mb and gb.

I think also the posts numbers could benefit from a number_format() as well to include commas for easier reading, but I couldn't find where that is sourced in the code.

@felipeelia
Copy link
Member

Thanks @columbian-chris. I've added that as a possible enhancement for future releases, but if you have the availability to craft a Pull Request we can try to add that to our next release.

@columbian-chris columbian-chris linked a pull request Oct 28, 2024 that will close this issue
4 tasks
@columbian-chris
Copy link
Contributor Author

Another issue I'm finding in my WP-CLI syncs is the timer suddenly resetting after a certain amount of time. Here's a couple different syncs I did that both provide an example:

...
Processed posts 60450 - 60600 of 571371. Last Object ID: 2360984
Processed posts 60600 - 60750 of 571371. Last Object ID: 2360289
Processed posts 60750 - 60900 of 571371. Last Object ID: 2359721
Processed posts 60900 - 61050 of 571371. Last Object ID: 2359256
Processed posts 61050 - 61200 of 571371. Last Object ID: 2358675
Processed posts 61200 - 61350 of 571371. Last Object ID: 2358137
Processed posts 61350 - 61500 of 571371. Last Object ID: 2357521
Time elapsed: 00:16:34.030000 (+23.225)
Memory Usage: 7425.17mb (Peak: 7428.55mb)
Processed posts 61500 - 61650 of 571371. Last Object ID: 2356882
Processed posts 61650 - 61800 of 571371. Last Object ID: 2356275
Processed posts 61800 - 61950 of 571371. Last Object ID: 2355709
Processed posts 61950 - 62100 of 571371. Last Object ID: 2355059
Processed posts 62100 - 62250 of 571371. Last Object ID: 2354480
Processed posts 62250 - 62400 of 571371. Last Object ID: 2353820
Processed posts 62400 - 62550 of 571371. Last Object ID: 2353118
Processed posts 62550 - 62700 of 571371. Last Object ID: 2352525
Processed posts 62700 - 62850 of 571371. Last Object ID: 2351908
Processed posts 62850 - 63000 of 571371. Last Object ID: 2351162
Time elapsed: 00:00:01.000000 (+-993.03)
Memory Usage: 7601.48mb (Peak: 7604.51mb)
Processed posts 63000 - 63150 of 571371. Last Object ID: 2350448
Processed posts 63150 - 63300 of 571371. Last Object ID: 2349869
Processed posts 63300 - 63450 of 571371. Last Object ID: 2349250
Processed posts 63450 - 63600 of 571371. Last Object ID: 2348614
Processed posts 63600 - 63750 of 571371. Last Object ID: 2347904
Processed posts 63750 - 63900 of 571371. Last Object ID: 2347331
Processed posts 63900 - 64050 of 571371. Last Object ID: 2346786
Processed posts 64050 - 64200 of 571371. Last Object ID: 2346151
Processed posts 64200 - 64350 of 571371. Last Object ID: 2345480
Processed posts 64350 - 64500 of 571371. Last Object ID: 2344973
Time elapsed: 00:00:01.000000 (+0)
Memory Usage: 7777.85mb (Peak: 7781.25mb)
Processed posts 64500 - 64650 of 571371. Last Object ID: 2344293
Processed posts 64650 - 64800 of 571371. Last Object ID: 2343679
Processed posts 64800 - 64950 of 571371. Last Object ID: 2342970
Processed posts 64950 - 65100 of 571371. Last Object ID: 2342439
...

It reports Time elapsed: 00:16:34.030000 (+23.225) and then the next it reports Time elapsed: 00:00:01.000000 (+-993.03).

...
Processed posts 58,050 - 58,200 of 571,364. Last Object ID: 2370731
Processed posts 58,200 - 58,350 of 571,364. Last Object ID: 2370157
Processed posts 58,350 - 58,500 of 571,364. Last Object ID: 2369594
Time elapsed: 00:15:58.630000 (+23.134)
Memory Usage: 6.92gb (Peak: 6.92gb)
Processed posts 58,500 - 58,650 of 571,364. Last Object ID: 2368914
Processed posts 58,650 - 58,800 of 571,364. Last Object ID: 2368281
Processed posts 58,800 - 58,950 of 571,364. Last Object ID: 2367718
Processed posts 58,950 - 59,100 of 571,364. Last Object ID: 2367110
Processed posts 59,100 - 59,250 of 571,364. Last Object ID: 2366504
Processed posts 59,250 - 59,400 of 571,364. Last Object ID: 2365729
Processed posts 59,400 - 59,550 of 571,364. Last Object ID: 2365186
Processed posts 59,550 - 59,700 of 571,364. Last Object ID: 2364632
Processed posts 59,700 - 59,850 of 571,364. Last Object ID: 2363986
Processed posts 59,850 - 60,000 of 571,364. Last Object ID: 2363360
Time elapsed: 00:16:24.450000 (+25.819)
Memory Usage: 7.09gb (Peak: 7.09gb)
Processed posts 60,000 - 60,150 of 571,364. Last Object ID: 2362763
Processed posts 60,150 - 60,300 of 571,364. Last Object ID: 2362163
Processed posts 60,300 - 60,450 of 571,364. Last Object ID: 2361561
Processed posts 60,450 - 60,600 of 571,364. Last Object ID: 2360971
Processed posts 60,600 - 60,750 of 571,364. Last Object ID: 2360272
Processed posts 60,750 - 60,900 of 571,364. Last Object ID: 2359697
Processed posts 60,900 - 61,050 of 571,364. Last Object ID: 2359244
Processed posts 61,050 - 61,200 of 571,364. Last Object ID: 2358639
Processed posts 61,200 - 61,350 of 571,364. Last Object ID: 2358119
Processed posts 61,350 - 61,500 of 571,364. Last Object ID: 2357484
Time elapsed: 00:00:01.000000 (+-983.45)
Memory Usage: 7.25gb (Peak: 7.26gb)
Processed posts 61,500 - 61,650 of 571,364. Last Object ID: 2356853
Processed posts 61,650 - 61,800 of 571,364. Last Object ID: 2356248
Processed posts 61,800 - 61,950 of 571,364. Last Object ID: 2355691
Processed posts 61,950 - 62,100 of 571,364. Last Object ID: 2355026
Processed posts 62,100 - 62,250 of 571,364. Last Object ID: 2354435
Processed posts 62,250 - 62,400 of 571,364. Last Object ID: 2353799
Processed posts 62,400 - 62,550 of 571,364. Last Object ID: 2353101
Processed posts 62,550 - 62,700 of 571,364. Last Object ID: 2352500
Processed posts 62,700 - 62,850 of 571,364. Last Object ID: 2351877
Processed posts 62,850 - 63,000 of 571,364. Last Object ID: 2351124
Time elapsed: 00:00:01.000000 (+0)
Memory Usage: 7.43gb (Peak: 7.43gb)
Processed posts 63,000 - 63,150 of 571,364. Last Object ID: 2350424
Processed posts 63,150 - 63,300 of 571,364. Last Object ID: 2349823
Processed posts 63,300 - 63,450 of 571,364. Last Object ID: 2349236
Processed posts 63,450 - 63,600 of 571,364. Last Object ID: 2348599
...

This one a similar story. Time elapsed: 00:16:24.450000 (+25.819) then Time elapsed: 00:00:01.000000 (+-983.45).

Any idea what could be causing this?

@felipeelia felipeelia modified the milestones: Future Release, 5.2.0 Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants