High Memory Usage

C

curdude

Member
Hello,

Having some memory usage issues. The memory usage according to the WHM panel is right around 85+% for the past 2 months. When KH migrated the VPS to WHM/cPanel it first ran around 65 to 70% then slowly ramped up to 90% and up making the VPS run out of memory in high usage burst times. At that point we upgraded to the next package adding another 1G (up to 3G). After the addition of the 1G the memory usage decreased to about 55% to 60% usage. It worked great for a while but noticed that the system was slowly taking more memory as it aged. As aforementioned, running at 85+% now.

Opened a ticked to have KH looked into this. The techs looked at the system, did some PHP optimizations and also MySQL optimizations. To be honest I was very impressed. The memory usage dropped back to around 55% to 60%. This was a week ago. We have been monitoring and it has again slowly increased back to the 80+% range eating up all the gains made. I followed up on the ticket and seems the MySQL is hogging the memory again. The tech now is telling us to get a MySQL developer to help with MySQL database optimization. Wow! Got a few WP sites running and a few Drupal sites. Nothing too heavy, no heavy traffic. Get a MySQL developer to help with MySQL database optimization?? Been through a few hosting companies but this is the first time I have heard this.

Wondering if anyone is having the same issue and if anyone have any suggestions besides add more memory or “Get a MySQL developer to help with MySQL database”.

BTW, never had any memory issues while running under PLESK (@ 2G). I am yet to be impressed by WHM/cPanel.

Thank you in advance.
 
I think what's happening to you is something oft confused in Linux - Kernel memory caching/buffering.

What part of WHM are you viewing the memory usage? If you're viewing "Current Memory Usage" on the Server Information page then free + cache = true free.

https://my.knownhost.com does not count memory quite like this. It only looks at used which includes cached and since cached is technically speaking "in use" by the kernel it's a catch-22 on how we present it to customers. It's technically in use, but only because the kernel found a way to take advantage of otherwise idle resources. If something in your system needs this memory, the kernel will give it to that and take it out of cached.

http://www.linuxatemyram.com/ is a pretty good explanation of all of this.

As for MySQL using memory - MySQL is a memory-dependent application. Memory is important to MySQL more than any other single resource. Take away memory, you get slowness and higher CPU consumption. I've done a lot of work on MySQL tweaking for extremely intensive applications and through doing so gained a deep understanding of how to get resource usage as low as possible while not harming performance. If you give me a ticket number I'd be happy to have a peek at your config.

Let KnownHost manage your VPS - US industry specialists with 24/7 support and 99.9% uptime.
 
Hello Johnathan,

This is where I see memory usage:

Home »Server Status »Service Status
System Information

Memory Used 82.15% (2,584,316 of 3,145,728)

Ticket numbers are: #655721; #658127

I see the MySQL takes 38+% of the slice of the memory at Home »Server Status »Daily Process Log . I do not know if this is reasonable for the few WP sites and even fewer Drupal sites that are on the VPS.

When the memory used goes above 80% I get nervous because you can tell that the sites are loading slower.

Appreciate your follow up.

Thank you.
 
So here are my findings after looking at your server briefly.

If anything, MySQL needs more memory. The fact that it doesn't have enough very likely explains why you see things slow down at 80+% usage.

You have 719MB of InnoDB data, yet the innodb_buffer_pool_size on your VPS is only 128M. InnoDB is very memory hungry - but it's also very fast. Your innodb_buffer_pool_size should always be larger than the combined size of your InnoDB data. I recommend setting the innodb_buffer_pool_size to 1G here to give it a little room for growth.

table_open_cache needs to be raised - it's set at 512, you have 7M table opens recorded. Basically you have far more tables than are allowed to be cached open at once. Increasing this figure will speed things up but also eat some more memory.

I see somewhere around 25 instances of your VPS running out of memory. You can see this with the command cat /proc/user_beancounters within your VPS.

Other than that, your VPS looks great. MySQL consuming 38% of the systems RAM is not surprising. In fact, I just checked a few of our internal servers and gathered some stats of how much RAM MySQL on each server is using out of the total, and then the even more relevant figure, what percent of the total used memory on the server is consumed by just MySQL:

% of total memory / % of used memory
Our support system: 58%/72%
Billing system: 43.7%/95%
This forum: 6.8% (it has far more RAM than it needs)/80%
An internal monitoring system: 51.7%/95% (this is quite a different use case, however)

Your server: 39%/71.2%

Hopefully that puts things into better perspective. In a general LAMP stack (which cPanel, Plesk, etc. are all built around) MySQL is expected to be the single largest consumer of memory, often times in excess of 50% of the system's total memory and 80% of the used memory footprint.

TL;DR: Upgrade your VPS and feed MySQL more RAM to speed up your sites.
 
Hello Johnathan,

Thank you for your feedback and analysis.

So in conclusion you are suggesting to upgrade the VPS to the next VPS plan and do the changes then? Or can we do anything else with what we have now?

Should we upgrade to the VPS plan or add Ram. What would you suggest?

Thank you.
 
In technical terms, it can be possible to decrease your overall memory footprint t by giving more memory to what really needs it but that's not going to gain us enough room for comfort even if it did work here given how close you are to your limit having run out some, and how much more MySQL really needs.

I'd recommend going to the next plan up over an a la carte upgrade. At the end of the day you get more value this way.

Our SSD plans are designed to be lower disk space, more "oomph" so if you don't need the added disk space from the next larger VPS plan it might make sense to look at the SSD line which have slightly more RAM anyway, and that should speed up your site and potentially help lower RAM usage anyway since things will process and get "in and out" faster from the additional speed SSDs bring to the table. SSDs are especially helpful to transactional operations, aka, MySQL ;)
 
Hello J,

Makes sense. One more thing, moving to SSD does that mean all new IPs etc or is it a simple switch over?

Thanks
 
Hello J,

I requested an upgrade to the next VPS level. Decided not to go SSD yet.

Your above mentioned tweaks cannot be made at the WHM interface or at least I cannot find it. Please advise.

Also what would you recommend the table_open_cache be set at and where is this on the interface?

Thanks.
 
Looks like I forgot to reply - these changes must be done via SSH in /etc/my.cnf. Feel free to open up a ticket and request me citing this thread and I'll make the adjustments for you.

For table_open_cache I'd recommend 4096 for starters.
 
Hello Jonathan,

Thanks for the follow up.

See ticket#658875

KH tech made some changes. Appreciate if you can take a look and optimize where you see fit.

FYI, mem usage back up to 80+% again even after the upgrade. Very very odd behavior.

Thank you in advance.
 
Sorry for chiming in late, wish I caught this before you already did that upgrade, but I too strongly recommend the SSD over just another Memory and HDD upgrade, so much so that I personally would opt for a little less memory and less space with an SSD, over more memory and more HDD space. SSDs are perfect for database intensive servers, which WordPress and especially Drupal sites make. Less IO time from the disk often results in less resource usage all around, allowing more room for MySQL usage and more resources for even more sites. IO time for any needed swap space becomes much faster when/if your Memory usage is nearly full so your applications don't have to wait as long on disk caching, to the point you probably won't even notice that lag. I assume from your OP that you were on VPS-3 (3GB ram / 105GB hdd), I think Jonathan would agree that even moving to SSD-2 (2.75GB ram / 70GB ssd for just $5 more a month), even though it reduces your ram by 0.25 GB, would greatly benefit you.

If you have cacheing plugins on your sites those are likely the cause. They do tend to cause memory usage to creep up over time. If you do, try disabling them all, monitor your sites for a couple days and see what happens.

Cheers!
 
  • Like
Reactions: VoX
I tweaked a few more settings and moved you from MySQL 5.6 to MariaDB 10.1.

Things look pretty happy right now, have a peek :D

Like @phpAddict said...SSDs - they're awesome.
 
Hello Jonathan,

I think I'm gonna have to downgrade now to a less beefy VPS. :p

I thought the tech last weekend did a great job bringing the Mem Usage down to around 65% very stable. Looked good for a day.

However, I am blown away of what MariaDB did to the Mem usage. Mem usage down to 40%, WOW!

Now lets see what the developers on the system says.

I agree with the SSD comment. That's going to be the next upgrade.

Appreciate the help guys. I'll keep ya'll posted.
 
Downgrade? Don't do that. Having extra room is a good thing. It prevents OOMs when something spikes unexpectedly.

MariaDB is awesome. It's the original MySQL devs leading it up so the project is really gaining momentum. Ever since Oracle bought MySQL it seems they've only focused on their enterprise offering, go figure.
 
Thanks @KH-Jonathan for the respond. I have some resellers on my VPS and I don't want to do anything that will break CMS/Forums that might be on some of these resellers accounts.
 
You must log in or register to reply here.
Top