• Registered: 2007-12-26
  • Posts: 114

Topic: Maxclients and memory

My app has been running on a 2GB Ubuntu cloud server for about 10 months without issues until 2 days ago. The apache web server suddenly stopped workign with an error message maxclients had been reached. Maxcklients was set at 150 at the time.

Now I notice that my memory usage is climbing constantly, at about 1GB per day.

My rails production log is normal with no errors reported, other than a 404 for robots.txt and some apple mobile precompiled jpgs.

Any suggestions where i can start to track this down?

  • jamesw
  • Administrator
  • Offline
  • From: Leeds, UK
  • Registered: 2008-06-07
  • Posts: 2,701

Re: Maxclients and memory

It's either a memory leak or a scaleability issue.
Have you had an inrease in traffic?

What you want and what you need are too often not the same thing!
When your head is hurting from trying to solve a problem, stop standing on it. When you are the right way up you will see the problem differently and you just might find the solution.
(Quote by me 15th July 2009)
  • Registered: 2007-12-26
  • Posts: 114

Re: Maxclients and memory

I should be more specific. My rack space ubuntu 10.4 server is 2GB and has been running 450 days. It recently ran about 9 months without a reboot and rarely db:sessions:clear.

The app is a sports app for basketball orgs and the like to manage their comps. I have about 24 sports organizations who use it heavily to manage their entire competitions plus 80,000 public views a month, mostly to get league results and ladders. Currently Rails 3.2.3, Ruby 1.9.3, Apache 2, Passenger 3.0.11, Postgres 9.0.3. App size 300MB.

Traffic is rising but its not huge. Some processes such as drawing games and booking courts can take a few seconds to return but theses are fairly infrequent.

Apache config

<IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    ServerLimit          256
    MaxClients           256
    MaxRequestsPerChild  0

  KeepAlives off

  PassengerMaxPoolSize 4

  PassengerMaxInstancesPerApp 0

Must be used with RailsBaseURI in Vhost
RailsAutoDetect off

LoadModule passenger_module /home/mark/.rvm/gems/ruby-1.9.3-p125/gems/passenger$
   PassengerRoot /home/mark/.rvm/gems/ruby-1.9.3-p125/gems/passenger-3.0.11
   PassengerRuby /home/mark/.rvm/wrappers/ruby-1.9.3-p125/ruby

I use external email at Rackspace.

top reports soon after reboot

Mem:   2051088k total,   453452k used,  1597636k free,    11312k buffers
Swap:  4194300k total,        0k used,  4194300k free,   127848k cached

This rises slowly to about 500,000 free after 12 hours or so.

I'm building a new server image with latest Ruby Rails, etc to see if the problem continues.

It's at sportdata.com.au

Last edited by markhorrocks (2012-10-29 06:43:03)

  • jamesw
  • Administrator
  • Offline
  • From: Leeds, UK
  • Registered: 2008-06-07
  • Posts: 2,701

Re: Maxclients and memory

First of all can I highly recommend Unicrn nd nginx rather tha\n passenger and apache. It's a hugely improved combination See - http://railscasts.com/episodes/293-nginx-unicorn
It's what I use on an Ubuntu setyup on Linode with great success.

Every visit invokes a new instance of the app. This could easily see you running out of memory.
There are tools available to help track down memory leaks

1) use the Bleak House plugin - http://www.rubyinside.com/bleakhouse-to … s-470.html
2) implement Scout monitoring specifically the memory usage profiler https://scoutapp.com/ Might be worth having a look at the ree trial options.
3) Try newrelic http://newrelic.com/ or another simple memory usage logger https://github.com/binarylogic/memorylogic

The first is a graphical exploration of memory usage by objects in the ObjectSpace.
The last two will help you identify specific usage patterns that are inflating memory usage, and you can work from there.
Watch anything that's dealing with file io, image processing or working with large strings.

Tjis could easily be an external process such as image magic or a cron job. Have you set up any backgroud processing recently?

What you want and what you need are too often not the same thing!
When your head is hurting from trying to solve a problem, stop standing on it. When you are the right way up you will see the problem differently and you just might find the solution.
(Quote by me 15th July 2009)
  • Registered: 2007-12-26
  • Posts: 114

Re: Maxclients and memory

No background processing set, I am running ImageMagick. I updated the box for the first time in a few months and troubles started shortly thereafter.

I will try using Nginx and Unicorn as well as the tools you suggest and will report back here.

Thanks

  • Registered: 2007-12-26
  • Posts: 114

Re: Maxclients and memory

I eventually created a new Ubuntu 12.04 instance on Rack Space, reinstalled everything and moved the data over. All seems good now.

Setup now is Nginx / Unicorn with rbenv, Ruby 1.9.3-p325, Rails 3.2.9 and Postgresql 9.2.

What i would love to see is a a clear explanation of how to serve multiple RoR domains on Nginx / Unicorn. I can't figure out whether it's better to put the server blocks in files in sites=available or in the nginx config. I have my main site working on a server as the single default domain as described by the Railscast. but progressing beyond that to multiple domains seems much more arcane than apache for me. If anybody has some links, I would love to see them.

Last edited by markhorrocks (2012-12-06 00:26:14)

  • jamesw
  • Administrator
  • Offline
  • From: Leeds, UK
  • Registered: 2008-06-07
  • Posts: 2,701

Re: Maxclients and memory

Hi Mark!
I use sites_available if that's any help.
Glad you got everything running ok. nginx and unicorn is a superb combination, just trying to figure out how to set up live streaming! lol!

What you want and what you need are too often not the same thing!
When your head is hurting from trying to solve a problem, stop standing on it. When you are the right way up you will see the problem differently and you just might find the solution.
(Quote by me 15th July 2009)