Apache2 process not releasing

[webdesign309]

So I am working with a large Park District as a client. There site was launched a couple months ago and they say that recently they have seen there server crash several times. They looked into the cause of this problem and are pointing fingers at tribes event calendar. We cannot get direct access to their servers but he did give me a copy of the files and database that they are using. They are quite large. I have loaded them onto a development server: http://ppd.webdesign309.com. I haven’t been able to create the same errors that they had yet on our server. I am going to copy and paste the direct email he sent to me. If anyone has any ideas please let me know. I am not a server admin so I can only speak to the code side of things. Thanks!
/////////////////////////////////
///////EMAIL HE SENT TO ME///////
/////////////////////////////////
I am following up on the continued CPU and Memory usage problem that our new site has been having. I have been able to find out which plugin(s) has been causing the problem and we currently have the offending plugin(s) disabled until we can reach resolution.

What we were seeing when the website was running is that it would start multiple instances of apache2 (normal for websites to do this), but it wasn’t releasing the processes when a user would leave the website and the memory and cpu usage would keep snowballing until the server locked up.

Here is a screenshot of top from our server while the problem is happening:

[cid:e9c5be43-04bb-2c0b-183e-59c306ea8c2b]

There were dozens of these processes spawned, and you can see how long they were open, the amount of memory they were consuming and the amount of CPU they were requesting. I looked at just one of these processes and found that 2 plugins where causing a loop that did not allow the apache2 process to die properly. The plugins that were causing the problem are:

The Events Calendar
&
Events Calendar Pro

Once we disabled those plugins, we now have a normal system that releases the apache2 processes, and our top now looks like this:

[cid:746a2cb8-7450-af69-c043-6947fb46efdf]

The website has never been faster since we disabled the plugins. I did watch the process while it was causing the problem and I have attached a text file with the loop the plugin had created that caused the processes to race out of control.

/////////////////////////////////
/////END EMAIL HE SENT TO ME/////
/////////////////////////////////

Here are the images he attached as proof. http://ppd.webdesign309.com/screenshot74.png
http://ppd.webdesign309.com/screenshot73.png
http://ppd.webdesign309.com/runawayplugins.txt

I will take any opinions on the matter and will try to provide as much access as possible.

[Barry]

Hi webdesign309,

That’s definitely going to be frustrating and we’d love to help – however, I’d like to note first of all this is the first time I recall someone raising this specific issue and I’m curious as to why it would happen only for this particular deployment of our plugins (I’m correct in understanding this is not something you can replicate)?

I looked at just one of these processes and found that 2 plugins where causing a loop that did not allow the apache2 process to die properly.

Do you or does your client maintain a duplicate site for testing purposes that lives in an otherwise identical hosting environment and can this be replicated there, with no other plugins activated and only a default, unmodified theme? Are certain levels of traffic necessary to trigger these “loops”?

Thanks!

[webdesign309]

His response to the issue was the following:

//////////////////////
///EMAIL HE SENT ME///
//////////////////////
For the first few hours the site is live it is OK. After the site comes under heavy load (3,000+ views per day) is when it starts acting up. I can prove it is the plugins causing the problem because we have them disabled and are not experiencing the problem. We have had this same site on 3 different servers, two of which I did not configure, and the problem existed.
///////////////////////
///END EMAIL HE SENT///
///////////////////////

It doesn’t sound like he has a server ready to test this out.

Obviously I understand that just because the plugin is disabled and all seems well does not necessarily mean that the plugin is to blame for the issue. Since I have never run into debugging on this level of implementation is there any recommendations you have to help me remedy this situation?

I do have a copy of all the code and such on our server. Are there any known plugins that conflict with The Events Calendar?

[Barry]

Thanks for sharing.

It sounds then primarily like a scaling issue. Our plugins inevitably mean different and arguably heavier-weight queries will run than would typically be the case if WordPress were running purely as a blog, for example, with no other plugins running on top of it.

If a certain tipping point is being reached, would it be possible to explore increasing the server resources/number of servers used to deal with requests or implementing caching at different layers (including query and object caching)?

Ultimately, yes, our plugin will add some extra demands. The practical reality though is if server resources can’t match the demands of the application when under load then either efficiencies need to be made in order to better utilize available resources, or additional server resources must be added, or a combination of those things – I’m not sure in this case that there is really any more to it than that.

Does that help at all? We’re definitely open to any further feedback you might provide.

 

[marcosnoy]

I believe I have the same problem on two different domains.
Last Friday I installed the events calender and the events calendar pro plugins and added one event (just to test it). It was nog accessible for visitors.

The next day my site was offline, and i received an email from my hosting provider that a script or application from my site was sending lots of requests causing their server to crash. This happened three times so far.

I tried to change several settings but as from now i’m positive it’s the events calendar causing my problems.

Yesterday evening I installed both of your plugins on a different domain (different server, same host) and a couple of hours ago this site crashed.

I’m no expert on databases and such but from the first comment of this topic my guess is that the plugin does something to the databases that is not making my hosting provider happy 😉

I would really like to make this plugin to work. I purchased it a while ago but had to find another solution because it conflicted with every theme that i had at that time. But i recently purchased the espresso theme and would really like to see these two together.

Thanks in advance

[webdesign309]

I am glad to hear that I am not the only one to have issues like this with the plugin. Hopefully they will help us to resolve our mutual problem.

[marcosnoy]

Same here, I’ve searched for quite a while before I found a similar problem. My hosting provider already completely deleted one of my websites because they were fed up with their crashes.

[Barry]

Hi marcosnoy, I’m sorry to hear you’ve also been facing problems along similar lines.

It would be appreciated though if you could form your own thread if you need further assistance with this issue as there are some fundamental differences between what you are describing and what webdesign309 has described:

Last Friday I installed the events calender and the events calendar pro plugins and added one event (just to test it). It was nog accessible for visitors.

If things are failing with only a single test event then that is definitely not par for the course and unless there is some sort of conflict with other plugins/your theme it may be that your own hosting environment is seriously underpowered, or poorly configured. It’s also rather different to the circumstances that are triggering problems in webdesgin309’s case.

The sheer number of people successfully running Events Calendar PRO with some pretty standard web hosting tends to suggest your own experiences are pretty atypical – but that said we’re definitely happy to take a slightly closer look at your problem if you want to create a new thread.

I am glad to hear that I am not the only one to have issues like this with the plugin. Hopefully they will help us to resolve our mutual problem.

What I can say is that, incrementally, the plugin is improving on multiple levels over time – making our interactions with the database more efficient is part and parcel of that. Yet at the same time I’d stand by my last reply: you, or your client, are going to require sufficient resources to field the number of incoming requests at peak times.

Following from my previous reply:

• Have you been able to explore caching options?
• Have you looked at increasing overall server resources to accommodate these spikes?

Thanks!

[Barry]

Hi guys, this was inadvertently marked as ‘resolved’, but (webdesign309 in particular) please do feel free to follow up once you’ve had a chance to review things. Apologies for any confusion!

[webdesign309]

This reply is private.

[Barry]

I’m not really sure what you’d be expecting us to look for there: didn’t you state that the problem you have described does not occur on your test site?

[webdesign309]

This reply is private.

[Barry]

I can completely understand your and your client’s frustration – however, you also need to understand that the level of support we can offer has limitations.

Taking a very high level view the problem is as simple as this: when the server is under load it is hitting problems. Now, what we can’t change overnight is how the core of our plugin operates – that’s simply not going to happen – and all we can say in terms of onward development is that we continually strive to make it a smarter, more efficient, more user friendly plugin with each iteration.

With that in mind, you’re best bet in the short term is to look at options on the live site. You don’t necessarily have to scale up or scale out, you could start by exploring caching options as already covered.

This web server has 16GB of memory and 4 3Ghz processors. I’m also curious to know if anyone actually looked at the log information I sent.

We have looked at the information that was provided, yes. Do also note that 16GB of RAM and 4 3GHz processors doesn’t actually mean a whole lot by itself: there is a plethora of information we are missing here. I don’t know if this is a virtual server, how many other sites it might be serving and what sort of load they are experiencing, what sort of connectivity it has to the wider internet or if the database resides on the same machine or a different one – to name but a handful of variables.

I’m not expecting you to answer those questions, because ultimately this is going beyond the level of support we can offer – I just wanted to give a sense of how complicated a problem this can be – and why given the price of the plugin we are unable to step in and offer hands-on assistance in this sort of situation.

I do hope that’s understandable and, as I really don’t think we can offer up much else here, I’m now going to close this thread.

However, if we can assist with anything else please don’t hesitate to create new threads as needed.

[Author]

Posts