Home › Forums › Calendar Products › Events Calendar PRO › Severe query performance issues with new meta chunker
- This topic has 10 replies, 7 voices, and was last updated 6 years, 9 months ago by Support Droid.
-
AuthorPosts
-
May 24, 2017 at 6:22 pm #1288679KevinParticipant
In 4.5.2, you introduced a new class called Tribe_Meta_Chunker. We are having serious query performance issues with this update on a high traffic site. We’ve had to revert back to 4.5.1 after only 5 minutes live on our site. Here are my questions.
1) Where is the supposed to be used, and why is it priming a cache on every page load?
2) Where is this mentioned in the changelog? The changelog seemed to be innocuous, but this is a significant change that isn’t mentioned anywhere. I do rely on reviewing changelogs before I roll code on to our production site and need to be able to rely on this sort of stuff being in there.The query that is running on each page load is this:
SELECT post_id, meta_key
FROM wp_postmeta
WHERE meta_key LIKE ‘_tribe_chunker_%’
AND meta_key NOT LIKE ‘_tribe_chunker_%_chunk’Our database returns 0 records for this query, yet it takes 0.4 – 1.0 sec to return an answer for a poorly-written MySQL query on a larger database like ours. (we have about 1.5M rows in our postmeta table) Our database is a dedicated box and is not typically under any measurable load until I activate this update.
May 26, 2017 at 8:57 am #1289447ShelbyParticipantHi there Kevin,
This is actually a documented bug that our developers are prioritizing right now. The goal is to resolve this in the next Maintenance Release. 🙂
I will add your thread to the list in the ticket, and associate your thread with the ticket so that you will get an update when this has been resolved!
Please let me know if you have any follow up questions or need any clarification.
Best,
Shelby 🙂
June 1, 2017 at 9:04 pm #1292181GeorgeParticipantHey Kevin,
We shipped a Maintenance Release today that includes some improvements for the queries you mentioned here.
Learn more here → https://theeventscalendar.com/maintenance-release-for-the-week-of-29th-may-2017/
Cheers,
GeorgeJune 2, 2017 at 12:19 am #1292236KevinParticipantI’ve deployed the latest version. Indeed, the improperly written query seems to be gone, but I’ve uncovered a new issue.
Can someone explain to me why this function is hooked to to normal posts? I’m seeing it firing on regular posts that are using get_post_meta, not just the tribe_events CPT. This should only fire on posts that need it.
I’m also not sure this is object cache friendly. Most posts are not going to have the “chunked post meta” set, but this function has to run its query to figure that out. You have added a new query to every page load for us that didn’t normally have to run before because the calls to get_post_meta were cached.
It appears that this is story post-specific settings in the options table as well? Any explanation here? The proper way to store a post-specific value is to store it in postmeta, not options
I’m going to reiterate what I said before about the changelog. Why is this not listed anywhere in the change log?
June 5, 2017 at 5:40 pm #1293488BrookParticipantHowdy Kevin!
I’d love to jump in and explain some things. Performance is a huge passion of mine.
I’m going to reiterate what I said before about the changelog. Why is this not listed anywhere in the change log?
This is some tremendously valuable feedback and I really appreciate your sharing it. I too rely on changelogs in the software I deploy.
This absolutely should have been in our changelogs. That it wasn’t is a slipup on our end. I have made a note to talk about this a bit more on our next dev scrum. We are in the throes of changing when our changelogs get written, and this is one is a good example of where we can improve our new process of a bit to make it as effective as the old one.
Can someone explain to me why this function is hooked to to normal posts? I’m seeing it firing on regular posts that are using get_post_meta, not just the tribe_events CPT. This should only fire on posts that need it. I’m also not sure this is object cache friendly. Most posts are not going to have the “chunked post meta” set, but this function has to run its query to figure that out. You have added a new query to every page load for us that didn’t normally have to run before because the calls to get_post_meta were cached.
Unfortunately there is no way to predict which queries a page will make before it makes them. Any page could have a widget, shortcode, or get_posts() request that would query a chunkable post. So it becomes necessary to do two things:
- Get a list of chunkable posts. It makes sense to store that list in the wp_options table, get it on autoload, and then cache it in memory for the duration of the page.
- Hook into the post meta filters. In the hooked function, check if the current post is the cached list. If not, bail thus incurring the minimal performance impact.
This would be a typical WordPress way of doing things, and near as I can conjure would be the most performant way to go about it.
It appears that this is story post-specific settings in the options table as well? Any explanation here? The proper way to store a post-specific value is to store it in postmeta, not options
From what I am seeing in the code, the chunked meta data itself is being stored in the postmeta table. But the settings, such as which post types are chunkable, is stored in wp_options. Is this behaving differently on your system? Or are only IDs and post types being stored in the options table.
Please do let me know if you have any more questions.
Cheers!
– Brook
June 9, 2017 at 6:37 pm #1296033KevinParticipantI still don’t think the implementation here is correct for a couple of reasons.
We have a plugin that runs get_post_metadata on regular posts. Watching the call stack in Query Monitor, this fires off the Tribe_Meta_Chunker on the get_post_metadata filter. I see it running filter_get_metadata, get_all_meta_for and get_all_meta, yet these posts are not “chunked” as this function describes. (i.e. There are no long meta values)
I assume this function was added to support long metadata generated by TEC. If so, why is it being applied to posts and not tribe_event post types only?
This function is running on posts regardless of whether there is chunked data or not, so it is not “bailing” as far as I can tell. On our system, it is making it all the way through to the custom DB query on get_all_meta.
Since you are circumventing the normal DB query for get_post_metadata completely, it appears that this is also bypassing all of the object cache logic present in WP core for the normal get_post_metadata function. This is why I noticed it because it is hitting on every pageload where previously this was easily cached.
I am the dev on a very high traffic site where query performance and caching is critical. On a recently viewed post, our page generation can often be accomplished 100% from the object cache without even hitting MySQL. Even if your function is relatively quick, you introduced a needless query where we don’t need it with no object caching.
I assume there is some reason that TEC needs extra long metadata strings, which is why this was introduced. If that is the case, it should apply only where it is absolutely needed and bail to the normal WP functions otherwise. Why wouldn’t TEC store chunked data on tribe_event and not posts?
And I’m not totally comfortable with the way this completely rewrites a WP core function. It is one thing to filter or alter the value in a WP core function. It seems something else to completely replace the DB queries and post cache like was done here that applies to all regular posts.
What consequence will there be if I turn off the chunker via the post type filter available in the plugin?
June 13, 2017 at 9:30 pm #1297609ShelbyParticipantHi there Kevin,
Thank you for expressing all of your concerns here. I’ll definitely be sure to pass them along, but this has gone a bit outside of the scope of our support on these forums.
Please be sure to reach out in the future if you have questions about our plugin features. Also, if you’d like to seek a refund for your plugin purchases, you can do so using this form here.
In the meantime, I will be closing this one down, as Brook has helped you as much as we are able to here. Thanks for understanding! 🙂
Best,
Shelby 🙂
June 14, 2017 at 12:19 pm #1298051Zach TirrellKeymasterKevin,
Upon closer inspection, I believe you could go ahead use the filter to remove “post” from the Meta_Chunker’s supported post types.
function tribe_remove_post_from_chunker( $post_types ) { return array_diff( $post_types, array( 'post' ) ); } add_filter( 'tribe_meta_chunker_post_types', 'tribe_remove_post_from_chunker' );
We are testing this now as a core change, but it won’t make it into a scheduled release until next week at the earliest.
June 26, 2017 at 7:11 am #1306216NicoMemberHi there Kevin,
As Zach anticipated in his last reply, we included the suggested fix in our release shipped last week (release notes).
Please let us know if you still experience issue after updating the plugins,
Best,
NicoJuly 18, 2017 at 9:36 am #1322714Support DroidKeymasterHey there! This thread has been pretty quiet for the last three weeks, so we’re going to go ahead and close it to avoid confusion with other topics. If you’re still looking for help with this, please do open a new thread, reference this one and we’d be more than happy to continue the conversation over there.
Thanks so much!
The Events Calendar Support Team -
AuthorPosts
- The topic ‘Severe query performance issues with new meta chunker’ is closed to new replies.