19 hour Aggregator Import
I am testing event calendar aggregator to see if it will work for our application. I set up a one-time import yesterday. After about 5 minutes it was getting no where, so I moved on to another task. I checked back this morning and it’s bee running for 19 hours, getting no where.
I’d love to be able to stop it, but turning it off doesn’t stop it. Do I need to kill a cron or something?
It’s basically a new site, so looking through the wp_post table for the tribe-ea-pending post type was not very difficult. That’s been deleted, and stopped the first import. I set up a new one-time import for just 3 events. The import is not happening and there is no error in the console or the php error log to let me know where the issue might be
Thanks for getting in touch with us!
I’m really sorry to hear about the troubles with Event Aggregator. I’d love to help you get this sorted.
I checked in our system and I see some successful imports already. Also I tested one of the feeds (ncf-edu_) and it imported events fine with my settings.
Are you still having an issue with the imports? If yes, then please send me the following data so I can dig further:
- the url you are importing from and you are having a problem with
- your import settings under Events > Settings > Imports tab (Import limit type and range)
- a screenshot of how your scheduled import is set up
Thanks and cheers,
Maybe that’s a problem. We have not had any successful imports. Your system is not reporting correctly.
The import is in progress, so I cannot view the import settings in the backend to give you a screen shot. It’s also just a one time import. Attached is not the current import.
here is the URL to one of the calendars we use, I’ve tried a couple:
Import settings for google calendar are set to global settings. attached are the globals.
So you guys built an intermediary server as well? I don’t have time to scale it up, but I’ve built something similar
Do the plugin read JSON, or just ICS format?
Thanks for getting back.
Your settings seem to be ok.
I ran a test import with that source you shared and it ran without an issue, the events got imported for me.
I’ll try to pull in some extra EA-muscle to check this.
While I do that…
I noticed you have some MU plugins as well. Do you think you could disable those somehow and run a test import like that? Alternatively set up a test site with only TEC at a host (even on your local machine) and try to test there?
The plugin works with JSON but cannot directly read it from the user interface at the moment.
I tried to go back to a previous version of the site and your licensing server went silly.
Your Event Aggregator license key is invalid. Visit the Events Calendar website to check your licenses.
We also ruled out caching on the host, or better said, we ruled out caching was not the only issue
I also was able to get an import to work on the staging environment. So the MU-Plugins are not the cause. Because there is no caching in admin for WPEngine’s live servers it is not a cashing error. The only real difference between the two places are the SSL cert and the domains. One is news.ncf.edu and the other is ncfnews.staging.wpengine.com.
We use a lets encrypt cert on the live site. If that matters.
I’d love to know what that WP_ERROR is that is being given in Argument 1 on line 300 of Queue_Processor.php. I tried to get it by using error_log( var_export( $record, true ) ); and received this:
Tribe__Events__Aggregator__Record__gCal::__set_state(array(\n ‘origin’ => ‘gcal’,\n ‘id’ => 10,\n ‘post’ => \n WP_Post::__set_state(array(\n ‘ID’ => 10,\n ‘post_author’ => ‘1’,\n ‘post_date’ => ‘2018-07-02 13:50:49’,\n ‘post_date_gmt’ => ‘2018-07-02 17:50:49’,\n ‘post_content’ => ”,\n ‘post_title’ => ‘Record: manual gcal’,\n ‘post_excerpt’ => ”,\n ‘post_status’ => ‘tribe-ea-pending’,\n ‘comment_status’ => ‘closed’,\n ‘ping_status’ => ‘manual’,\n ‘post_password’ => ”,\n ‘post_name’ => ‘record-manual-gcal’,\n ‘to_ping’ => ”,\n ‘pinged’ => ”,\n ‘post_modified’ => ‘2018-07-02 13:51:02’,\n ‘post_modified_gmt’ => ‘2018-07-02 17:51:02’,\n ‘post_content_filtered’ => ”,\n ‘post_parent’ => 0,\n ‘guid’ => ‘https://news.ncf.edu/?post_type=tribe-ea-record&p=10’,\n ‘menu_order’ => 0,\n ‘post_type’ => ‘tribe-ea-record’,\n ‘post_mime_type’ => ‘ea/gcal’,\n ‘comment_count’ => ‘0’,\n ‘filter’ => ‘raw’,\n )),\n ‘meta’ => \n array (\n ‘hash’ => ‘}mn4i T]$^0Ql(wY 7DesfTzE]XMS&A*’,\n ‘preview’ => ‘1’,\n ‘allow_multiple_organizers’ => ‘1’,\n ‘origin’ => ‘gcal’,\n ‘type’ => ‘manual’,\n ‘keywords’ => ”,\n ‘location’ => ”,\n ‘start’ => ‘2018-09-01’,\n ‘source’ => ‘https://calendar.google.com/calendar/ical/ncf.edu_3el9b9gm7ogtv8oi2u7kcop1v8%40group.calendar.google.com/public/basic.ics’,\n ‘next_batch_hash’ => ‘a2b5cc27ae5e38d9a54a1be42af5f062’,\n ‘import_id’ => ‘d0a65355fe77b5e95e237e85dfb4665f285f2d868770cd4c6ccf21e0ec78c672’,\n ‘should_queue_import’ => ”,\n ‘source_name’ => ‘OCM Events’,\n ‘post_status’ => ‘publish’,\n ‘ids_to_import’ => \n array (\n 0 => ’00E86F14-7319-4DF6-A921-7A5892F46C580′,\n ),\n ‘finalized’ => ‘1’,\n ‘interactive’ => ‘1’,\n ‘queue_id’ => ‘tribe_queue_ea_import_events_batch_d461d12a9ad84054bf1f326983489’,\n ‘queue’ => ‘1’,\n ‘in_progress’ => ‘1’,\n ‘activity’ => \n Tribe__Events__Aggregator__Record__Activity::__set_state(array(\n ‘items’ => \n array (\n ‘tribe_events’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘tribe_organizer’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘tribe_venue’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘tribe_events_cat’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘post_tag’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘attachment’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ),\n ‘last_status’ => NULL,\n ‘map’ => \n array (\n ‘tribe_events’ => ‘tribe_events’,\n ‘event’ => ‘tribe_events’,\n ‘events’ => ‘tribe_events’,\n ‘tribe_organizer’ => ‘tribe_organizer’,\n ‘organizer’ => ‘tribe_organizer’,\n ‘organizers’ => ‘tribe_organizer’,\n ‘tribe_venue’ => ‘tribe_venue’,\n ‘venue’ => ‘tribe_venue’,\n ‘venues’ => ‘tribe_venue’,\n ‘tribe_events_cat’ => ‘tribe_events_cat’,\n ‘category’ => ‘tribe_events_cat’,\n ‘categories’ => ‘tribe_events_cat’,\n ‘cat’ => ‘tribe_events_cat’,\n ‘cats’ => ‘tribe_events_cat’,\n ‘post_tag’ => ‘post_tag’,\n ‘tag’ => ‘post_tag’,\n ‘tags’ => ‘post_tag’,\n ‘attachment’ => ‘attachment’,\n ‘attachments’ => ‘attachment’,\n ‘image’ => ‘attachment’,\n ‘images’ => ‘attachment’,\n ),\n ‘total’ => 0,\n )),\n ),\n ‘type’ => ‘manual’,\n ‘frequency’ => NULL,\n ‘is_schedule’ => false,\n ‘is_manual’ => true,\n ‘last_wpdb_error’ => ”,\n ‘origin_import_policies’ => \n array (\n ‘url’ => \n array (\n 0 => ‘show_map_link’,\n ),\n ),\n ‘last_child’ => NULL,\n ‘temp_event_count’ => 0,\n ‘image_uploader’ => \n Tribe__Image__Uploader::__set_state(array(\n ‘featured_image’ => NULL,\n )),\n)), referer: https://news.ncf.edu/wp-admin/edit.php?post_type=tribe_events&page=aggregator
Thanks for your help in investigating the issue. The information you shared is really valuable.
In the meantime we also did some testing with WP engine and ended up with the same results.
It looks like this is a glitch when using WPEngine. I have logged a ticket for this and one of our wizards will investigate this further to fix this.
I believe the SSL certificate shouldn’t be an issue as long as the certificate is valid.
I am going to set the status of this ticket to “pending fix” and we will update it once the fix is released. If you find out more about the issue on your side, please post it here.
For any new questions or issues please create a new ticket and we’ll help you out there.
Thanks and cheers,
Just wanted to share with you that a new release of out plugins is out, including a fix for this issue.
Find out more about this release → https://theeventscalendar.com/maintenance-release-for-the-week-29-july-2018/
We apologize for the delay and appreciate your patience while we worked on this.
As always, we suggest to setup a dev/staging site so you can make all plugin updates without disrupting the live site. Please update the plugins and let us know if the fix works for your site.
Thanks for reporting back! This is indeed very curious.
Can you please confirm to me, if on the staging site you do or do not have the three MU plugins you have on production? So far I see that is one major difference between your sites and can be the source of this.
Meanwhile I will try to pull in some extra muscle for this as well, to get a different perspective. I ask for your kind patience.
In Production, Staging, and Development I have
and two folders
This morning I did the following
- created a production back up (it’s the only way to quickly stop the run-away process)
- made sure object caching was disabled and cleared the cache in the WPE dashboard
- opened an incognito chrome window so as to avoid any client-side cache
- Use transient manager plugin to delete all transients
- Turned off all plugins except the events calendar, switched the theme to twentyseventeen
- purged the cache in the wordpress admin
- set an import for 8 events from a google calendar ICS
- hit import, and no luck, nothing ever imports and the progress bar never progresses. I let it spin for 60 seconds just to make sure
- restored from the back up to stop the import.
I’m really sorry that this issue is so persistent. I have escalated the problem and one of our developers is going to take a deeper look into this.
I would be interested in one more thing, if you have a bit of time and the will to try.
If you set up a brand new site from scratch (it could event be under a subdomain), only with TEC, are you having the same problem that the events do not import? You can use the same EA license key, it can be used on 10 different domains.
To give you a bit of background information for my request of a new site test:
“I think what happened is that the user deleted the records without deleting the queue information as well. The queue processor got stuck trying to start a queue related to a record that no more exists. We need to handle this case. Why it happened in the first place, seeing the date of the first post, was a queue bug we fixed.”
And a note:
“The line the user reports too shows the user is at least one TEC version behind.”
Also, the data that is on the site has been pushed from development or from a previous backup. this would be impossible: “deleted the records without deleting the queue information as well.” I immediately went back to an older version of the data every time so as to not have bad data in the database