Help Desk

19 hour Aggregator Import

  • Posts: 47 Topics: 7
    | Permalink

    I am testing event calendar aggregator to see if it will work for our application. I set up a one-time import yesterday. After about 5 minutes it was getting no where, so I moved on to another task. I checked back this morning and it’s bee running for 19 hours, getting no where.

    I’d love to be able to stop it, but turning it off doesn’t stop it. Do I need to kill a cron or something?

    Posts: 47 Topics: 7
    | Permalink

    It’s basically a new site, so looking through the wp_post table for the tribe-ea-pending post type was not very difficult. That’s been deleted, and stopped the first import. I set up a new one-time import for just 3 events. The import is not happening and there is no error in the console or the php error log to let me know where the issue might be

    Posts: 47 Topics: 7
    | Permalink

    Same thing on the latest

    Posts: 6977 Topics: 2
    | Permalink

    Hi Adam,

    Thanks for getting in touch with us!

    I’m really sorry to hear about the troubles with Event Aggregator. I’d love to help you get this sorted.

    I checked in our system and I see some successful imports already. Also I tested one of the feeds (ncf-edu_) and it imported events fine with my settings.

    Are you still having an issue with the imports? If yes, then please send me the following data so I can dig further:

    • the url you are importing from and you are having a problem with
    • your import settings under Events > Settings > Imports tab (Import limit type and range)
    • a screenshot of how your scheduled import is set up

    Thanks and cheers,
    Andras

    Posts: 47 Topics: 7
    | Permalink

    Maybe that’s a problem. We have not had any successful imports. Your system is not reporting correctly.

    The import is in progress, so I cannot view the import settings in the backend to give you a screen shot. It’s also just a one time import. Attached is not the current import.

    here is the URL to one of the calendars we use, I’ve tried a couple:
    https://calendar.google.com/calendar/ical/ncf.edu_3el9b9gm7ogtv8oi2u7kcop1v8%40group.calendar.google.com/public/basic.ics

    Import settings for google calendar are set to global settings. attached are the globals.

    So you guys built an intermediary server as well? I don’t have time to scale it up, but I’ve built something similar
    https://valiant-circuit-729.appspot.com/?cal=ncf
    Do the plugin read JSON, or just ICS format?

    Posts: 47 Topics: 7
    | Permalink

    Also, I can take the nuclear option with this site if need be. There are only two posts that are the content creator just trying out the custom fields. please let me know if you recommend that

    Posts: 6977 Topics: 2
    | Permalink

    Hi Adam,

    Thanks for getting back.

    Your settings seem to be ok.

    I ran a test import with that source you shared and it ran without an issue, the events got imported for me.

    I’ll try to pull in some extra EA-muscle to check this.

    While I do that…

    I noticed you have some MU plugins as well. Do you think you could disable those somehow and run a test import like that? Alternatively set up a test site with only TEC at a host (even on your local machine) and try to test there?

    The plugin works with JSON but cannot directly read it from the user interface at the moment.

    Cheers,
    Andras

    Posts: 47 Topics: 7
    | Permalink

    unfortunately, the MU-Plugins are required by our host (WPEngine). I migrated down to dev and tested and it worked. So I guess it’s something with the Mu-plugins. Have you had any success on WPEngine? I’m gonna go see what their support says

    Posts: 47 Topics: 7
    | Permalink

    I tried to go back to a previous version of the site and your licensing server went silly.

    Your Event Aggregator license key is invalid. Visit the Events Calendar website to check your licenses.

    We also ruled out caching on the host, or better said, we ruled out caching was not the only issue

    Posts: 47 Topics: 7
    | Permalink

    WPEngine will not let me remove the MU-Plugins. They are wpengine-common and force-strong-passwords

    Posts: 47 Topics: 7
    | Permalink

    OK, so I got on chat with Advanced Support at WPEngine. We saw a couple things, but the error is not being reported. We say a continuous post to admin-ajax. We also saw the following error message pop up in the error console.

    Posts: 47 Topics: 7
    | Permalink

    I also was able to get an import to work on the staging environment. So the MU-Plugins are not the cause. Because there is no caching in admin for WPEngine’s live servers it is not a cashing error. The only real difference between the two places are the SSL cert and the domains. One is news.ncf.edu and the other is ncfnews.staging.wpengine.com.

    We use a lets encrypt cert on the live site. If that matters.

    I’d love to know what that WP_ERROR is that is being given in Argument 1 on line 300 of Queue_Processor.php. I tried to get it by using error_log( var_export( $record, true ) ); and received this:

    Tribe__Events__Aggregator__Record__gCal::__set_state(array(\n ‘origin’ => ‘gcal’,\n ‘id’ => 10,\n ‘post’ => \n WP_Post::__set_state(array(\n ‘ID’ => 10,\n ‘post_author’ => ‘1’,\n ‘post_date’ => ‘2018-07-02 13:50:49’,\n ‘post_date_gmt’ => ‘2018-07-02 17:50:49’,\n ‘post_content’ => ”,\n ‘post_title’ => ‘Record: manual gcal’,\n ‘post_excerpt’ => ”,\n ‘post_status’ => ‘tribe-ea-pending’,\n ‘comment_status’ => ‘closed’,\n ‘ping_status’ => ‘manual’,\n ‘post_password’ => ”,\n ‘post_name’ => ‘record-manual-gcal’,\n ‘to_ping’ => ”,\n ‘pinged’ => ”,\n ‘post_modified’ => ‘2018-07-02 13:51:02’,\n ‘post_modified_gmt’ => ‘2018-07-02 17:51:02’,\n ‘post_content_filtered’ => ”,\n ‘post_parent’ => 0,\n ‘guid’ => ‘https://news.ncf.edu/?post_type=tribe-ea-record&p=10’,\n ‘menu_order’ => 0,\n ‘post_type’ => ‘tribe-ea-record’,\n ‘post_mime_type’ => ‘ea/gcal’,\n ‘comment_count’ => ‘0’,\n ‘filter’ => ‘raw’,\n )),\n ‘meta’ => \n array (\n ‘hash’ => ‘}mn4i T]$^0Ql(wY 7DesfTzE]XMS&A*’,\n ‘preview’ => ‘1’,\n ‘allow_multiple_organizers’ => ‘1’,\n ‘origin’ => ‘gcal’,\n ‘type’ => ‘manual’,\n ‘keywords’ => ”,\n ‘location’ => ”,\n ‘start’ => ‘2018-09-01’,\n ‘source’ => ‘https://calendar.google.com/calendar/ical/ncf.edu_3el9b9gm7ogtv8oi2u7kcop1v8%40group.calendar.google.com/public/basic.ics’,\n ‘next_batch_hash’ => ‘a2b5cc27ae5e38d9a54a1be42af5f062’,\n ‘import_id’ => ‘d0a65355fe77b5e95e237e85dfb4665f285f2d868770cd4c6ccf21e0ec78c672’,\n ‘should_queue_import’ => ”,\n ‘source_name’ => ‘OCM Events’,\n ‘post_status’ => ‘publish’,\n ‘ids_to_import’ => \n array (\n 0 => ’00E86F14-7319-4DF6-A921-7A5892F46C580′,\n ),\n ‘finalized’ => ‘1’,\n ‘interactive’ => ‘1’,\n ‘queue_id’ => ‘tribe_queue_ea_import_events_batch_d461d12a9ad84054bf1f326983489’,\n ‘queue’ => ‘1’,\n ‘in_progress’ => ‘1’,\n ‘activity’ => \n Tribe__Events__Aggregator__Record__Activity::__set_state(array(\n ‘items’ => \n array (\n ‘tribe_events’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘tribe_organizer’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘tribe_venue’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘tribe_events_cat’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘post_tag’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ‘attachment’ => \n stdClass::__set_state(array(\n ‘created’ => \n array (\n ),\n ‘updated’ => \n array (\n ),\n ‘skipped’ => \n array (\n ),\n )),\n ),\n ‘last_status’ => NULL,\n ‘map’ => \n array (\n ‘tribe_events’ => ‘tribe_events’,\n ‘event’ => ‘tribe_events’,\n ‘events’ => ‘tribe_events’,\n ‘tribe_organizer’ => ‘tribe_organizer’,\n ‘organizer’ => ‘tribe_organizer’,\n ‘organizers’ => ‘tribe_organizer’,\n ‘tribe_venue’ => ‘tribe_venue’,\n ‘venue’ => ‘tribe_venue’,\n ‘venues’ => ‘tribe_venue’,\n ‘tribe_events_cat’ => ‘tribe_events_cat’,\n ‘category’ => ‘tribe_events_cat’,\n ‘categories’ => ‘tribe_events_cat’,\n ‘cat’ => ‘tribe_events_cat’,\n ‘cats’ => ‘tribe_events_cat’,\n ‘post_tag’ => ‘post_tag’,\n ‘tag’ => ‘post_tag’,\n ‘tags’ => ‘post_tag’,\n ‘attachment’ => ‘attachment’,\n ‘attachments’ => ‘attachment’,\n ‘image’ => ‘attachment’,\n ‘images’ => ‘attachment’,\n ),\n ‘total’ => 0,\n )),\n ),\n ‘type’ => ‘manual’,\n ‘frequency’ => NULL,\n ‘is_schedule’ => false,\n ‘is_manual’ => true,\n ‘last_wpdb_error’ => ”,\n ‘origin_import_policies’ => \n array (\n ‘url’ => \n array (\n 0 => ‘show_map_link’,\n ),\n ),\n ‘last_child’ => NULL,\n ‘temp_event_count’ => 0,\n ‘image_uploader’ => \n Tribe__Image__Uploader::__set_state(array(\n ‘featured_image’ => NULL,\n )),\n)), referer: https://news.ncf.edu/wp-admin/edit.php?post_type=tribe_events&page=aggregator

    Posts: 6977 Topics: 2
    | Permalink

    Hi Adam,

    Thanks for your help in investigating the issue. The information you shared is really valuable.

    In the meantime we also did some testing with WP engine and ended up with the same results.

    It looks like this is a glitch when using WPEngine. I have logged a ticket for this and one of our wizards will investigate this further to fix this.

    I believe the SSL certificate shouldn’t be an issue as long as the certificate is valid.

    I am going to set the status of this ticket to “pending fix” and we will update it once the fix is released. If you find out more about the issue on your side, please post it here.

    For any new questions or issues please create a new ticket and we’ll help you out there.

    Thanks and cheers,
    Andras

    Posts: 47 Topics: 7
    | Permalink

    András,

    Any movement on this, we are coming up on a month and would like to apply for a refund if fixing it is not a priority.

    Posts: 6977 Topics: 2
    | Permalink
    This reply has been marked as private.
    Posts: 47 Topics: 7
    | Permalink

    excellent, thank you

    Posts: 6977 Topics: 2
    | Permalink

    Sure thing! And thanks for hanging in there.

    Posts: 6228 Topics: 17
    | Permalink

    Hi Adam!

    Just wanted to share with you that a new release of out plugins is out, including a fix for this issue.

    Find out more about this release → https://theeventscalendar.com/maintenance-release-for-the-week-29-july-2018/

    We apologize for the delay and appreciate your patience while we worked on this.

    As always, we suggest to setup a dev/staging site so you can make all plugin updates without disrupting the live site. Please update the plugins and let us know if the fix works for your site.

    Best,
    Victor

    Posts: 47 Topics: 7
    | Permalink

    I did try 4.6.21 as soon as it was released with no luck. same behavior

    Posts: 47 Topics: 7
    | Permalink

    I just verified that the plug in is still working on WPEngine’s staging environment and my local environment, but not WPEngine’s Production environment. Is there anything that needs to be done to ease up on caching?

    Posts: 47 Topics: 7
    | Permalink

    Also, the work around for this at the moment would be to import in staging and push the site down to live

    Posts: 6977 Topics: 2
    | Permalink

    Hi Adam,

    Thanks for reporting back! This is indeed very curious.

    Can you please confirm to me, if on the staging site you do or do not have the three MU plugins you have on production? So far I see that is one major difference between your sites and can be the source of this.

    Meanwhile I will try to pull in some extra muscle for this as well, to get a different perspective. I ask for your kind patience.

    Cheers,
    Andras

    Posts: 6977 Topics: 2
    | Permalink

    If you didn’t do so yet, please clear your transients and cache and try running the import again.

    A.

    Posts: 47 Topics: 7
    | Permalink

    In Production, Staging, and Development I have
    wp-migrate-db-pro-compatibiulity
    stop-long-comments
    sit-force-strong-passwords
    mu-plugin

    and two folders
    wpengine-common
    force-strong-passwords

    This morning I did the following

        created a production back up (it’s the only way to quickly stop the run-away process)
        made sure object caching was disabled and cleared the cache in the WPE dashboard
        opened an incognito chrome window so as to avoid any client-side cache
        Use transient manager plugin to delete all transients
        Turned off all plugins except the events calendar, switched the theme to twentyseventeen
        purged the cache in the wordpress admin
        set an import for 8 events from a google calendar ICS
        hit import, and no luck, nothing ever imports and the progress bar never progresses. I let it spin for 60 seconds just to make sure
        restored from the back up to stop the import.
    Posts: 6977 Topics: 2
    | Permalink

    Hi Adam,

    I’m really sorry that this issue is so persistent. I have escalated the problem and one of our developers is going to take a deeper look into this.

    I would be interested in one more thing, if you have a bit of time and the will to try.

    If you set up a brand new site from scratch (it could event be under a subdomain), only with TEC, are you having the same problem that the events do not import? You can use the same EA license key, it can be used on 10 different domains.

    Andras

     

    Posts: 6977 Topics: 2
    | Permalink

    To give you a bit of background information for my request of a new site test:

    “I think what happened is that the user deleted the records without deleting the queue information as well. The queue processor got stuck trying to start a queue related to a record that no more exists. We need to handle this case. Why it happened in the first place, seeing the date of the first post, was a queue bug we fixed.”

    And a note:

    “The line the user reports too shows the user is at least one TEC version behind.”

    Thanks,

    Andras

    Posts: 47 Topics: 7
    | Permalink

    We need to launch this site in about 2 weeks, so rebuilding from the ground up is not an option at the moment.

    I am currently on event calendar 4.6.21

    Can you just tell me where the queue information is?

    Posts: 47 Topics: 7
    | Permalink

    I tried to install on another domain and it would not take the keys

    Posts: 47 Topics: 7
    | Permalink

    here is a screen cap of that

    Posts: 47 Topics: 7
    | Permalink

    Also, the data that is on the site has been pushed from development or from a previous backup. this would be impossible: “deleted the records without deleting the queue information as well.” I immediately went back to an older version of the data every time so as to not have bad data in the database

The topic ‘19 hour Aggregator Import’ is closed to new replies.