Changed/Removed events in Google Calendar not reflected after import

[randy hurst]

We have a daily import via Event Aggregator to load from a Google calendar at the following address: https://calendar.google.com/calendar/ical/berean-baptist.org_2733lpv2r2j1v6mdamd16ptgco%40group.calendar.google.com/public/basic.ics

Every time that import runs, it says there is 1 new and 64 updated entries. But there are many events that have been changed or removed on the Google calendar for which the old version remains on our website. There are no errors shown in the admin control panel, but in the PHP error log, I see the following line appear each time the import is run:

[26-Jun-2018 19:36:11 America/Denver] PHP Warning: Unknown: Input variables exceeded 1000. To increase the limit change max_input_vars in php.ini. in Unknown on line 0

With my current host, I don’t have access to change the php.ini, and other guidance I’ve seen suggests that 1000 fits with Modern Tribe’s expectations for max_input_vars.

I’m currently running 4.6.19 of Events Calendar and 4.4.28 of Events Calendar Pro. Please let me know what I can do to resolve this.

[randy hurst]

Oh… and – for at least the last several days – every time that line appears in the PHP error log, it can be tied to a calendar import, and there are no other messages in the PHP error log for the last ten days, though this line shows up 15 times. (We have four calendars with nightly scheduled imports.)

[Andras]

Hi Randy,

Thanks for reaching out to us!
Sorry to hear about the issue with the imports. Let me try to help you out with this.

I will need to check with the developers about that error message, please give me some time.

As for the rest:

If an event is deleted from your google calendar, that will not be deleted from your events calendar. There is no secure way of knowing if an event has been deleted or not.

As for changes, can you give me some example events and dates where this is happening?

If I see your import is set up to import events one month into the future. Events before the current date and after 30 days from today will not be touched in this case, unless…

Unless your import is set up to import events starting from a certain date. Than that date is the starting point. (screenshot: https://cloudup.com/ccW41XSAwb1)

Cheers,
Andras

[randy hurst]

If we cannot automatically delete events from our website calendar as they are removed from the Google calendar, then our subscription to Events Calendar Pro has lost all of its value. We will need to remove the plugin and revert to embedding our Google calendar directly so that we can be sure our website calendar is accurate.

Here are two options your developers could use to ensure that events deleted from Google calendar are removed from the website:

1. Each event from the Google calendar includes a UID that uniquely represents the event from Google. Here’s an example: [email protected]. Events Calendar could store that UID with each event within WordPress. After an import is complete, any of the imported category’s existing events whose UID was not found in the most recent import could be removed.

2. An option could be added to the import definition to delete all events from the associated calendar immediately prior to the import. This is less desirable, as any problem with the import would leave the website with an empty calendar. Still, in our case, it would be preferred over leaving obsolete events displayed on our website.

I recognize that either of these approaches would not work for a website that imports multiple external calendars into a single category. But for a website like ours where each external calendar is tied to exactly one category, the ability to keep them perfectly in sync (including deletes) is absolutely essential. Please give us the option to delete events that are not found in the import.

[randy hurst]

Here’s an example of an event that recently changed but has not been updated:

BEGIN:VEVENT
DTSTART;VALUE=DATE:20180618
DTEND;VALUE=DATE:20180701
DTSTAMP:20180628T141355Z
UID:[email protected]
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;CN=Church
 ;X-NUM-GUESTS=0:mailto:berean-baptist.org_2733lpv2r2j1v6mdamd16ptgco@group.
 calendar.google.com
CREATED:20170816T194647Z
DESCRIPTION:
LAST-MODIFIED:20180611T172602Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Teen Mission Trip
TRANSP:TRANSPARENT
END:VEVENT

After I run the import and wait for it to complete (green check mark in the History tab), a search for that Event from the admin Events page claims that the event ends on July 10 (which is incorrect, as the date in the ICS says it ends on July 1).

Actually, when I run a preview on this import, it also claims to believe that the event ends on July 10. So it may not be a failure to update; it may be a failure to properly interpret the ICS during the import.

[Andras]

Hi Randy,

Thanks for your feedback.

We are aware of the UID in the events, but using that (or whatever identifier) for deletion still holds risks.

Think about the following scenario:

You make an import of your events from google calendar for the coming 3 months. All get imported.

Next you run an import for the coming 30 days. Then events past 30 days will not be in the import feed, thus their UIDs will be missing. Should they be deleted or not?

Obviously this is a very simple and edge case, but I believe it shows the challenges with deleting events.

We are investigating different options to be able to achieve this feature, but that is going to take a while.

As for your second option, yes, that is an absolutely viable workaround to the issue. If that is something you would like as a feature I encourage you to visit our User Voice Page and either upvote an existing request or make a new request there. If you decide to do this, please give as detailed a description as you can including ideas how you would imagine handling cases like past events, venues, organizers, etc.

I suppose a rough version of this could be done with a snippet, hooking into the following action:

do_action( 'tribe_aggregator_before_insert_posts', $items, $this->meta );

Hope this helps. Let me know if you have any further questions.

Cheers,
Andras

[randy hurst]

I have looked into the tribe_aggregator_before_insert_posts action, and that appears to be triggered once for each imported event, with $items containing only a single item. In scouring the source code for the scheduled queues in The Events Calendar, I have been unable to find a hook that provides a suitable hook to delete the imported category’s events prior to the import’s execution.

I have added a vote to this User Voice submission.

Until that User Voice issue is addressed or a suitable hook is available for us to delete the events through a custom plugin of our own, we are unable to use the events calendar for our website.

If you can let me know of a trigger I can hook into that executes only before the import and provides the category being imported, we can apply our own workaround. If not, please cancel our support subscription, as we will be uninstalling The Events Calendar from our website.

[Andras]

Hi Randy,

Thanks for your vote on UserVoice! Much appreciated!

During a discussion with a dev we touched on the event deletion topic and I gathered some extra info for you. Food for thought.

“Each event from the Google calendar includes a UID that uniquely represents the event from Google.” — While this is true for single events, that is not the case for recurring events.

“After an import is complete, any of the imported category’s existing events whose UID was not found in the most recent import could be removed.” — I touched on this from a different angle before; What should happen with all past events? Would be deleted?

These are questions / scenarios which make the implementation of full sync difficult.

And now to the question at hand:

I have been unable to find a hook that provides a suitable hook to delete the imported category’s events prior to the import’s execution.

Bad news: currently a hook like that does not exists.

Good news: A fix for this was suggested and if all goes well it will be included in the next maintenance release, which is scheduled for release in about a week. If you ping me after the MR ships I can check for you and let you know the details.

Cheers,
Andras

[randy hurst]

Thanks, András.

I’ve added some details to the User Voice request at https://tribe.uservoice.com/forums/195723-feature-ideas/suggestions/34418881-allow-event-aggregator-to-delete-events-which-have to address the developer’s comments.

[Andras]

Thanks for that Randy, much appreciated!

It looks like the new hook will be released this week in the maintenance release, if all goes according to plan.

You’ll receive an update here when that happens.

Cheers,
Andras

[Victor]

Hi Randy!

Just wanted to share with you that a new release of our plugins is out, including the new tribe_aggregator_record_finalized action to allow developers to act before Event Aggregator imports start.

Find out more about this release → https://theeventscalendar.com/maintenance-release-for-the-week-of-9-july-2018/

You can use the following snippet as a reference to create your own customization:

https://gist.github.com/lucatume/2a56c51e28d7e8d830d277852fcc8601

Best,
Victor

[randy hurst]

The new tribe_aggregator_record_finalized hook seems to do the trick for me.

Here’s what I ended up with, and it seems to work great – even with every variant of recurring events and exceptions that I could throw at it…

<?php
/**
Created by PhpStorm.
Plugin Name: Berean Baptist Church Calendar Import Fixer
Description: Forces removal of events no longer represented in the imported calendar.
Version: 0.0.1
Author: Steve Dwire for Berean Baptist Church
Author URI: https://www.berean-baptist.org/
License: GPLv2
 */

class BBC_EC_ImportFixer {
    const VERSION = '0.0.1';

    public function __construct() {
        add_action( 'init', array( $this, 'init' ) );
    }

    public function init() {
        add_action( 'tribe_aggregator_record_finalized', array( $this, 'handle_before_insert_posts'), 10, 2 );
    }

    public function handle_before_insert_posts($record_id, $record_meta ) {
        $this_category_term_id = Tribe__Utils__Array::get( $record_meta, 'category', false );

        if ( empty( $this_category_term_id ) ) {
            return;
        }

        $relevant_existing_events = tribe_get_events( [
            'fields'     => 'ids',
            'start_date' => date( 'Y-m-d H:i:s', time() ), // starting now or later
            'tax_query'  => [
                [
                    'taxonomy' => Tribe__Events__Main::TAXONOMY,
                    'field'    => 'term_id',
                    'terms'    => [ $this_category_term_id ],
                    'operator' => 'IN'
                ]
            ],
            'meta_key' => '_uid',
            'meta_value' => $record_meta['ids_to_import'],
            'meta_compare' => 'NOT IN'
        ] );

        foreach ( $relevant_existing_events as $event_id ) {
            wp_delete_post( $event_id, true );
        }

    }
}

new BBC_EC_ImportFixer();

[Andras]

That looks nice, great job!

Here’s also a snippet from our developer that might be of some inspiration.

https://gist.github.com/lucatume/2a56c51e28d7e8d830d277852fcc8601

Is there anything else related to this topic or can we close it as resolved?

Andras

[randy hurst]

That snippet already was the inspiration for the plugin code we ended up with. This is enough to close this issue. Thanks for your help.

[Andras]

Oh, awesome!

You’re welcome!

I am going to close this ticket then, but if you need anything else related to this topic or another please create a new ticket and we’ll be happy to help.

Cheers,
Andras

[Author]

Posts