RSS Feed: error on line 260 at column 31: Input is not proper UTF-8

[Jonathan Goldford]

We are having the same issue that was listed on https://theeventscalendar.com/support/forums/topic/rss-feed-error-on-line-97-at-column-25-input-is-not-proper-utf-8-indicate-enc/. We’re having to remove those invalid characters manually once we notice the RSS feed breaks. We use Community Events with a basic textarea for the description, but we’re still seeing this issue.

Any idea what we can do to fix this? Any way to strip out those characters automatically?

Thanks for the help.

Jonathan

[Geoff]

Hey there, Jonathan!

Are the invalid characters in the event post content itself or some other field? I imagine the cleanest way to do this would be manually, but you could also try doing a search and replace in the database for each specific character. There’s a plugin that might help make that a little more palatable.

Sorry I don’t have more of a concrete solution for you here, but does this at least help? Please let me know. 🙂

Cheers!
Geoff

[Jonathan Goldford]

Thanks Geoff. The invalid characters are in the post content itself which is pulled from the Event Description field within Community Events. Unfortunately, we can’t do them manually because they aren’t actually visible within the WordPress editor, and there are multiple events a day, so the amount of time would be fairly significant, even if we used some sort of search and replace plugin.

Any other suggestions?

[Geoff]

Ah, gotcha. Yeah, I think your best bet is to search and replace from the database itself.

There is a plugin that is supposed to clean out non-UTF-8 characters from posts, but it hasn’t been updated in a while and I’m not sure if it supports custom post types. Still, might be worth checking out.

Cheers!
Geoff

[Jonathan Goldford]

Thanks Geoff, but that would still be a ton of regular manual work. Wouldn’t it make some sense to look at removing those characters on output of the rss feed via code? That way there wouldn’t be any manual work involved at all.

Jonathan

[Geoff]

Hey Jonathan!

I see how that plugin is still cumbersome. How about this one, which does the conversion in the database in one fell swoop?

You could certainly write a function that checks the output of RSS. However, that’s not something we have or support offhand, so using existing solutions might be more efficient here.

Cheers!
Geoff

[Jonathan Goldford]

Thanks Geoff. It looks like the database table are already storing in UTF-8, so I don’t think that plugin is what we need.

Hopefully, this will help someone else, but we decided to go a fix that removes invalid characters on output for both the content and the description in the feed. Here is the code we used:

/**
 * Adjust the RSS feed description and content to remove invalid UTF-8 characters.
 *
 * Invalid characters were causing issues with the RSS feed pulling into the calendar because people were copying 
 * into the Community Events system with invalid UTF-8 characters. This causes errors within the RSS feed that kept it from working correctly when trying to pull events to our other sites.
 */
add_filter( 'the_content_feed', 'wi_remove_invalid_characters', 10, 2 );
add_filter( 'the_excerpt_rss', 'wi_remove_invalid_characters' );
function wi_remove_invalid_characters( $content, $feed_type = '' ){

	if( $feed_type == 'rss2' ){
		return mb_convert_encoding( $content, 'UTF-8', 'UTF-8'); //Removes invalid characters
	}

	return $content;
}

[Jonathan Goldford]

Thanks Geoff. It looks like the database table are already storing in UTF-8, so I don’t think that plugin is what we need.

Hopefully this will help someone else, but we decided to go a fix that removes invalid characters on output for both the content and the description in the feed. Here is the code we used:

/**
 * Adjust the RSS feed description and content to remove invalid UTF-8 characters.
 *
 * Invalid characters were causing issues with the RSS feed pulling into the calendar because people were copying 
 * into the Community Events system with invalid UTF-8 characters. This causes errors within the RSS feed that kept it from working correctly when trying to pull events to our other sites.
 */
add_filter( 'the_content_feed', 'wi_remove_invalid_characters', 10, 2 );
add_filter( 'the_excerpt_rss', 'wi_remove_invalid_characters' );
function wi_remove_invalid_characters( $content, $feed_type = '' ){

	if( $feed_type == 'rss2' ){
		return mb_convert_encoding( $content, 'UTF-8', 'UTF-8'); //Removes invalid characters
	}

	return $content;
}

Thanks again for the help.

Jonathan

[Geoff]

Rock on, nice work Jonathan! Thanks for sharing the solution too–I’m sure others in a similar situation will find that handy. 🙂

I’ll go ahead and close this thread but please feel free to start a new one if any other questions pop up and we’d be happy to help.

Cheers!
Geoff

[Author]

Posts