This is clearly a bug – if you are looking at the calendar from a category filter it should remain in that category. We should not have to vote on a bug fic. I posted the same issue this morning
https://theeventscalendar.com/support/forums/topic/event-categories-day-view/