Hi,
Thanks for using the Events Calendar.
You are correct that only Admins can import using the CSV.
We do not have a way to change by a filter you would have to edit the plugin to change that.
You could try using a plugin like Adminimize and User Role Editor to create a limited admin role that can use the csv importer.
That is about all we have to help with this, if you think that is something you would like as a feature I encourage you to visit our User Voice Page and either upvote an existing request or make a new request there.
http://tribe.uservoice.com/
Thanks