Howdy Stephan,
Good question. I often hear the referred to as a “paginator”. I have never seen anyone build one for The Events Calendar. However, it’s definitely possible to do using theme overrides and by running a query to determine how many future/past events there are in total. Our plugin does not run that query by default, as this affects performance and usually users never actually use the numbered links.
But, if the small sacrifice in performance is worth it to you, you could definitely run a query to see how many events there are, and build a paginator from it. There are many tutorials outy there for building a paginator or pagination. This even shares some code that would be very applicable to a WordPress.
If you are looking for a prebuilt one though, then as I said I am not aware of anyone who has already built one.
Does that info help answer your question? Cheers!
– Brook