Hi Bryan,
Unfortunately it is not currently possible to change details like the venue for specific instances of a recurring event, although we are working on changing this for the future. I would recommend casting your vote for this feature request here, which helps us gauge user interest in new features and help guide future development efforts.
For the time being, your best bet would be to duplicate the events as you mentioned before and then change any details as needed. You could also create the events as recurring and then break the events you need to change from the series. Just note that this will make them independent events, meaning that changes made to the series will no longer affect those events.