See the trick to this is that you go the first time they ask and show up absolutely hungover. Not like “I’m kinda tired” hungover but like “walking dead” hungover. Rock the sunglasses and hoodie inside and require everyone to be very quiet and keep a trash can close by just in case you gotta puke.

Make it so bad that you just don’t get asked to do those things again. You gotta set the bar right up front and ensure when they go “who should we ask to be here?” that the first person they think of is you. Not because they want you there but they know that you are immediately disqualified from the discussion

Now if they need you to go golfing or something like that which you are ok with doing you won't get invited there either. But you can always go golf on your own time.