Yes, this is an age old problem, which people can give you suggestions but you have to find *your* answer for.
We used to run around like crazy trying to fit everything in. We finally figured out that things were becoming a hassle and not as much fun.
For the holidays, we try to have have Thanksgiving at our house. My sister does Thanksgiving with her husband's family, so we have my parents over and my husbands family. (I don't have any other relation around us). On Christmas, we have Christmas eve dinner (early) with my family and open presents with them. We then go over to my husband's Aunts house. Christmas day we stay home. Our daughter wasn't real thrilled with having to rush in the morning to leave after opening presents, so we just stay home. If people want to see us, they are welcome in our home, but we don't leave. We then have a second Christmas with my husbands family (parents, brother and sister) at his parents house the day after.
My husbands brother (who has a wife and 2 kids) alternate. They go to Thanksgiving at one side then Christmas Eve at the other each year, then switch the next year. That way they get a full holiday with each and don't have to rush around.
There is no perfect solution.