Why is leisure so stressful?

So the kids have become much easier, and I have realized that I must not cram an activity and/or childhood experience into each moment of free time, so we keep ending up with days for which nothing has been planned. Some people revel in free time, for me, it is near torture. Martin worked a conference booth this morning, so I baked pumpkin muffins, played Alphabet Go Fish with the kids, went for a bike ride, whipped up some nearly organic minestrone soup for lunch and started making Christmas presents. It sounds so Martha Stewart-ish, but the day feels like it was 20 hours long not having another adult with whom to talk.
Yesterday was my kind of day- much more social. We celebrated a friend's birthday with a picnic at the V. Sattui winery in Napa. It was a beautiful Fall day, and she brought along a babysitter for the group. We tasted wines, and sat and enjoyed the kids playing, and ate pate and cheese and beautiful sandwiches. I think that must be what it is like to have a nanny. You get to experience the joy of watching your children and having them near, but can also actually have a conversation and enjoy yourself with friends. I think I'd like to give that a try- now if I can only win the lottery. I know money doesn't make you happy, but I'm pretty happy already, so I think it would be a nice thing to try for a while.