Yeah, I agree. I like to go fast and I would be really annoyed with someone who wanted to just walk or trot the whole time.
I'm not too picky. I'll go with anyone! I'm just happy to spend time with horses and horse people and that means I'm willing to go fast or slow.
I guess I'm lucky in that I've got one horse who LOVES to go fast, and one who wants to take it nice and easy. I'm good riding for how ever long someone else wants to ride. I think some flexibility is good as well when trying to find a trail riding buddy! And, having some other shared interests never hurts for when you're out on the trail to talk about too!