I'm curious to know what your set position is like. A set position should not encumber you so much that it takes you out of the play. Instead, a good set position will get you READY for the play.
What exactly do you do when you go set? Are your hands on your knees when you go set? Do you find that your set position requires an extra motion or two (such as standing up straight) before you start running?
Yes, yes, I am a slow pitch guy. But we still go set, too, when on the bases and behind the plate. And we also have stealing in SP.
Getting "set" does not mean getting stuck. It means getting set up to get a good start once the ball is in play.