Ozzy ...
Ahem ...
It is in the umpire's job description to get a ball to the pitcher as soon as possible. If throwing it himself can accomplish this, and he's willing and capable, he can and should do it if he's so inclined.
And the liability question could possibly be a concern with a Little Leaguer with a lawyer dad, but come on ... a geezer throwing a ball to an accomplished athlete is not dangerous.
If you can throw a ball 60 feet with 99 percent accuracy, and look like a non-girl when you do it, then throw the ball back to save time if you want to. If you throw like a girl, or you make the pitcher reach too much, give it to the catcher. No big deal.