Okay, first of all, you were enforcing a safety rule so you were doing the right thing. But some things were messed up.
- Coach leaves the field then returns with a board member - No, unless the coach leaves to take care of an injured player (or something to that effect) or get equipment - he is not coming back to the field.
- Board member calls you to the backstop - Politely tell him that you are umpiring a game and if he wishes to talk, you will gladly meet him after the game. Return to your position.
- Coach goes to mound and board member joins him - eject both the coach & the board member as your last act because they have now taken over your authority. Hand them the balls, wish them good luck and leave!
You are better off not working for a chicken$hit outfit like this.