The reason that walking away is a bad idea is it make you appear unapproachable. That will only lead to more problems.

Also, it makes it look like you "blinked" like stated earlier. But most important, it also appears that you don't believe your call enough to "discuss" it with the coach.

If you are a good official, and well respected, they are not going to go too ballistic on you, and usually the exchange is going to be short.