Missed base - what to do, what to do?

[Linknblue]

ASA slow pitch, adult men (ha!). Batter hits gapper and is heading for 3b. Off line throw in and ball arrives wide of third toward home and about 6' toward 2b. A tag play ensues and I'm blocked out and can't tell if tag was applied or not. I rule "safe" and immediately go to my partener for help on tag. He walks toward me and we meet on 3b side of pitching area. Before I get my question answered he says, "He missed 2b anyway". I cringe at the loudness of his statement and I say, don't care and ask again if there was tag. He says "no" and I walk away and signal again, safe.

Now the pitcher appeals to me that the runner the missed base (everyone in the world heard my partner). I tell him to go to my partner. My partner stands there dumbfounded looking at me. I point to him and tell him it's your call. He stands there looking at me a little longer and finally makes the call and rules runner out.

Of course all heck breaks loose.

My question, since the umpire tipped the defense should this be allowed? I let it stand and just took the heat. This was the same game that I walked off on in my previous post by the way.