Appeal goes wrong

[Coach007]

Runner on 2nd, batter hits long fly ball to centre right caught by the right fielder - batter out. Runner at 2nd clearly leaves before the ball is first touched and moves on to 3rd. Fielder throws ball back to 2nd. Second baseman stands on 2nd with the ball and looks at base umpire who after several seconds gives the safe sign.
An appeal was made and lost. Could I have appealed an appeal given the FACT that the runner absolutely did not tag-up. The wrong call was made but never got corrected. What went wrong?
After the game I asked the base umpire why he didn't call the runner out since he did not tag-up. Said he didn't see the play(runner tagging or not tagging) therefor was unable to make an out call. Ask as to why he did not see it indicated it was not his responsiblity for that call as he was moving infield to watch the batter-runner touch 1st base in case there was no catch; why not ask partner for help - he couldn't confer with partner since it was a live ball appeal rather than a dead ball appeal.
Whose call was it? Did the base umpire handle the appeal correctly whether or not it was or wasn't his call?