Double First Base Appeal

[PaREF]

A situation came up last night in a co-ed league game in which I was playing that I had not seen before. A slow ground ball back to the pitcher who fielded it and threw to first. The BR was ruled safe in a close play (no argument about him beating the throw).

After the ball was returned to the pitcher he said to the umpire that he would like to appeal the runner interfering with the first base(wo)man by stepping on the white part of first base instead of the orange part. The umpire said that it was too late for an appeal since the runner had already returned to first base.

The pitcher said that since he had not begun pitching to the next batter that he should be able to make an appeal but the umpire said that particular appeal play had to be made prior to the runner returning to first base.

I did a little ASA umpiring about 10 years ago but nowhere that used a double first base. I've watched many high school and college games where it is used but I've never seen an appeal of the runner not touching the orange base. Could someone help me out and tell me if the umpire was correct in his ruling?

I really don't know which base the runner touched, but let's just say for the sake of argument that his foot came down on the white base. Could he be called out on whatever is a proper appeal?