Late Libero Replacement

[Vb Scrub]

I have a question regarding a late libero replacement with respect to NFHS rules.

In a JV match last night, the JV coach was sending the libero in while her team was serving. The coach was very late in her decision to make the replacement. The ref was blowing the whistle for serve as the player was entering the court. The player being replaced ran off the court, and in my opinion was off prior to the ball being served. The ref had a moment of hesitation and didn't look quite sure what to call. He blew his whistle and signaled out of rotation as the ball was being contacted for serve.

I am more familiar with NCAA and USAV rules than NFHS. So in NFHS, what would the correct response/call in this situation? Also, what would the correct call/procedure be if the libero ran onto the court after the ref beckon for serve?