Back Swing Interference or Play On

[tankmjg24]

So from my understanding of the back swing interference rule it exists to help the defense and not the offense. FED rules. Batter swings and makes contact with the pitch and hits a grounder right down the first base line. In his follow through his bat makes contact with the catchers helmet. I have a no call and the first baseman fields the ball and tags first and the inning ends. Neither coach says a word and the game continues. Was I correct?