"1. Can a defensive player break the plane as long as he doesn't touch player or ball while still in players hand?"
No. Not until the ball has touched a player on the court.
"2. Is it violation for defense to step on the OOB line, defending inbounds play? what is the penalty if violation "
Yep. The OOB line is part of the plane. Same rule applies. Penalty is a warning for delay of game. Reported to the table (at which time the scorer should record time that it happened)and the coach. Second infraction for delay of game is a team technical (not assessed to the coach as either direct or indirect). BTW, not to state the obvious, but there are other instances besides a throw-in plane violation by the defense that will incure a delay of game warning/technical. Also, the offense is not allowed to break the plane either.
"Connected to this. Is it an inbounds violation when the offense releases the ball and hits OOB floor before inbounds or player inbounds? "
yes
check the Rule 9 section in your funny book (Simplified and Illustrated NFHS rule book). It does a good job of explaining your questions.....with pictures even...bonus