I had this happen in a game last fall and after doing a little research, I have decided that I didn't call it right. I am on a wing. I have a defensive end on most plays simply grabbing the offensive tackle and wrestling him to the ground. No effort to make a play on the ball carrier. He was basically giving himself up in order to take the tackle out of the play. I brought it up to my WH during a timeout and he said there was nothing wrong with that. In looking at rule 9-2-3c, it says a defensive player shall not use his hands or arms to hook, lock, clamp,etc in an effort to restrain an opponent other than the runner. Am I interpreting this correctly? During the game was no time to argue with my WH about it and I wasn't sure of the rule. It just didn't seem right for the DE to do that. What's your position/expirience with this?

Canadian Interpretation
 

To call holding on D there has to be some form of advantage gained. An example would be to allow another D player through the line unblocked. If the tackle's job was to block the end and all the end did was grab the tackle then I have nothing since the tackle should be pleased that he accomplished his goal.

I would be careful as well
 

I would have to see it to be sure, but I would tend to let it go, unless as stated above he allowed a teammate to get to the ball directly by his actions....and certainly don't pass on it in the 1st period then nail it in the 4th LOL

Holding back a pulling lineman is also a foul. Make sure there's an impact before calling it.

This is definitely a foul but you have to see it in context to decide whether to call it or not. As the others mentioned, if the idea of holding a tackle is to let a defensive linebacker charge through the line, then it has to be called for sure. If the tackle is going after the end, then there is not impact so you let it go. If the offensive tackle is not going after the and is taking off after someone else, then I would call it as B's holding is disrupting Team A's blocking scheme.

I had a high school game as Umpire once and the offensive center and defensive nose guard held each other on every play. When possession switched, the center became the nose guard and vice versa and they continued to dance with each other. it made my job very easy as I had too less players to watch.