I suppose they can technically, but in practice it doesn't matter if they can decline the loss of down penalty or not.
Example:
3rd and 6 from B's 30 yard line.
Ineligible A53 touches a forward pass at B's 27 yard line and the ball is:
a) caught by A53 and he is downed at the 25 yard line.
b) incomplete
c) caught by A53 and he is downed at B's 20 yard line

If the penalty is accepted, it will be a), c) 4th down from B's 32 yard line, b) 4th down from B's 35 yard line.
If the penalty is declined, it will be a) 4th down from B's 25 yard line, or b) 4th down from B's 30 yard line, c) first and 10 for A at B's 20 yard line.

In practice, B would never decline the penalty in c).