Okay so if I'm reading this correctly, if my pitcher actually touched the pitching rubber as he bent down to pick up the ball, regardless of whether he throws a warm up pitch or not, it would count as an appearance which answers my question of whether he needed to actually pitch or just touch the ball.
I can't recall if he did that, (he might have), but it looked to me like he just walked over and picked up the ball which was just in the vicinity of the mound, (okay just the gravel as our parks don't have real mounds!), where the previous pitcher on the other team dropped it.
All in all, I guess what bugs me the most is that I didn't just get a courtesy 'warning' and instead had to re-juggle my lineup again as my closer was now effectively 'out' and everyone around me had a 'huh?' look on their faces.
Oh well, it was my mistake and it is a lesson learned and I won't be doing that ever again!
Thanks for the responses.