Your never going to get everyone to agree on what a sport is. Hell, some people think mud wrestling and poker are sports.
I think we each have our own opinion on what constitutes a sport.
In my humble opinion I would only allow sports into the Olympics that have no judges. Say goodbye to figure skating, gymnastics, boxing(unless they go until a knockout or TKO), diving, style points in sky jumping, "artistic" snowboarding events, etc.
Having said that though, I would also get rid of the 2-man luge. Any activity where one guy is lying on another and moving together as one is something I don't want to see.