^^^^ I don't find it odd that Damon went back to Elena. He's drawn to her. Her slapping him and resisting him, probably makes him want her more now, since he knows he can't have her. I dunno, that's my theory anyways.
Plus Damon knows that Stefan put that herb in her necklace to protect her from Damon and to be able to resist him. It was an herb right, that Stefan put in the necklace, or did I hear that wrong?