I agree - I think he would've seen Lily smiling at him, letting him know that he was forgiven. It's losing her friendship that devasted him, then when she died ... it broke him.
I think I've known how I've felt since the first time I met you...saw you. I know I'm hard to talk to sometimes. I DO hate Portwenn...but it's where I want to be because you're here. Because of you. Cause if I'm with you, nothing else matters. What I'm trying to say is...I love you. (MARTIN to LOUISA - Doc Martin)