I felt it more with S3 especially in terms of how M/M was written. S3E1 started out strong and then after that ep it was just so-so. I didn't understand why Mary kept bugging Matthew about Lavinia's money. Yes I got that it should help Robert's money problems and he was being dramatic about it but then Robert's financial problems were not really cause for alarm because there was Lavinia's money to conveniently solve this. And the whole baby problem was introduced rather late and if Matthew will die, then they have to resolve it right away and that's exactly what happened.