I know one person who uses “whom” correctly in spoken English, a former co-worker of mine, and when I heard him do it my respect for him instantly went quite a bit higher. I don’t even know why but it seemed like it showed a good window on something good within him.