What you said is true, lover is a big word, it has sevral meanings:
(1).A couple in love with each other. 恋爱中的一对;
(2). A paramour.情妇,情夫;
(3). A sexual partner.性伴侣;
However, it's not that bad to call your boyfriend or husband ----lover~~~~~
Generally speaking, lover doesn't stand for the sexual partner~~~at least I believe!!!