Are You Waiting for Me or Expecting Me?

One word in Russian that I always have trouble with is ждать.  It can be used to mean to wait for or to expect.  In my English brain there is a big difference between the two and it's made worse because I hate to be late.  So if a Russian speaker tells me they're expecting me, I always panic and hear that they're waiting for me, even if I know we'd agreed on a later time.

I'm finally starting to get a little more rational and after thinking they're waiting, I realize they're simply expecting me at the appointed time.