Am I wrong for fantasizing about someone else?

Ok so I date this guy but we aren't together however we are intimately involved. But during intimacy with this guy I always fantasy about another guy I really like. Is this wrong?