That is a good question.
Is the desire and attraction to another person in itself sin? No, of course not. God created each of us to appreciate the beauty with which He created other people.
But I agree with Paeter, and I think his biblical references are spot on. Allowing ourselves to dwell on and keep the feelings current and active in our minds and hearts is where we start to cross intop the realm of sin. It's natural to look at and be attracted to someone else who is married. But we must not allow ourselves to think "I want to be with her/him." If it's a former friend/flame, allow yourself to remember those feelings that were once there, but you must also recognize that those feelings were then, not now. God allowed two people to come together for a time, but that time is over, and He has something else planned for those two.
Each person we encounter and have feelings for helps shape us for the ultimate end reward that God has planned for us. I admit that I see pictures from old flames and love interests, and I have flashes of what it would have been like had things developed between me and them. But then I remember that God has blessed me with a wonderful fiance (wedding is in April), and I know that this really is God's blessing in my life. And those thoughts for my old flames last only a micro-second, because they are overwhelmed by my live for my fiance.
Hope that helps, and doesn't sound too rambling.