That's completely unfair. What makes them so special that they get to follow their beliefs only when they want to? Can I make a legal agreement then choose to go back on any part of that contract based on my whims? No, of course not. This is absolutely no different.
People need to be held to their beliefs. We need intellectual honestly in society, not a society poisoned by lust and based on whims.
If you practice homosexuality and that doesn't go against your beliefs, not my problem. But if you adhere to a set of beliefs I'll hold you to them.