Looking on women is not an impediment to spiritual progress, but looking on women with a view of sense gratification is detrimental. The actual fact is that attachment for sense gratification is not at all congenial for spiritual progress. In this country intermingling with women is very easy, and sometimes our mind becomes agitated. Therefore we have to take little precaution and the best precaution is to raise oneself in Krsna Consciousness. Lord Caitanya said that His mind becomes agitated even by seeing a wooden model. By seeing women, if one's mind is agitated, that is quite natural. If you take the words of Lord Caitanya that His mind is agitated by seeing a wooden model, then what to speak of us by seeing actual women.
The real fact is therefore that we have to check ourself by advancement of Krishna Consciousness. But if it becomes too difficult for us, then one should get himself married and thus check his sex disturbance, and peacefully prosecute Krishna Consciousness. But if one can avoid sex life and the attachment is overturned for Krishna Consciousness, his position is very laudable. So there is no need of becoming hypocritical; better one can get himself married and be peaceful.