Since there is lot of hatred going on between people of different religions (Hindu's and Muslims) which never made sense to me no matter what be the reason, but a good logical explanation to certain questions can sometimes handle things in a better way. So I am sharing this below article which I read somewhere but since i found it worth I am putting all the points here. Please use your own intellectual mind before or after reading.
Non-Muslims are not allowed in Mecca because it is a holy sanctuary. One has to qualify certain requirements to be there and be a believer. Understand it like this: civilians are allowed in cantonment areas or high security areas. You need a certain permit to be there. It’s the same with Mecca. In this case the permit is being a Muslim. Also please note that Mecca as a city has no significance. The only significance is the presence of Kaaba.
Mecca and Madinah are cities of great importance in Islamic tradition - centers of pilgrimage and prayer, sacred places where Muslims are free from the distractions of daily life. Restricting access to these holy cities is intended to provide a place of peace and refuge for Muslim believers and preserve the sanctity of the holy cities. At this time, millions of Muslims visit the cities each year, and additional tourist traffic would simply add to the congestion and detract from the spirituality of the pilgrimage visit.
Mecca was under the control of the Ottoman Empire from 1229 to 1923. The quote below is from a text available from 1564 in which it states that in those times only people who did not believe in one God were forbidden entry into Mecca. ‘No Muslims and believers in the unity of God should be hindered in any way if he wishes to visit the Holy Cities and circumambulate the luminous Ka’ba [in Mecca].’42 42. M.D., vol. 6, f. 17, firman no. 39, 1564-5, quoted by Faroqhi, op. cit. 147 (Ottoman imperial firman).
The Quran says: “Oh you who believe! Truly the idolaters are unclean; so let them not, after this year, approach the Sacred Mosque….” (Quran – 9:28) Mosques or holy places in other religions as well as places reserved for meditation usually have basic requirements for entry - cleanliness and purity. Prayer is a form of meditation and meditation requires cleanliness. This basic principle drives the restriction to not allow someone 'not clean' into a Mosque - be it a Muslim or a non Muslim.
Although it is said that even Muslims are prohibited to enter the Mosque if they are not clean, but no one can keep a check on this. It is believed that every Muslim will follow cleanliness guidelines as per Islam which mentions a specific way to take bath and perform ablution to touch the holy book of Quran or perform prayers. It is obviously assumed that people from other religions being unaware of these rules are not purified in that sense and hence denied entry to Mecca.
There are numerous stories of non Muslims visiting Mecca in the past, including the Kaaba, such as Sri Guru Nanak. Another famous incident of a non-Muslim visiting Mecca was the visit by the British explorer Sir Richard Burton in 1853. Burton disguised himself as an Afghani Muslim to visit and write Personal Narrative of a Pilgrimage to Al Madinah and Mecca.
Guru sahib dressed as a Fakeer, holy man, which meant a chola of khuddar (rough material) and a pair of wooden sandals with his companion Baba Mardana, who was from a Muslim background, infiltrated the city of Mecca. “Donning blue attire then Baba Nanak went to Mecca. He held staff in his hand, pressed a book under his arm, caught hold of a metal pot and mattress. “ (Vaar 1, Pauree 32 – Bhai Gurdaas Ji)
The major reason is the policy of Saudi Arabia as a country. There are two major events annually - Haj and Ramzan, that draw huge Muslim crowd from around the world pouring into the city of Mecca. It is also evident that the crowd grows every year and Saudi Arab administration has to pull majority of the resources from the country to manage the two events apart from the usual tourists who don't come to take part in these two.
There is even a cap on the number of Muslim visitors each year that are serviced by Saudi Arabian administration which differs from country to country. There are usual stories of mishaps happening despite so much management of the event. Apart from these two main events, there are year long visits that maintain a constant flow of Muslims during the 'Off season'. The sheer magnitude of Muslim crowd and several other reasons force the Saudi administration to prevent tourists and visitors from visiting Mecca
All adults in Saudi Arabia carry some form of government identification, whether it's their resident's permit, their national ID, or the visa in their passports. All of these IDs indicate whether the bearer is Muslim or non-Muslim (Saudis are assumed to all be Muslim). On the highways into Mecca and Madinah, there are checkpoints. If the guards at the checkpoint have any suspicions about those passing through, they simply ask to see the ID. If it says 'Non-Muslim', they are turned around.
The system is not foolproof. Back in 2002, the Italian ambassador to Saudi Arabia was discovered in Mecca. He told authorities that he was actually Muslim, but his story had more than a bit of back-and-fill to it. Non-Muslims found in either of the two holy cities definitely get in trouble. They can be deported; they can be put in jail for a while. However, it is no longer a capital crime as it was in the 19th C.
Haj is the fifth pillar of Islam. Every Muslim who has the means (physically & financially) to go to Mecca needs to go there at least once in his/her lifetime. Also, its a feeling of unity during which millions of Muslims set aside all differences of race, caste, economic status, nationality, and even sect as they unite in the holy pilgrimage, each donning the simple two-piece white garb of the non-attached traveler, allowing nearly no apparent difference whatsoever between men as they stand together before their Creator.
Mecca has the holiest place for Muslims Masjid al haram or the grand mosque. Inside the grand mosque is the kaa'ba. The kaa'ba is a one-room building, covered in black stone,it is the first house of monotheistic worship in the world where Muslims all around the globe turn to 5 times every day for prayers. Note that Muslims do not pray to or worship the Kaa'ba. It is merely the focus point for Islamic prayer.
Owing to all such the points it doesn't make sense to criticize the prohibition to enter the city or giving confusing reasons like 'Non believers not permitted' to the question. The problem is completely logical and the solutions obvious. There are similar cases of prohibitions in Hinduism and Christianity, namely a Temple in Kerala and several orthodox churches that say “no entry to people of different faith”.