Some negative attitudes of disability may be coming from Old Testament Mosaic law. One example that I included below from Leviticus seems to imply that a person with the physical "blemishes" described below is not holy enough to approach God as a priest.
"16 The Lord spoke to Moses, saying: 17 Speak to Aaron and say: No one of your offspring throughout their generations who has a blemish may approach to offer the food of his God. 18 For no one who has a blemish shall draw near, one who is blind or lame, or one who has a mutilated face or a limb too long, 19 or one who has a broken foot or a broken hand, 20 or a hunchback, or a dwarf, or a man with a blemish in his eyes or an itching disease or scabs or crushed testicles. 21 No descendant of Aaron the priest who has a blemish shall come near to offer the Lord’s offerings by fire; since he has a blemish, he shall not come near to offer the food of his God. 22 He may eat the food of his God, of the most holy as well as of the holy. 23 But he shall not come near the curtain or approach the altar, because he has a blemish, that he may not profane my sanctuaries; for I am the Lord; I sanctify them. 24 Thus Moses spoke to Aaron and to his sons and to all the people of Israel." - Leviticus 21:16-24
Another biblical example is from the first chapter of Daniel describing events related to Judah being conquered by King Nebuchadnezzar of Babylon.
"3 Then the king commanded his palace master Ashpenaz to bring some of the Israelites of the royal family and of the nobility, 4 young men without physical defect and handsome, versed in every branch of wisdom, endowed with knowledge and insight, and competent to serve in the king’s palace." - Daniel 1:3-4 (emphasis mine)
Daniel was one of these who were brought to King Nebuchadnezzar's palace.
Plato also wrote in his Republic, "And then, as the children are born, they’ll be taken over by the officials appointed for the purpose, who may be either men or women or both, since our offices are open to both sexes. Yes. I think they’ll take the children of good parents to the nurses in charge of the rearing pen situated in a separate part of the city, but the children of inferior parents, or any child of the others that is born defective, they'll hide in a secret and unknown place as appropriate. It is, if indeed the guardian breed is to remain pure."
Greco-Roman philosophy has had a huge influence on our culture, and not always in very positive ways.