Generally speaking, I'd say that if the fantasy world contains magical creatures (elves, orcs, dragons, etc) it would be tough to have no magic. Perhaps no magic spell casting or wizards, but at least magical weaponry.
Fantasy settings without magic spells or wizards would work fine in a fantasy setting, even to the point of all magical items being rare (or unique of its type -- there is only one Vorpal Sword or a single Displacer Cloak in the world). Or changing the rules of magic (e.g. having "talents" as in the Xanth books or using a mana-based system where magic only works in certain areas and its strength is variable). Or only having certain races able to perform magic (the Elves or the Istari, for example) or only allowing certain types of magic (clerical spells or illusionist, um, illusions) can add to the "mystique" of magic in the world.
However, a high fantasy setting generally assumes magic exists and someone (or some thing) is able to control it. Otherwise it's just funny looking demihumans knocking about each other with various weapons.
Now if you stray out of high fantasy...