The city's name is two names joined. The steep conical hill in the middle of town, 226 metres high among volcanic mountains, has drawn people since the Hittites, and although it was defensible already it was ringed with three lines of wall — lower, middle and upper. Each ruler named it: Hapanuwa to the Hittites, Akroinon to Romans and Byzantines, Karahisar-ı Devle and Karahisar-ı Sahip to the Seljuks, the principalities and the Ottomans.

The other half comes from a plant. The opium poppy has been grown here since the 2nd century BC as a source of livelihood, and the Latin opium, said locally as afion, gave the town its other name. The two were eventually put together as Afyonkarahisar, a form first found in court records of the 17th century.