Little is known about Sivas before the Hittites, under whom it was an important settlement. Cimmerian and Scythian invasions hit it in the 7th century BC; the Medes held it at the start of the 6th century and the Persians by the middle of the same century. Alexander took the city in the second half of the 4th century BC, after which it was attached to the kingdom of Cappadocia, and in AD 17 it passed with Cappadocia to Rome.

Under Rome it was a walled city and an important metropolitan see, and from the 3rd century Christianity spread here quickly. Local tradition credits a saint from Sivas, Blaise, with much of that spread through the miracles attributed to him; Muslims here regard him as a saint of the eyes, and his grave is held to lie near the Gök Medrese, outside the walls of the time.