Downtown here is small, few square blocks, and right next to FSU. Aside from the outstandingly good co-ed type scenery I'd avoid living there like the plague. I drive through there all the time, but wouldn't want to live there again unless I was the right age to be hunting... It would, however, be worth it to eat dinner downtown on a Fri/Sat night to get a sense of the area, and go for a bit of a wander after dinner. I'm wondering if, even with students, a heavy night here would be like a quiet Monday there...
Town overall is small enough that I'd encourage finding your office, then find the nearest edge of town and start shopping. North East, look for the Kilearn area, then avoid it, go just west of there for much newer areas where you'll power that stays on. Mid East, look out towards Chairs. South East look around Southwood up to Old St Augustine, anything in that area. You can get pretty far out and still easily be less than 45 minutes here.
West side is more old growth- prices will be lower, homes mostly older.
FWIW, Zillow has a good handle on the town, and The Wife has been house hunting off & on for a few years. If ya'll come down we'd be happy to meet for lunch- she'd love to give ya'll the skinny.