While Palm Springs is famous for its mid-century glam and poolside lounging, the magic and charm also lie in its more eccentric corners.