"I'm on my break right now." As in, he was technically working right now. It might not be his turn to shadow Rufus or stand guard near him at the gazebo, but that didn't mean he was off the job. But saying he's working right now, would only draw more questions he doesn't want to answer.