You can tag him anytime to play a duel, or he does notes about it too.