Do I have the keyword filter setup right? Does it pull the whole thread, or only the note containing the keyword?

with keywords it does a basic substring match on content.. you could allow tags for replies, but it could get spammy..