You can play with to see what happens.
In short: without a passphrase, the 12/24 words are entropy and get hashed into your private key. With a passphrase, this is added to the entropy, creating a completely separate wallet.
The difference between the two is that the bip39 encoding (the 12/24 words) have a checksum built in, so if you make a mistake, it most likely will catch that. You can try this by changing one of the seed words if you generate a random seed.
The passphrase does not have a checksum, so any typo will create a completely new wallet.
Not the best explanation, but mainly a shill for Ian's great tool.
BIP39 - Mnemonic Code
