As DrDrew already explained it's normal with TCP/IP. Whenever you are downloading anything it needs to send back "ACK" (Acknowledge) to the sending server to let it know that it received the data in good condition.
Just for example - say the server sends you 1Kb packet your PC sends back an ACK for that, now you probably have a 15,000Kbps or higher connection so it has to send back tons of ACKs each for each of those packets.
Downloading at say 50Mbps, you'll need to send back about 1Mbps of ACKs or so (ok maybe that's pushing it
- i'm just guessing here). So if your YouTube upload is using 4.5 of your 5Mbps upload then you can't ACK fast enough to download faster.
Using UDP now is another story but yeah we'll leave that alone for another day as hopefully that was already too confusing... LOL