Maybe your upload speeds are too slow. Or maybe there's something wrong with the router setup. Did you try with a direct connection, bypassing the router?
There could be a lot of reasons why you might not be getting good download speeds, but one of the first things you should try are other torrent clients.
Transmission, for example, it has far fewer options than Azureus and is supposed to work very well, meaning there are fewer settings to mess up and fewer things to diagnose. And while you're trying it, make SURE that your port is open (there's a test for that) not just that it's SET to be open.
Personally, I like Azureus better but there are a million options to tweak and if you're going to mess with them you should know pretty much what you're doing. A lot of people swear Transmission is the best so be ready to switch if it performs better.
Here are some more
torrent clients.
And finally, here's the
Vuze forum if you want to stick with Azureus.
Otherwise, maybe someone else has a suggestion.