So I read this and this and solved my problem. Basically I bind /temp/download to /media/$USER/download and then I can set the download directory in Chromium and Firefox snaps to /media/$USER/download and my downloads go where I want them. This was actually harder to figure out than it should have been.