Nate R, it's not a santee under the toilet, it's a heel outlet 90. Which is a terrible way to do it.
internatrolet, I realize space is limited, and that you were most likely recreating what was there in PVC, but this entire installation is done incorrectly. You should have redone the layout of the bathroom and used a wye for the toilet, you should have used a wye and not a santee on its side for the sink branch, there is the issue of venting, there is the issue that you used galvanized steel strapping to secure your copper tubing you naughty boy, and there is the issue of how badly the joists were destroyed simply because a properly sized hole saw was not used!
This layout needs to be planned out properly. For example, I could have taken out two bends from the sink branch simply by using a wye were you have the first 90 after the toilet fitting. I can see that you put in time to try and do this thing well, but you have exceeded your skill level.
On the note of venting and your smell, the only correct thing about your layout, is that fact that in your current wet vent setup, the toilet is placed first. Because of that, I doubt it is sucking the traps dry, however since you used the heel outlet 90, it is only compounding the venting problem. I suspect that enough air is being pushed/pulled in the lines to possibly break seal in one or both of the traps. So, as you know, you need to vent each fixture properly, and then go from there. If you happen to have everything open again, you should also double check the cast iron stack to see if they were damaged at all during the renovations.
Also, I know you have said that you reinforced your joists, but I sure hope you went overboard sistering and blocking to attempt to reinforce that mess!
I suggest, if you are unwilling to hire a plumber to take over and get you past the rough in stage, that you do ALOT more reading and learning. This is your home, and you want it to be done correctly just like everyone else right!