Thanks for your reply. What I learned through going into the diagnostics is that when all the water is off, there is a flow rate of 0.5 gpm being used, so I likely have a leak I don't know about :/
This is no longer a water softener problem and is now a general plumbing issue, unfortunately.