Your Korky 528 is an anti-siphon fill valve, which means that if there is suction within your potable water piping in your house (i.e. you drain your house piping for some reason), the water in the toilet tank isn't going to be sucked back into your potable water system. (Eeeew.)
Because of this anti-siphon feature on the fill valve, it's going to drip water from a few places in the valve while filling. It won't drip water when it's not filling. The purpose of the valve is to put water into the toilet tank, so it doesn't matter whether a few drips fall off of it into the tank or not; where the water comes from on the valve is of no importance as long as it ends up in the tank.
Korky hears this concern over and over and over again, and so the instructions on the valve used to be quite clear in explaining this; now they are a little more muted "Normal water flow from 3 areas" -- it doesn't really say don't worry about the drips anymore, but that's what they are saying. See Item 6(c) and the associated drawing here:
http://www.korky.com/PDF/instructions/528.pdf
You can use waterproof tape around the top of the overflow riser to seal the crack; it's a common place that overflow risers crack. If you want to replace it, Korky makes a pretty-decent flush valve with a twist-lock that lets you easily raise and lower the level of the overflow; you don't have to cut it with a hacksaw like most overflow riser replacements. It's not a difficult install; usually the hardest part is getting the underneath-the-tank nut on the old flush valve to turn; the metal ones tend to rust in place and I have had to actually cut off the nut with a Dremel tool on a couple of our toilets because no amount of leverage would turn the nut on the shaft.
Even though it's not complicated to replace the flush valve, it is still more of a pain than replacing a fill valve, which is easy. If you can seal the crack with little effort, I would probably be inclined to try that first, although it's not what a plumber would do.
Also, if the water level in the toilet as you have now adjusted it gives you a decent flush, you can just leave it as it is and save the water...