OK, I've done a little more research, and I think I understand the current situation re. my PRV.
I noticed that the pressure at a hose bib after the PRV measures roughly 95 PSI static. This is the same as the pressure at the hose bib before the PRV.
What's strange is that adjusting the PRV appears not to make any difference to the static pressure, it always stays at 95 PSI regardless of the setting of the adjustment screw. I tried to put the adjustment screw back where I thought it was when I started, but didn't spend too much effort doing this because I thought it wouldn't make any difference anyway, since the unit appeared to have failed.
Later this evening, while giving my 3 year old a shower / bath, that the pressure did seem to be down a little; perhaps my adjustments had made a difference after all.
I looked around the 'net a little more, and found some comments saying that the PRV is faulty if the pressure shown at the hose bib drops significantly when a single faucet is fully opened. I believe the figure suggested as being acceptable was a 15 PSI drop, with 8 PSI being the performance to accept from a new unit.
When I opened a single faucet, the pressure gauge read only 40 PSI; a 55 PSI drop! I had noticed this before actually, that the pressure drops significantly whenever there is flow, and slowly builds back up again over about a 20 second period to the street pressure of 95 PSI.
Now I'm thinking that the pressure under flow function of the PRV is actually working OK, and I've now turned it down a little too far by setting it to 40 PSI by accident. It was probably set OK in fact, and the 95 PSI static reading was a red herring! The PRV has failed however, and is leaking water past the seals at rest, resulting in the elevated 95 PSI.
Obviously this still isn't good for the plumbing in my home, but at least I think I understand the problem now.
Interestingly, I measured the static pressure at several of my neighbour's houses this earlier this evening. Of the 4 I measured, 2 read 95 PSI, 1 read 80 PSI and 1 read 60 PSI... I have a feeling that almost all of the PRVs in our 10 year old development now have this fault!
The photo in my post above shows the location... I'm considering a repair kit because getting the old unit out might be tough.