3 way switch wiring problem

I have 4 wires going into the box that had a fixture that was controlled by a three way switch. Two are clearly white and black and when I test them I get 120 volts. Two are so old that they both look black. So I took a guess and connected what I thought was the white with the definitely white one coming from the power. When I turned on the switch I had 120 volts coming from the white and black. But when I turned off the switch I had about 3 volts. So I switched the two unknown wires thinking I had connected the black and white together but got the exact same results. 120 volts when switched on and about 3 volts when off. So no matter if I have the hot and hot connected or the hot and neutral connected I sill get about 3 volts when it is switched off. Any idea what my problem is?

 

There is no problem. Digital Multi Meters are notorious for random readings when neither of its leads are connected to ground. Ignore the 3V.

 

Oh, ok but since I get the same reading either way I hook up the two wires coming apparently from the switch does it matter? I can't tell which is white and which is black.

 

They could both be black in some cases, particularly when the wires are in conduit.

In a 3-way switch, there are two switched wires that are called travelers. One will always float, and will serve as an antenna picking up electric fields. Even if you saw 20 volts instead of 3 volts, still no worry.