Is your heating system running off the hot water heater? (Rare, but it happens.)
Most homes with radiators would have a separate boiler for the heating system, but in those homes it's usually higher net-efficiency to set up an "indirect" hot water heater operating as a zone off the heating system than to have a separate stand-alone atmospheric-drafted beast like the one in the picture. It's more money, but it's more efficient, and much better overall hot water performance.
The valve on the side is a temperature & pressure relief valve to avoid the exploding water heater problem should something go wrong in such a way that the burner never turns off. The plumbing out the wall is probably the outlet for that valve, but the picture cuts off the likely connecting plumbing.