There's probably an air leak in the window unit framing, or between the window unit and the rough framing around it. The forced air heating is creating sufficient room to room pressure differences that it's driving air out that leak when the air handler is running, pulling air in from some other location, using "the great outdoors" as part of the return path. (It's probably coming back in via the ERV.)
If it's a vinyl window the dimensions of the window can change quite a bit with weather, and it may leak more during negative double-digit weather than when it's in positive digits territory, or conversely. Finding and fixing the leak might be possible with a blower door and infra-red imaging, but the ice-drip location is doing at least some of that locating for you. You might be able to fix it with a 3/8" drill hole and some non-expanding latex foam sealant (eg DAPtex Plus) into any hollow points of the window unit framing, or between the window unit and other framing.
It's remotely possible the air leak from the interior side is at the kickboards near the floor (that can be sealed with polyurethane caulk) or an electrical outlet box in a framing bay just under the window (sealable with can-foam or caulk), but cellulose is pretty air-retardent, making this less likely.
Adjusting the return paths for the heating supply air to reduce the room to room pressure differences to Energy Star standard, which is under 3 pascals (~0.012 water-inches) when the air handler is running, whether or not the door to that room (or other rooms) is closed or open should help too. Cheap hand held dual port manometers with a 0.01" resolution are good enough to find the biggest offending rooms, and the direction of pressurization/depressurization. Any reading greater than 0.01" would indicate a duct imbalance issue. For this room it's apparent that it's being pressurized- it's only a matter of how much. It's unlikely that mere stack-effect pressures are big enough to move that much air/water into a leak around a window, unless that's a tall house, and the ERV intake is at ground level. (At -10F outdoors, 7oF indoors you're looking at a stack effect of about 4 pascals per story, less at lower temperature differences.)