If you want the photograph, go to Maryland Heights. That is not a close call. The Overlook Cliff is the only place that produces the classic image of the town, and no amount of walking on the Virginia side substitutes for it.
If you want a view with almost no effort, go to Bolivar Heights. Flat, mown, its own parking area, and a wide look at the ridges from the middle of the 1862 battlefield.
Come to Loudoun Heights if you have already done the other two, if you are walking the Appalachian Trail anyway, if you want four hours in the woods within sight of a town of 269 people, or if you specifically want the Confederate artillery ground rather than the Union works across the river.
Come in winter. With the canopy off, the outcrops on this crest open toward the confluence and the town in a way they simply do not in July, and the walk turns from a good forest outing into a genuinely scenic one.
Do not come here first. A visitor with one day and one climb in them should spend it on the Maryland side.