steering rack??

 

Get an alignment, more caster would help and check the toe in/out settings. It shouldn't feel like that.

Make sure it is all OK, nothing bent/broken/missing/overly-worn.

What tyres are you running? If it's the Longreach Outback with the light truck tyres try swapping them for sedan type radials.

 

More caster angle will make a significant difference, especially at speed.

There is a relatively easy way to get more caster, and that is to swap the front lower control arms across. I.E. Put the left in the right and vice versa. On an XH (which has the same X-member as an EF/EF) you will need to get new poly inner bushes and install the bushes backwards on each arm (so they are orientated correctly once the arm is installed on the opposite side of the car). This will give you around 4-6 degrees of caster more than before, and you will still be able to fine tune the caster and camber angles using the shims on the upper arms