I tend to work from the road backwards as that tends to give you the right priority in your thinking
Are your tyres the correct size, a decent brand, not excessively old and in good condition?
Are tyre pressures correct?
Are the tyres mounted on the correct width and offset of rim?
Are the wheels aligned correctly (toe and camber make a huge difference)
Are the wheel bearings in good condition (no play when jacket up, or noise when driving)?
Are all of the bolts in the front suspension tight (once over with a torque wrench and paint mark them)?
Are the balljoints in good condition, boots intact and no play (worn balljoints go slack and have a much worse effect than soft bushes)? Check bottom balljoints and inner and outer track rod ends
Are the bushes in good condition? Off the top of my head, topmount, lower wishbone, steering rack. ARB too but it wont give the symptoms you describe.
Once you know that the various pivots are in the correct place and controlling movement properly, then look at damping and springs.
Far too many cars on the road with nice spring and damper kits and worn bushes or balljoints because £400 on konis looks better on the internet than £400 at a garage to have your bushes and balljoints done.
Unless you're going to be competing, and therefore uprating lots of bits, I'd work through the standard parts, replacing any which are worn until it drives nicely and then start your upgrades.