It all depends how mad you want to go really:
My helmet was c.£350
An entry level HANS device is c.£250
I got my suit off a friend for £100 (someone had bought it, but only realised it didn't fit after it was too late to exchange it)
MSA starter pack is £99
Licence test is £180
I think you have to pay for the licence afterwards, which is about £50
Even if you don't get lucky with a suit like I did, you could spend a little less on a helmet (Hedtec seem a decent budget option) to help counter the cost. You're looking at around £1000-1100 for basic gear to get you started (unless I've forgotten something obvious!)
The car very much depends on spec obviously. Many people would recommend buying a car that has already been converted. I have seen stage rally cars sell for as little as £1500-2000 but, at this price point, they might well need new seats and harnesses or just be a bit untidy. I followed this advice with my Escort and bought a car that was already converted. Unfortunately, I ended up stripping it entirely, as there was a few things I wasn't happy with. Buying a used competition car can be a minefield.
If you don't want to buy something and you just want to convert a car to gain an MSA logbook, so do the bare minimum required with no performance modifications, you're probably looking at £2-3k, providing that you can do a lot of the work yourself. The bulk of it is in the cage, seats and harnesses, but all the little bits can soon add up. There are some good deals out there though. Obviously you're going to be at the entry-level end when it comes to seats etc, but it depends how much it worries you. Some people are happy to use entry level stuff, while others insist on using really good quality safety gear.
If you enjoy it, you can always add performance parts as you go to try and make the car more competitive. I think my best advice when it comes to the car is to start with the right one. My Escort is getting old and isn't that popular for rallying, so finding performance parts (and even standard parts) can be a challenge. It's a bit boring to choose a Corsa/Clio/205/206/106/Saxo etc in a way but, the advantage is, there's lots of knowledge out there, lots of spare parts and lots of tuning options. If I was starting again, I would pick a Clio. They go well out of the box, spares are generally cheap and there is quite a lot of tuning gear out there for them. The likes of Pure Motorsport can supply you with lots of suspension upgrades, it's relatively easy (if not that cheap) to get an LSD and short final drive for one. It all just makes it a bit easier. Lots of used spares and upgrades out there too. In fact, when my Escort is done, I will probably sell it and use the Clio instead!