I've just had this dilema - and i have just bought a vee (i pick it up on Saturday and i can't wait) so i know exactly what you mean.
As others have said, it is definitely a heart purchase rather than one you think about too much. The biggest smile the car has put in may face so far (not having really driven it much yet!) is my friends reactions. It is such a bonkers car that the craziness of it makes everybodies day just talking about it and laughing about how absurd it is. It's the same when you drive it around and see peoples jaws drop - I'm sure others who've had their vees for a while would agree? You need that kind of craziness in your life at least once - think of it as a public service!
You don't (or i certainly didn't) buy a car like the vee for any other reason besides driving it and owning it - if you really want the car, you'll fit the luggage / GF in somehow i'm sure!
One thing i would say though, beware test-drives. As soon as you hear that engine and see the reactions of other people on the road - you'll find it hard to remain objective! I did - 2 days and counting 'till collection...