I have the x100s and got it really as a decent travel camera - which it's great at, so small and light weight. I have hardly used my DSLR since I bought it it's that good.
That said, it does take quite a bit of getting used to - the fixed lens, you do end up having to change the way you take photos. But I always looked at that as a creative challenge. It has a lot of 'Instagram' style filters built-in, which although are fun to play around with at first, I found myself getting a little bored of them. The film effects are much better - I
love shooting black & white stuff with the S, it's great! It's also great in low-light (lack of noise at high ISO), but it does work differently to an SLR in very low light situations - i.e. almost darkness and shutter speeds of around 30secs.
The only issue I've had is the buttons on the back stopped working. Ended up going back to Fuji who repaired it and serviced it for free. I haven't heard of this problem from anyone else - so just one of those freak things I suppose.
I'm very much still learning how to get the best from mine, but it's definitely a camera I'd recommend.
My Flickr