I've got a pretty reasonable collection of lenses and my motto is to stretch myself and buy the best I can (so L lenses if that's what is needed). But, I have to say that my 17-55mm is the lens that time and time again I look at my pictures and notice that there is something just a touch better about this one - better colours, sharper etc...
It's so pronounced that it's enough for me to want to keep hold of my 40d rather go full frame and get a 5d Mark II - this lens doesn't really have a full frame equivalent. There's the 24-70mm 2.8, but it lacks IS and is *much* heavier.
Ok so the body isn't the metal quality of my L series lens, and some of them attract dust but -
1) The dust can be removed if you really need to, most people find it makes no difference to their pictures (BTW I have no dust)
2) The body may not be L quality but it is far from bad, it's solid enough for me
3) Some stories of ropey IS, haven't had an issue myself but if you are concerned switch it off when in decent light and save it for indoor or night shots when you really need it.
If this lens wasn't as good as it is I would have sold my 10-22mm, 60mm macro, my 400d and 40d bodies to move to full frame, but this bad boy is enough for me to stay with the crop bodies.