I would think the type of box, and the woofer you select are more important factors than size. Size really just determines efficiency, however, for a given cost, you might find you get a better quality speaker if you pick a smaller one.
I don't really know too much, but a non-ported (ie sealed) box could be the way to go. Basically you're going to be trading quality for efficiency (ie volume at a given amp power). Fast bass is all about eliminating distortion and resonance.