Whate the...sammy where did you learn this cr*p?!
It's true that you want to keep the gas energy as high as possibile, but there are so many elements to consider...
I mean, sound comes from the exhaust pulse from the engine, in the end the internal combustion engine works with small explosions, which creates pressure waves that you hear as noise...if you want the best performance you need as much speed out of the combustion chamber as you can. You use the wave produced by one cylindre to "pull" the exhaust of the next one, so a good manifold is better than a set of pipes, one for each cylinder, with pipes lenght and diameter depending on the engine characteristic.
After that what you need is:
-low backpressure
-low gas inertia
The trick is to find an exhaust that is both very freeflowing but at the same time let the gas come out as fast as possibile, so if you go with large pipes you have low backpressure but high intertia (the gas will be colder, there will be more of it in terms of volume AND density--->mass--->inertia), if you go with small pipes you'll have a lot of backpressure.
Noise is just a consequence, and in a race car you just don't consider this (if there are no noise limits).
If you put a silencer on the exhaust line, you'll have a loss of energy, as the gas goes through the holes in the pipe, it'll get colder, slower, denser and heavier.
A silencer is just a chamber that convert energy (in the form of pressure waves, that you hear as sound) in heat... energy that is lost!
On a road car you try to do this with the highest efficiency you can...but it's still a compromise.
The real reasone behind the fact thay many aftermarket reduce the mid and low torque (and power, as it depends on torque and revs) is because they just are the wrong diameter, usually too big.
In fact, to be specific, if you fit to a completely standard car a 60 mm catback, you lose a bit of torque at low rpm...you can get over that with a sports cat, so you can have the same torque at mid rpm, and higher at high rev.
An exhaust should always be developed with a dyno or at least taking acceleration times in gear and in differen rev range. The best is to measure back pressure before and after the cat so you can work on it and on torque and power values.