how do u make door handles flush with new doorcards?

[J9HHY]

I have made some new doorcards up from some sheet plastic which are about 5mm thick. Velcro'ed to the doors but when I replace the handles they stick out a bloody mile! What have other people done to make them flush? Also are the handles made from aluminum shallower?