Any "all purpose cleaner" mate. work it in with an interior brush if needed and then wipe off. Something like Meguiar's APC diluted 10:1 would work fine.
Easiest way to do it, is hold something like a towel in one hand with the area of seatbelt you want to clean on top of the towel, spray some APC on and leave to dwell for a few seconds, then work it in with hte brush (gently, or itl go everywhere - the towel will prevent/limit the 'mess') and then wipe clean.