Yep go for it, Get the wash and prep ready, then see how it is from there.
Work on a panel at a time, on the polish stage so you can go back to it if it starts to rain.
If it starts to rain and fairly light, then once it's ended you can go back to it by "quick detail spray" cleaning the panels backup and get it ready for the polish.
the only thing though is sealants and wax etc some of them need time to bond etc which is a pain if it rains.