Possibly a fault with the AC demand line to the ECM or even a compressor fault causing the belt run to drag a little more than it should. The system is designed to send a signal to the ECM via a specific line when you press the AC On button that tells the ECM to increase the idle speed to compensate for the request for additional load on the engine. If there's a problem with this the ECM won't know to increase the idle and so cause the slower, jumpy idle condition.
Ideally it needs diagnostics to say for sure if the ECM is seeing the 'request' but it depends on how much of an issue it is for you. If it is a faulty compressor unit it could eventually cause the aux belt run to seize and cause other damage... honestly, I'm an optimistic person but I prefer to plan for the worst and hope for the best when it comes to working on other peoples cars.
Mick