In DP-SGD proposed by M Abadi in 2016, noise is applied to the gradient, so every round of training needs to be applied. My questions are:
Can I choose to apply noise that meets the DP requirements to the final model after the model training is completed? Will the total amount of noises applied by this operation be less?
Noise addition is not the only thing that needs to be done to obtain differential privacy: you also need to bound the contribution from each user's gradient update, and calibrate the noise accordingly.
If a user can have an arbitrarily large influence on the ML model, because the gradient update from this user is unbounded, then you can't know how much noise needs to be added afterwards to hide this arbitrarily large contribution. So no, you can't add noise to the model after training is completed and obtain DP, or at least not in a straightforward way.
Alternative approaches to DP-SGD exist, like PATE, which kinds of corresponds to your idea of "adding noise after the training is completed", but that method involves multiple "teacher" ML models "voting" in a private way to train a "student" model, so it's more complicated than simply adding noise to the model.
