Personalized Federated Learning for Heterogeneous Residential Load Forecasting

Using DP algorithm in FL is an effective way to obtain a strong privacy guarantee. However, DP’s sacrifice of accuracy hinders its entry into practical application scenarios. Here we adopt GAN-DP^{[ 34 ]}, a modified GAN model, in the above FL setting, which improves the load forecasting accuracy while complying with the DP requirements.

As shown in Fig. 2 , GAN-DP contains a generator, a discriminator, and the DP identifier (DPI) whose structure is similar to the discriminator. In our system, the local parameters are obtained through local training by the client. Then, the local model parameters are fed into the generator, and the generator produces a group of synthesized parameters. The generated synthesized parameters are then treated as inputs to the discriminators and the DPI. If the synthesized parameter satisfies the requirements of the two perceptrons, the parameter is taken as the output result and goes to the next step. Table 1 provides brief instructions for notations in this chapter.

10.26599/BDMA.2022.9020043.F002 Fig. 2GAN-DP. “A” and “R” indicate acceptance and rejection, respectively.

Generator: Utilizing the original local model parameters, the generator produces synthesized parameters and submits them to the discriminator for identification. The $N_{d_i}$ noise samples $\{y_1,y_2,\mathrm{\dots },y_n\}$ from $p_g\left(y\right)$ are inputted and updated by

(6) $${\nabla }_{{\theta }_g}\frac{1}{N_{d_i}}\underset{i=1}{\overset{N_{d_i}}{\sum }}\log \left(1-D\left(G\left(y_i\right)\right)\right)$$

Discriminator: After several iterations, if the discriminator identifies the synthesized parameters as the original model parameters, the final result is obtained. Here we select $N_{d_i}$ data samples $\{d_1,$ $d_2,\mathrm{\dots },d_{N_{d_i}}\}$ from $\delta (d_i)$ , and the gradient ascent of discriminator in the update process can be formulated by

(7) $${\nabla }_{{\theta }_d}\frac{1}{N_{d_i}}\underset{i=1}{\overset{N_{d_i}}{\sum }}\left[\log D\left(d_i\right)+\log \left(1-D\left(G\left(y_i\right)\right)\right)\right]$$

DP identifier: Since the DPI serves as a discriminator, the discriminator and DPI interact with the generator in parallel. Unlike the discriminator, DPI attempts to confirm whether the synthesized model parameters satisfy the DP requirements. The update procedure can be expressed as

(8) $${\nabla }_{{\theta }_{N_{d_i}}}{\displaystyle \frac{1}{N_{d_i}}}{\displaystyle \underset{i=1}{\overset{N_{d_i}}{\sum }}}\times$$ $$[\log S(d_i\mid D;I)+\log (1-S(G\left(y_i\right)\mid D;I))]$$

In GAN-DP, there is a min-max game established between the generator, discriminator, and DPI. Based on the above formulas, this problem can be modeled as

(9) $$\underset{G}{\min }\underset{S}{\max }{E}_{x\sim \delta \left(d_i\right)}\left[\log S(d_i\mid D;I)\right]+$$ $$E_{y\sim p_g(y)}\left[\log (1-S(G\left(y_i\right)\mid D;I))\right]$$

In Formula ( 9 ), we use $\underset{G}{\min }$ to reduce the likelihood of discrimination and $\underset{S}{\max }$ to increase the likelihood of deception.

The PFL for heterogeneous residential load forecasting is shown in Algorithm 1.