Memory Error

[FortunateStarstar86]

Memory Error

[PingChang818]

The data preprocessing stage has nothing to do with GPU memory yet. You may want to change the parameter of number of threads in step 2 according to your CPU specification (mine is 20).

[FortunateStarstar86]

Thanks for the reply. I changed 20 to 14、8、2，still stucked, the memory usage is 100%【63.7/63.7g】,but no error reported.
I don't know the reason, is it about the pkl file too large？😟

[PingChang818]

Yes, and then you would need to change the code for all data to be processed segment by segment, in order to keep the memory from exploding.

[FortunateStarstar86]

def forecast(self, samples_x, samples_y, info, n_samples): generation = torch.zeros(n_samples, samples_y.shape[0], samples_y.shape[-1]).to(self.device) for i in range(n_samples): samples_y[:, 2] = torch.randn_like(samples_y[:, 2]) * samples_y[:, 3] #mask for t in range(self.num_steps - 1, -1, -1): mask_x = samples_x[:, 3] mask_y = samples_y[:, 3] samples_x[:, 0] = torch.where(mask_x == 1, samples_x[:, 0], self.lv) samples_x[:, 1] = torch.where(mask_x == 1, samples_x[:, 1], -1) samples_y[:, 0] = torch.where(mask_y == 1, samples_y[:, 0], self.lv) samples_y[:, 1] = torch.where(mask_y == 1, samples_y[:, 1], -1) predicted = self.res_model(samples_x, samples_y, info, torch.tensor([t]).to(self.device)) coeff1 = 1 / self.alpha_hat[t] ** 0.5 coeff2 = (1 - self.alpha_hat[t]) / (1 - self.alpha[t]) ** 0.5 samples_y[:, 2] = coeff1 * (samples_y[:, 2] - coeff2 * predicted) * samples_y[:, 3] if t > 0: noise = torch.randn_like(samples_y[:, 2]) * samples_y[:, 3] sigma = ((1.0 - self.alpha[t - 1]) / (1.0 - self.alpha[t]) * self.beta[t]) ** 0.5 samples_y[:, 2] += sigma * noise generation[i] = samples_y[:, 2].detach()

Normalized predicted data are stored in generation[], how can I denormaliza the data to get the real results ？👀

HR、SBP and DBP, 3 target features are all included in the generation[]？, thier mean and std are diffrent with each other..
(；′⌒`)

[PingChang818]

real value = prediction * std + mean, where std and mean come from training set.