Comments (9)
HuiZeng
commented on June 9, 2024
Hi, this is weird.
Can you provide the training log so that I can see more details.
from image-adaptive-3dlut.
hkzhang95
commented on June 9, 2024
Only the evaluation result of each epoch was automatically logged into the file as follows:
[PSNR: 13.024651] [max PSNR: 13.024651, epoch: 0]
[PSNR: 15.862108] [max PSNR: 15.862108, epoch: 1]
[PSNR: 16.551597] [max PSNR: 16.551597, epoch: 2]
[PSNR: 17.034064] [max PSNR: 17.034064, epoch: 3]
[PSNR: 17.280534] [max PSNR: 17.280534, epoch: 4]
[PSNR: 17.015003] [max PSNR: 17.280534, epoch: 4]
[PSNR: 17.049055] [max PSNR: 17.280534, epoch: 4]
[PSNR: 16.576314] [max PSNR: 17.280534, epoch: 4]
[PSNR: 19.402531] [max PSNR: 19.402531, epoch: 8]
[PSNR: 18.187922] [max PSNR: 19.402531, epoch: 8]
[PSNR: 17.973623] [max PSNR: 19.402531, epoch: 8]
[PSNR: 17.398157] [max PSNR: 19.402531, epoch: 8]
[PSNR: 17.081280] [max PSNR: 19.402531, epoch: 8]
[PSNR: 20.217006] [max PSNR: 20.217006, epoch: 13]
[PSNR: 17.813052] [max PSNR: 20.217006, epoch: 13]
[PSNR: 19.693702] [max PSNR: 20.217006, epoch: 13]
[PSNR: 17.425721] [max PSNR: 20.217006, epoch: 13]
[PSNR: 20.432641] [max PSNR: 20.432641, epoch: 17]
[PSNR: 16.874766] [max PSNR: 20.432641, epoch: 17]
[PSNR: 18.723524] [max PSNR: 20.432641, epoch: 17]
[PSNR: 16.708122] [max PSNR: 20.432641, epoch: 17]
[PSNR: 14.556466] [max PSNR: 20.432641, epoch: 17]
[PSNR: 20.554481] [max PSNR: 20.554481, epoch: 22]
[PSNR: 19.499911] [max PSNR: 20.554481, epoch: 22]
[PSNR: 21.427115] [max PSNR: 21.427115, epoch: 24]
[PSNR: 18.385184] [max PSNR: 21.427115, epoch: 24]
[PSNR: 18.826073] [max PSNR: 21.427115, epoch: 24]
[PSNR: 19.206979] [max PSNR: 21.427115, epoch: 24]
[PSNR: 20.076136] [max PSNR: 21.427115, epoch: 24]
[PSNR: 19.784653] [max PSNR: 21.427115, epoch: 24]
[PSNR: 22.328354] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.246163] [max PSNR: 22.328354, epoch: 30]
[PSNR: 16.059745] [max PSNR: 22.328354, epoch: 30]
[PSNR: 16.430683] [max PSNR: 22.328354, epoch: 30]
[PSNR: 16.121500] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.025921] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.263769] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.452778] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.062120] [max PSNR: 22.328354, epoch: 30]
[PSNR: 16.947569] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.037626] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.034605] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.953418] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.194347] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.118746] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.405536] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.389394] [max PSNR: 22.328354, epoch: 30]
[PSNR: 15.672612] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.525883] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.027628] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.993712] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.662860] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.951025] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.452927] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.682631] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.337622] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.506282] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.394288] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.044646] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.093883] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.910223] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.916111] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.895379] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.788592] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.218885] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.431236] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.214837] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.459107] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.700800] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.507151] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.283327] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.066402] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.522686] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.033237] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.127934] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.044623] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.522969] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.866807] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.651115] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.118083] [max PSNR: 22.328354, epoch: 30]
[PSNR: 16.748753] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.347939] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.526505] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.650681] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.337028] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.537075] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.897517] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.826204] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.901154] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.223558] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.469346] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.178158] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.606856] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.548949] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.970423] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.775911] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.767456] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.784937] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.560459] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.102979] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.733206] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.659968] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.495032] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.476901] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.283543] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.370069] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.832772] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.747057] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.739513] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.664269] [max PSNR: 22.328354, epoch: 30]
[PSNR: 21.837813] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.508567] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.248248] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.439892] [max PSNR: 22.328354, epoch: 30]
[PSNR: 19.935328] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.388210] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.458763] [max PSNR: 22.328354, epoch: 30]
[PSNR: 17.634312] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.970842] [max PSNR: 22.328354, epoch: 30]
[PSNR: 18.963656] [max PSNR: 22.328354, epoch: 30]
[PSNR: 20.002165] [max PSNR: 22.328354, epoch: 30]
[PSNR: 23.107576] [max PSNR: 23.107576, epoch: 121]
[PSNR: 21.135807] [max PSNR: 23.107576, epoch: 121]
[PSNR: 20.485475] [max PSNR: 23.107576, epoch: 121]
[PSNR: 18.292977] [max PSNR: 23.107576, epoch: 121]
[PSNR: 15.257979] [max PSNR: 23.107576, epoch: 121]
[PSNR: 19.438947] [max PSNR: 23.107576, epoch: 121]
[PSNR: 17.423877] [max PSNR: 23.107576, epoch: 121]
[PSNR: 19.194832] [max PSNR: 23.107576, epoch: 121]
[PSNR: 17.156946] [max PSNR: 23.107576, epoch: 121]
[PSNR: 18.661156] [max PSNR: 23.107576, epoch: 121]
...
If you need more, I have to modify the code to log the printed information during the iterations into the file and comment later.
from image-adaptive-3dlut.
HuiZeng
commented on June 9, 2024
Can you copy the training log in command window?
from image-adaptive-3dlut.
hkzhang95
commented on June 9, 2024
Unfortunately, the experiment was conducted last week and most of the log is cleared away. Here are some recent logs(not the baseline_0, a little messed in tmux):
[PSNR: 21.024096] [max PSNR: 22.176984, epoch: 268]
[Epoch 397/400] [Batch 0/4500] [psnr: 7.665617, tv: 0.000041, wnorm: 12.969558, mn[Epoch 397/400] [Batch 1/4500] [psnr: 7.649276, tv: 0.000041, wnorm: 9.787478, mn:[
Epoch 397/400] [Batch 2/4500] [psnr: 12.411872, tv: 0.000041, wnorm: 17.314035, m[Epoch 397/400] [Batch 3/4500] [psnr: 11.234340, tv: 0.000041, wnorm: 10.183039, m[E
poch 397/400] [Batch 4/4500] [psnr: 13.208837, tv: 0.000041, wnorm: 20.633810, m[Epoch 397/400] [Batch 5/4500] [psnr: 1.168683, tv: 0.000041, wnorm: 19.835659, mn[Ep
och 397/400] [Batch 6/4500] [psnr: 2.368635, tv: 0.000041, wnorm: 18.089636, mn[Epoch 397/400] [Batch 7/4500] [psnr: 2.786110, tv: 0.000041, wnorm: 8.863331, mn:[Epo
ch 397/400] [Batch 8/4500] [psnr: 4.055999, tv: 0.000041, wnorm: 9.777662, mn:[Epoch 397/400] [Batch 9/4500] [psnr: 5.094710, tv: 0.000041, wnorm: 5.088161, mn:[Epoc
h 397/400] [Batch 10/4500] [psnr: 5.748492, tv: 0.000041, wnorm: 8.375555, mn[Epoch 397/400] [Batch 11/4500] [psnr: 6.261108, tv: 0.000041, wnorm: 6.350131, mn[Epoch
397/400] [Batch 12/4500] [psnr: 6.778671, tv: 0.000041, wnorm: 13.861743, m[Epoch 397/400] [Batch 13/4500] [psnr: 7.057873, tv: 0.000041, wnorm: 6.106193, mn[Epoch
397/400] [Batch 14/4500] [psnr: 7.197341, tv: 0.000041, wnorm: 6.146301, mn[Epoch 397/400] [Batch 15/4500] [psnr: 7.583547, tv: 0.000041, wnorm: 7.032446, mn[Epoch 3
97/400] [Batch 16/4500] [psnr: 7.816548, tv: 0.000041, wnorm: 8.771721, mn[Epoch 397/400] [Batch 17/4500] [psnr: 7.849949, tv: 0.000041, wnorm: 13.493182, m[Epoch 39
7/400] [Batch 18/4500] [psnr: 7.930104, tv: 0.000041, wnorm: 9.542694, mn[Epoch 397/400] [Batch 19/4500] [psnr: 7.903939, tv: 0.000041, wnorm: 11.381426, m[Epoch 397
/400] [Batch 20/4500] [psnr: 7.904896, tv: 0.000041, wnorm: 16.691589, m[Epoch 397/400] [Batch 21/4500] [psnr: 8.041148, tv: 0.000041, wnorm: 10.820745, m[Epoch 397/
400] [Batch 22/4500] [psnr: 7.973785, tv: 0.000041, wnorm: 14.625758, m[Epoch 397/400] [Batch 23/4500] [psnr: 7.975224, tv: 0.000041, wnorm: 12.936926, m[Epoch 397/4
00] [Batch 24/4500] [psnr: 8.049949, tv: 0.000041, wnorm: 4.909339, mn[Epoch 397/400] [Batch 25/4500] [psnr: 8.073010, tv: 0.000041, wnorm: 10.975361, m[Epoch 397/40
0] [Batch 26/4500] [psnr: 8.171595, tv: 0.000041, wnorm: 18.058958, m[Epoch 397/400] [Batch 27/4500] [psnr: 8.214958, tv: 0.000041, wnorm: 10.123698, m[Epoch 397/400
] [Batch 28/4500] [psnr: 8.186530, tv: 0.000041, wnorm: 15.152336, m[Epoch 397/400] [Batch 29/4500] [psnr: 8.163373, tv: 0.000041, wnorm: 21.057343, m[Epoch 397/400]
[Batch 30/4500] [psnr: 8.070497, tv: 0.000041, wnorm: 13.038300, m[Epoch 397/400] [Batch 31/4500] [psnr: 8.198836, tv: 0.000041, wnorm: 6.020540, mn[Epoch 397/400]
[Batch 32/4500] [psnr: 8.132882, tv: 0.000041, wnorm: 7.469786, mn[Epoch 397/400] [Batch 33/4500] [psnr: 8.277606, tv: 0.000041, wnorm: 7.211593, mn[Epoch 397/400] [
Batch 34/4500] [psnr: 8.352780, tv: 0.000041, wnorm: 17.872486, m[Epoch 397/400] [Batch 35/4500] [psnr: 8.436387, tv: 0.000041, wnorm: 8.160758, mn[Epoch 397/400] [B
atch 36/4500] [psnr: 8.481443, tv: 0.000041, wnorm: 3.937635, mn[Epoch 397/400] [Batch 37/4500] [psnr: 8.522345, tv: 0.000041, wnorm: 8.029694, mn[Epoch 397/400] [Ba
tch 38/4500] [psnr: 8.572394, tv: 0.000041, wnorm: 8.407439, mn[Epoch 397/400] [Batch 39/4500] [psnr: 8.628732, tv: 0.000041, wnorm: 15.459891, m[Epoch 397/400] [Bat
ch 40/4500] [psnr: 8.698949, tv: 0.000041, wnorm: 7.809579, mn[Epoch 397/400] [Batch 41/4500] [psnr: 8.749084, tv: 0.000041, wnorm: 12.786201, m[Epoch 397/400] [Batc
h 42/4500] [psnr: 8.774793, tv: 0.000041, wnorm: 7.421274, mn[Epoch 397/400] [Batch 43/4500] [psnr: 8.855392, tv: 0.000041, wnorm: 14.250587, m[Epoch 397/400] [Batch
44/4500] [psnr: 8.831794, tv: 0.000041, wnorm: 20.826065, m[Epoch 397/400] [Batch 45/4500] [psnr: 8.817753, tv: 0.000041, wnorm: 21.786612, m[Epoch 397/400] [Batch
46/4500] [psnr: 8.861986, tv: 0.000041, wnorm: 16.656706, m[Epoch 397/400] [Batch 47/4500] [psnr: 8.831289, tv: 0.000041, wnorm: 20.881237, m[Epoch 397/400] [Batch 4
8/4500] [psnr: 8.876238, tv: 0.000041, wnorm: 8.556711, mn[Epoch 397/400] [Batch 49/4500] [psnr: 8.919388, tv: 0.000041, wnorm: 13.717594, m[Epoch 397/400] [Batch 50
/4500] [psnr: 9.001214, tv: 0.000041, wnorm: 12.229441, m[Epoch 397/400] [Batch 51/4500] [psnr: 9.057196, tv: 0.000041, wnorm: 6.626611, mn[Epoch 397/400] [Batch 52/
4500] [psnr: 9.035701, tv: 0.000041, wnorm: 13.314039, m[Epoch 397/400] [Batch 53/4500] [psnr: 9.062985, tv: 0.000041, wnorm: 8.531548, mn[Epoch 397/400] [Batch 54/4
500] [psnr: 9.073870, tv: 0.000041, wnorm: 16.118582, m[Epoch 397/400] [Batch 55/4500] [psnr: 9.095042, tv: 0.000041, wnorm: 15.153574, m[Epoch 397/400] [Batch 56/45
00] [psnr: 9.160988, tv: 0.000041, wnorm: 9.626263, mn[Epoch 397/400] [Batch 57/4500] [psnr: 9.157246, tv: 0.000041, wnorm: 7.792549, mn[Epoch 397/400] [Batch 58/450
0] [psnr: 9.189761, tv: 0.000041, wnorm: 11.553389, m[Epoch 397/400] [Batch 59/4500] [psnr: 9.221601, tv: 0.000041, wnorm: 14.208018, m[Epoch 397/400] [Batch 60/4500
] [psnr: 9.277210, tv: 0.000041, wnorm: 7.484590, mn[Epoch 397/400] [Batch 61/4500] [psnr: 9.296562, tv: 0.000041, wnorm: 11.689133, m[Epoch 397/400] [Batch 62/4500]
[psnr: 9.355585, tv: 0.000041, wnorm: 4.735834, mn[Epoch 397/400] [Batch 63/4500] [psnr: 9.374899, tv: 0.000041, wnorm: 12.560919, m[Epoch 397/400] [Batch 64/4500]
[psnr: 9.404433, tv: 0.000041, wnorm: 14.708694, m[Epoch 397/400] [Batch 65/4500] [psnr: 9.422749, tv: 0.000041, wnorm: 19.920300, m[Epoch 397/400] [Batch 66/4500] [
psnr: 9.454526, tv: 0.000041, wnorm: 16.677513, m[Epoch 397/400] [Batch 67/4500] [psnr: 9.534675, tv: 0.000041, wnorm: 21.085806, m[Epoch 397/400] [Batch 68/4500] [p
snr: 9.601607, tv: 0.000041, wnorm: 10.718325, m[Epoch 397/400] [Batch 69/4500] [psnr: 9.616656, tv: 0.000041, wnorm: 16.337803, m[Epoch 397/400] [Batch 70/4500] [ps
nr: 9.659377, tv: 0.000041, wnorm: 11.646683, m[Epoch 397/400] [Batch 71/4500] [psnr: 9.656486, tv: 0.000041, wnorm: 21.594137, m[Epoch 397/400] [Batch 72/4500] [psn
r: 9.626568, tv: 0.000041, wnorm: 18.755720, m[Epoch 397/400] [Batch 73/4500] [psnr: 9.639391, tv: 0.000041, wnorm: 7.496913, mn[Epoch 397/400] [Batch 74/4500] [psnr
: 9.620467, tv: 0.000041, wnorm: 12.687343, m[Epoch 397/400] [Batch 75/4500] [psnr: 9.707358, tv: 0.000041, wnorm: 14.409729, m[Epoch 397/400] [Batch 76/4500] [psnr:
9.744850, tv: 0.000041, wnorm: 14.818538, m[Epoch 397/400] [Batch 77/4500] [psnr: 9.773979, tv: 0.000041, wnorm: 6.790950, mn[Epoch 397/400] [Batch 78/4500] [psnr:
9.813790, tv: 0.000041, wnorm: 17.636044, m[Epoch 397/400] [Batch 79/4500] [psnr: 9.838610, tv: 0.000041, wnorm: 12.757740, m[Epoch 397/400] [Batch 80/4500] [psnr: 9
.888091, tv: 0.000041, wnorm: 9.899735, mn[Epoch 397/400] [Batch 81/4500] [psnr: 9.915395, tv: 0.000041, wnorm: 9.673935, mn[Epoch 397/400] [Batch 82/4500] [psnr: 9.
962446, tv: 0.000041, wnorm: 13.399908, m[Epoch 397/400] [Batch 83/4500] [psnr: 9.987729, tv: 0.000041, wnorm: 11.860794, m[Epoch 397/400] [Batch 84/4500] [psnr: 10.
050310, tv: 0.000041, wnorm: 7.814270, m[Epoch 397/400] [Batch 85/4500] [psnr: 10.063048, tv: 0.000041, wnorm: 7.225454, m[Epoch 397/400] [Batch 86/4500] [psnr: 10.1
18764, tv: 0.000041, wnorm: 8.022103, m[Epoch 397/400] [Batch 87/4500] [psnr: 9.841078, tv: 0.000041, wnorm: 11.488071, m[Epoch 397/400] [Batch 88/4500] [psnr: 9.870
503, tv: 0.000041, wnorm: 9.206477, mn[Epoch 397/400] [Batch 89/4500] [psnr: 9.931853, tv: 0.000041, wnorm: 9.366801, mn[Epoch 397/400] [Batch 90/4500] [psnr: 9.9338
36, tv: 0.000041, wnorm: 18.848469, m[Epoch 397/400] [Batch 91/4500] [psnr: 9.968377, tv: 0.000041, wnorm: 15.802090, m[Epoch 397/400] [Batch 92/4500] [psnr: 10.0139
47, tv: 0.000041, wnorm: 10.169043, [Epoch 397/400] [Batch 93/4500] [psnr: 10.067529, tv: 0.000041, wnorm: 19.104902, [Epoch 397/400] [Batch 94/4500] [psnr: 10.10118
7, tv: 0.000041, wnorm: 25.650671, [Epoch 397/400] [Batch 95/4500] [psnr: 10.151808, tv: 0.000041, wnorm: 13.330386, [Epoch 397/400] [Batch 96/4500] [psnr: 10.176736
, tv: 0.000041, wnorm: 18.241131, [Epoch 397/400] [Batch 97/4500] [psnr: 10.240372, tv: 0.000041, wnorm: 24.600332, [Epoch 397/400] [Batch 98/4500] [psnr: 10.299246,
tv: 0.000041, wnorm: 16.987484, [Epoch 397/400] [Batch 99/4500] [psnr: 10.275846, tv: 0.000041, wnorm: 17.395145, [Epoch 397/400] [Batch 100/4500] [psnr: 10.339614,
tv: 0.000041, wnorm: 16.784666,[Epoch 397/400] [Batch 101/4500] [psnr: 10.353459, tv: 0.000041, wnorm: 24.470762,[Epoch 397/400] [Batch 102/4500] [psnr: 10.414776,
tv: 0.000041, wnorm: 16.173203,[Epoch 397/400] [Batch 103/4500] [psnr: 10.453341, tv: 0.000041, wnorm: 19.981476,[Epoch 397/400] [Batch 104/4500] [psnr: 10.427253, t
v: 0.000041, wnorm: 21.928307,[Epoch 397/400] [Batch 105/4500] [psnr: 10.442645, tv: 0.000041, wnorm: 14.268560,[Epoch 397/400] [Batch 106/4500] [psnr: 10.488598, tv
: 0.000041, wnorm: 9.648129, [Epoch 397/400] [Batch 107/4500] [psnr: 10.504957, tv: 0.000041, wnorm: 19.872574,[Epoch 397/400] [Batch 108/4500] [psnr: 10.495929, tv:
0.000041, wnorm: 12.112583,[Epoch 397/400] [Batch 109/4500] [psnr: 10.498427, tv: 0.000041, wnorm: 15.864108,[Epoch 397/400] [Batch 110/4500] [psnr: 10.481000, tv:
0.000041, wnorm: 14.371432,[Epoch 397/400] [Batch 111/4500] [psnr: 10.524941, tv: 0.000041, wnorm: 21.098476,[Epoch 397/400] [Batch 112/4500] [psnr: 10.585391, tv: 0
.000041, wnorm: 16.114401,[Epoch 397/400] [Batch 113/4500] [psnr: 10.706692, tv: 0.000041, wnorm: 15.885933,[Epoch 397/400] [Batch 114/4500] [psnr: 10.730267, tv: 0.
000041, wnorm: 14.237982,[Epoch 397/400] [Batch 115/4500] [psnr: 10.773127, tv: 0.000041, wnorm: 16.456051,[Epoch 397/400] [Batch 116/4500] [psnr: 10.832099, tv: 0.0
00041, wnorm: 10.099601,[Epoch 397/400] [Batch 117/4500] [psnr: 10.771185, tv: 0.000041, wnorm: 9.972964, [Epoch 397/400] [Batch 118/4500] [psnr: 10.886664, tv: 0.00
0041, wnorm: 14.210288,[Epoch 397/400] [Batch 119/4500] [psnr: 10.914074, tv: 0.000041, wnorm: 8.628948, [Epoch 397/400] [Batch 120/4500] [psnr: 10.934752, tv: 0.000
041, wnorm: 12.512462,[Epoch 397/400] [Batch 121/4500] [psnr: 10.935435, tv: 0.000041, wnorm: 16.630299,[Epoch 397/400] [Batch 122/4500] [psnr: 10.962891, tv: 0.0000
41, wnorm: 17.509464,[Epoch 397/400] [Batch 123/4500] [psnr: 10.972371, tv: 0.000041, wnorm: 11.465030,[Epoch 397/400] [Batch 124/4500] [psnr: 10.971020, tv: 0.00004
1, wnorm: 14.460867,[Epoch 397/400] [Batch 125/4500] [psnr: 11.042186, tv: 0.000041, wnorm: 16.473225,[Epoch 397/400] [Batch 126/4500] [psnr: 11.037900, tv: 0.000041
, wnorm: 17.409645,[Epoch 397/400] [Batch 127/4500] [psnr: 11.037775, tv: 0.000041, wnorm: 12.781590,[Epoch 397/400] [Batch 128/4500] [psnr: 11.062324, tv: 0.000041,
wnorm: 13.714447,[Epoch 397/400] [Batch 129/4500] [psnr: 11.098305, tv: 0.000041, wnorm: 13.681095,[Epoch 397/400] [Batch 130/4500] [psnr: 11.179373, tv: 0.000041,
wnorm: 19.258726,[Epoch 397/400] [Batch 131/4500] [psnr: 11.191704, tv: 0.000041, wnorm: 18.257055,[Epoch 397/400] [Batch 132/4500] [psnr: 11.186326, tv: 0.000041, w
norm: 13.574450,[Epoch 397/400] [Batch 133/4500] [psnr: 11.283364, tv: 0.000041, wnorm: 21.516388,[Epoch 397/400] [Batch 134/4500] [psnr: 11.344401, tv: 0.000041, wn
orm: 17.321621,[Epoch 397/400] [Batch 135/4500] [psnr: 11.444050, tv: 0.000041, wnorm: 19.458408,[Epoch 397/400] [Batch 136/4500] [psnr: 11.424252, tv: 0.000041, wno
rm: 16.181046,[Epoch 397/400] [Batch 137/4500] [psnr: 11.478426, tv: 0.000041, wnorm: 13.830401,[Epoch 397/400] [Batch 138/4500] [psnr: 11.559789, tv: 0.000041, wnor
m: 23.426380,[Epoch 397/400] [Batch 139/4500] [psnr: 11.582889, tv: 0.000041, wnorm: 10.484896,[Epoch 397/400] [Batch 140/4500] [psnr: 11.609100, tv: 0.000041, wnorm
: 23.133141,[Epoch 397/400] [Batch 141/4500] [psnr: 11.680590, tv: 0.000041, wnorm: 18.982960,[Epoch 397/400] [Batch 142/4500] [psnr: 11.737036, tv: 0.000041, wnorm:
21.379736,[Epoch 397/400] [Batch 143/4500] [psnr: 11.796875, tv: 0.000041, wnorm: 16.730297,[Epoch 397/400] [Batch 144/4500] [psnr: 11.827318, tv: 0.000041, wnorm:
13.031458,[Epoch 397/400] [Batch 145/4500] [psnr: 11.830605, tv: 0.000041, wnorm: 7.581544, [Epoch 397/400] [Batch 146/4500] [psnr: 11.916550, tv: 0.000041, wnorm: 2
3.063812,[Epoch 397/400] [Batch 147/4500] [psnr: 11.939693, tv: 0.000041, wnorm: 16.216505,[Epoch 397/400] [Batch 148/4500] [psnr: 11.969823, tv: 0.000041, wnorm: 11
.465057,[Epoch 397/400] [Batch 149/4500] [psnr: 12.000636, tv: 0.000041, wnorm: 11.605320,[Epoch 397/400] [Batch 150/4500] [psnr: 12.016580, tv: 0.000041, wnorm: 16.
945423,[Epoch 397/400] [Batch 151/4500] [psnr: 12.025816, tv: 0.000041, wnorm: 17.927998,[Epoch 397/400] [Batch 152/4500] [psnr: 12.019142, tv: 0.000041, wnorm: 14.6
33986,[Epoch 397/400] [Batch 153/4500] [psnr: 11.982463, tv: 0.000041, wnorm: 8.584014, [Epoch 397/400] [Batch 154/4500] [psnr: 11.973023, tv: 0.000041, wnorm: 26.84
6418,[Epoch 397/400] [Batch 155/4500] [psnr: 12.001305, tv: 0.000041, wnorm: 15.020472,[Epoch 397/400] [Batch 156/4500] [psnr: 12.036859, tv: 0.000041, wnorm: 15.730
929,[Epoch 397/400] [Batch 157/4500] [psnr: 12.041844, tv: 0.000041, wnorm: 8.466520, [Epoch 397/400] [Batch 158/4500] [psnr: 12.049036, tv: 0.000041, wnorm: 13.8489
36,[Epoch 397/400] [Batch 159/4500] [psnr: 12.087392, tv: 0.000041, wnorm: 15.735527,[Epoch 397/400] [Batch 160/4500] [psnr: 12.090212, tv: 0.000041, wnorm: 15.15536
1,[Epoch 397/400] [Batch 161/4500] [psnr: 12.122702, tv: 0.000041, wnorm: 9.918506, [Epoch 397/400] [Batch 162/4500] [psnr: 12.117361, tv: 0.000041, wnorm: 18.665501
,[Epoch 397/400] [Batch 163/4500] [psnr: 12.144146, tv: 0.000041, wnorm: 13.792322,[Epoch 397/400] [Batch 164/4500] [psnr: 12.141682, tv: 0.000041, wnorm: 21.227892,
[Epoch 397/400] [Batch 165/4500] [psnr: 12.149450, tv: 0.000041, wnorm: 20.486576,[Epoch 397/400] [Batch 166/4500] [psnr: 12.203664, tv: 0.000041, wnorm: 19.321556,[
Epoch 397/400] [Batch 167/4500] [psnr: 12.226527, tv: 0.000041, wnorm: 10.483467,[Epoch 397/400] [Batch 168/4500] [psnr: 12.230875, tv: 0.000041, wnorm: 16.386330,[E
poch 397/400] [Batch 169/4500] [psnr: 12.276515, tv: 0.000041, wnorm: 22.319822,[Epoch 397/400] [Batch 170/4500] [psnr: 12.304491, tv: 0.000041, wnorm: 11.802592,[Ep
och 397/400] [Batch 171/4500] [psnr: 12.318371, tv: 0.000041, wnorm: 9.538180, [Epoch 397/400] [Batch 172/4500] [psnr: 12.377411, tv: 0.000041, wnorm: 18.860796,[Epo
ch 397/400] [Batch 173/4500] [psnr: 12.362942, tv: 0.000041, wnorm: 11.326637,[Epoch 397/400] [Batch 174/4500] [psnr: 12.387124, tv: 0.000041, wnorm: 25.639343,[Epoc
h 397/400] [Batch 175/4500] [psnr: 12.425894, tv: 0.000041, wnorm: 18.937248,[Epoch 397/400] [Batch 176/4500] [psnr: 12.405849, tv: 0.000041, wnorm: 18.646120,[Epoch
397/400] [Batch 177/4500] [psnr: 12.412427, tv: 0.000041, wnorm: 18.207243, [Epoch 397/400] [Batch 178/4500] [psnr: 12.362272, tv: 0.000041, wnorm: 30.910599,[Epoch
397/400] [Batch 179/4500] [psnr: 12.405912, tv: 0.000041, wnorm: 14.351250,[Epoch 397/400] [Batch 180/4500] [psnr: 12.385636, tv: 0.000041, wnorm: 10.323959,[Epoch 3
97/400] [Batch 181/4500] [psnr: 12.423840, tv: 0.000041, wnorm: 20.529549,[Epoch 397/400] [Batch 182/4500] [psnr: 12.429792, tv: 0.000041, wnorm: 22.119268,[Epoch 39
7/400] [Batch 183/4500] [psnr: 12.435167, tv: 0.000041, wnorm: 9.638505, [Epoch 397/400] [Batch 184/4500] [psnr: 12.390193, tv: 0.000041, wnorm: 16.101397,[Epoch 397
/400] [Batch 185/4500] [psnr: 12.393475, tv: 0.000041, wnorm: 13.214029,[Epoch 397/400] [Batch 186/4500] [psnr: 12.416774, tv: 0.000041, wnorm: 19.605091,[Epoch 397/
400] [Batch 187/4500] [psnr: 12.434418, tv: 0.000041, wnorm: 8.372734, [Epoch 397/400] [Batch 188/4500] [psnr: 12.435624, tv: 0.000041, wnorm: 13.418395,[Epoch 397/4
00] [Batch 189/4500] [psnr: 12.439954, tv: 0.000041, wnorm: 15.847446,[Epoch 397/400] [Batch 190/4500] [psnr: 12.465614, tv: 0.000041, wnorm: 11.212442,[Epoch 397/40
0] [Batch 191/4500] [psnr: 12.469553, tv: 0.000041, wnorm: 11.971138,[Epoch 397/400] [Batch 192/4500] [psnr: 12.511465, tv: 0.000041, wnorm: 15.146491,[Epoch 397/400
] [Batch 193/4500] [psnr: 12.533478, tv: 0.000041, wnorm: 14.408431,[Epoch 397/400] [Batch 194/4500] [psnr: 12.559102, tv: 0.000041, wnorm: 18.582952,[Epoch 397/400]
[Batch 195/4500] [psnr: 12.605914, tv: 0.000041, wnorm: 12.923022,[Epoch 397/400] [Batch 196/4500] [psnr: 12.609720, tv: 0.000041, wnorm: 19.639206,[Epoch 397/400]
[Batch 197/4500] [psnr: 12.589039, tv: 0.000041, wnorm: 22.962660,[Epoch 397/400] [Batch 198/4500] [psnr: 12.632129, tv: 0.000041, wnorm: 25.862808,[Epoch 397/400] [
Batch 199/4500] [psnr: 12.694962, tv: 0.000041, wnorm: 23.427755,[Epoch 397/400] [Batch 200/4500] [psnr: 12.686179, tv: 0.000041, wnorm: 23.956572,[Epoch 397/400] [B
atch 201/4500] [psnr: 12.722173, tv: 0.000041, wnorm: 15.876011,[Epoch 397/400] [Batch 202/4500] [psnr: 12.729199, tv: 0.000041, wnorm: 14.592208,[Epoch 397/400] [Ba
tch 203/4500] [psnr: 12.731857, tv: 0.000041, wnorm: 15.626945,[Epoch 397/400] [Batch 204/4500] [psnr: 12.705456, tv: 0.000041, wnorm: 17.552233,[Epoch 397/400] [Bat
ch 205/4500] [psnr: 12.705355, tv: 0.000041, wnorm: 12.715961,[Epoch 397/400] [Batch 206/4500] [psnr: 12.675564, tv: 0.000041, wnorm: 16.830984,[Epoch 397/400] [Batc
h 207/4500] [psnr: 12.683896, tv: 0.000041, wnorm: 11.929790,[Epoch 397/400] [Batch 208/4500] [psnr: 12.731402, tv: 0.000041, wnorm: 17.395096,[Epoch 397/400] [Batch
209/4500] [psnr: 12.721115, tv: 0.000041, wnorm: 23.161232,[Epoch 397/400] [Batch 210/4500] [psnr: 12.748116, tv: 0.000041, wnorm: 27.835873,[Epoch 397/400] [Batch
211/4500] [psnr: 12.725086, tv: 0.000041, wnorm: 10.491733,[Epoch 397/400] [Batch 212/4500] [psnr: 12.739573, tv: 0.000041, wnorm: 9.872424, [Epoch 397/400] [Batch 2
13/4500] [psnr: 12.718847, tv: 0.000041, wnorm: 15.372284,[Epoch 397/400] [Batch 214/4500] [psnr: 12.740006, tv: 0.000041, wnorm: 14.021896,[Epoch 397/400] [Batch 21
5/4500] [psnr: 12.728060, tv: 0.000041, wnorm: 11.204655,[Epoch 397/400] [Batch 216/4500] [psnr: 12.722014, tv: 0.000041, wnorm: 7.631651, [Epoch 397/400] [Batch 217
/4500] [psnr: 12.735264, tv: 0.000041, wnorm: 22.374599,[Epoch 397/400] [Batch 218/4500] [psnr: 12.759862, tv: 0.000041, wnorm: 12.098412,[Epoch 397/400] [Batch 219/
4500] [psnr: 12.753608, tv: 0.000041, wnorm: 16.623009,[Epoch 397/400] [Batch 220/4500] [psnr: 12.742892, tv: 0.000041, wnorm: 12.703166,[Epoch 397/400] [Batch 221/4
500] [psnr: 12.756233, tv: 0.000041, wnorm: 22.068161,[Epoch 397/400] [Batch 222/4500] [psnr: 12.746337, tv: 0.000041, wnorm: 9.317842, [Epoch 397/400] [Batch 223/45
00] [psnr: 12.768417, tv: 0.000041, wnorm: 15.253014,[Epoch 397/400] [Batch 224/4500] [psnr: 12.758974, tv: 0.000041, wnorm: 14.365180,[Epoch 397/400] [Batch 225/450
0] [psnr: 12.772966, tv: 0.000041, wnorm: 14.064993,[Epoch 397/400] [Batch 226/4500] [psnr: 12.773640, tv: 0.000041, wnorm: 19.480755,[Epoch 397/400] [Batch 227/4500
] [psnr: 12.778570, tv: 0.000041, wnorm: 16.854555,[Epoch 397/400] [Batch 228/4500] [psnr: 12.777434, tv: 0.000041, wnorm: 22.665142,[Epoch 397/400] [Batch 229/4500]
[psnr: 12.770323, tv: 0.000041, wnorm: 10.848636,[Epoch 397/400] [Batch 230/4500] [psnr: 12.778416, tv: 0.000041, wnorm: 14.310637,[Epoch 397/400] [Batch 231/4500]
[psnr: 12.778725, tv: 0.000041, wnorm: 16.481733,[Epoch 397/400] [Batch 232/4500] [psnr: 12.756402, tv: 0.000041, wnorm: 20.985716,[Epoch 397/400] [Batch 233/4500] [
psnr: 12.751744, tv: 0.000041, wnorm: 15.320323,[Epoch 397/400] [Batch 234/4500] [psnr: 12.761698, tv: 0.000041, wnorm: 16.398132,[Epoch 397/400] [Batch 235/4500] [p
snr: 12.761783, tv: 0.000041, wnorm: 17.206656,[Epoch 397/400] [Batch 236/4500] [psnr: 12.750005, tv: 0.000041, wnorm: 13.988460,[Epoch 397/400] [Batch 237/4500] [ps
nr: 12.768505, tv: 0.000041, wnorm: 17.732670,[Epoch 397/400] [Batch 238/4500] [psnr: 12.786333, tv: 0.000041, wnorm: 14.095824,[Epoch 397/400] [Batch 239/4500] [psn
r: 12.790160, tv: 0.000041, wnorm: 17.406763,[Epoch 397/400] [Batch 240/4500] [psnr: 12.783321, tv: 0.000041, wnorm: 8.784737, [Epoch 397/400] [Batch 241/4500] [psnr
: 12.772394, tv: 0.000041, wnorm: 13.910925,[Epoch 397/400] [Batch 242/4500] [psnr: 12.769479, tv: 0.000041, wnorm: 9.294963, [Epoch 397/400] [Batch 243/4500] [psnr:
12.787131, tv: 0.000041, wnorm: 9.127213, [Epoch 397/400] [Batch 244/4500] [psnr: 12.785637, tv: 0.000041, wnorm: 16.487518,[Epoch 397/400] [Batch 245/4500] [psnr:
12.783176, tv: 0.000041, wnorm: 11.359836,[Epoch 397/400] [Batch 246/4500] [psnr: 12.762864, tv: 0.000041, wnorm: 10.191836,[Epoch 397/400] [Batch 247/4500] [psnr: 1
2.763574, tv: 0.000041, wnorm: 14.214747,[Epoch 397/400] [Batch 248/4500] [psnr: 12.789700, tv: 0.000041, wnorm: 18.224504,[Epoch 397/400] [Batch 249/4500] [psnr: 12
.809088, tv: 0.000041, wnorm: 17.422081,[Epoch 397/400] [Batch 250/4500] [psnr: 12.848433, tv: 0.000041, wnorm: 16.253834,[Epoch 397/400] [Batch 251/4500] [psnr: 12.
849790, tv: 0.000041, wnorm: 17.023808,[Epoch 397/400] [Batch 252/4500] [psnr: 12.835462, tv: 0.000041, wnorm: 14.862154,[Epoch 397/400] [Batch 253/4500] [psnr: 12.8
32857, tv: 0.000041, wnorm: 12.447542,[Epoch 397/400] [Batch 254/4500] [psnr: 12.800949, tv: 0.000041, wnorm: 24.353672,[Epoch 397/400] [Batch 255/4500] [psnr: 12.79
4062, tv: 0.000041, wnorm: 13.662555,[Epoch 397/400] [Batch 256/4500] [psnr: 12.785375, tv: 0.000041, wnorm: 12.628111,[Epoch 397/400] [Batch 257/4500] [psnr: 12.774
189, tv: 0.000041, wnorm: 13.121723,[Epoch 397/400] [Batch 258/4500] [psnr: 12.784222, tv: 0.000041, wnorm: 10.134653,[Epoch 397/400] [Batch 259/4500] [psnr: 12.7715
68, tv: 0.000041, wnorm: 18.599812,[Epoch 397/400] [Batch 260/4500] [psnr: 12.775996, tv: 0.000041, wnorm: 17.328573,[Epoch 397/400] [Batch 261/4500] [psnr: 12.78822
3, tv: 0.000041, wnorm: 17.932533,[Epoch 397/400] [Batch 262/4500] [psnr: 12.793996, tv: 0.000041, wnorm: 19.845177,[Epoch 397/400] [Batch 263/4500] [psnr: 12.784150
, tv: 0.000041, wnorm: 15.656282,[Epoch 397/400] [Batch 264/4500] [psnr: 12.791818, tv: 0.000041, wnorm: 16.392004,[Epoch 397/400] [Batch 265/4500] [psnr: 12.774738,
tv: 0.000041, wnorm: 12.135654,[Epoch 397/400] [Batch 266/4500] [psnr: 12.758027, tv: 0.000041, wnorm: 6.136451,
Please inform me if you need more logs.
from image-adaptive-3dlut.
HuiZeng
commented on June 9, 2024
The results are quite abnormal. The tv regularization is very small which means the learned LUTs are almost flat.
And I am not sure of the problem.
from image-adaptive-3dlut.
hkzhang95
commented on June 9, 2024
I find that training is not stable and I do not know whether the reason is randomness. I get such results as follows just now, the training PSNR decreases from 8 to -66 in the first 2 epochs.:
2020-11-10 09:27:10,965 train INFO: [Epoch 2/400] [Batch 1734/4500] [psnr: -66.475265, tv: 0.005609, wnorm: 12975531.000000, mn: 0.041363] ETA: 15:05:25.072297
2020-11-10 09:27:10,997 train INFO: [Epoch 2/400] [Batch 1735/4500] [psnr: -66.475841, tv: 0.005610, wnorm: 13587256.000000, mn: 0.041366] ETA: 15:53:42.895106
2020-11-10 09:27:11,030 train INFO: [Epoch 2/400] [Batch 1736/4500] [psnr: -66.476129, tv: 0.005610, wnorm: 14353512.000000, mn: 0.041369] ETA: 16:37:03.805405
2020-11-10 09:27:11,059 train INFO: [Epoch 2/400] [Batch 1737/4500] [psnr: -66.477173, tv: 0.005611, wnorm: 13731207.000000, mn: 0.041372] ETA: 14:22:38.594463
2020-11-10 09:27:11,090 train INFO: [Epoch 2/400] [Batch 1738/4500] [psnr: -66.475591, tv: 0.005611, wnorm: 13951518.000000, mn: 0.041375] ETA: 15:19:09.982300
2020-11-10 09:27:11,121 train INFO: [Epoch 2/400] [Batch 1739/4500] [psnr: -66.476207, tv: 0.005612, wnorm: 14638966.000000, mn: 0.041378] ETA: 15:23:59.181575
2020-11-10 09:27:11,153 train INFO: [Epoch 2/400] [Batch 1740/4500] [psnr: -66.476734, tv: 0.005613, wnorm: 14317000.000000, mn: 0.041382] ETA: 15:45:53.056498
2020-11-10 09:27:11,186 train INFO: [Epoch 2/400] [Batch 1741/4500] [psnr: -66.477550, tv: 0.005613, wnorm: 14254380.000000, mn: 0.041385] ETA: 16:24:49.898978
2020-11-10 09:27:11,216 train INFO: [Epoch 2/400] [Batch 1742/4500] [psnr: -66.477421, tv: 0.005614, wnorm: 14009974.000000, mn: 0.041388] ETA: 14:58:46.392149
2020-11-10 09:27:11,248 train INFO: [Epoch 2/400] [Batch 1743/4500] [psnr: -66.477670, tv: 0.005614, wnorm: 14475339.000000, mn: 0.041390] ETA: 16:06:50.128294
2020-11-10 09:27:11,282 train INFO: [Epoch 2/400] [Batch 1744/4500] [psnr: -66.479107, tv: 0.005615, wnorm: 13924079.000000, mn: 0.041393] ETA: 16:53:00.810089
2020-11-10 09:27:11,311 train INFO: [Epoch 2/400] [Batch 1745/4500] [psnr: -66.479878, tv: 0.005616, wnorm: 14460758.000000, mn: 0.041399] ETA: 14:24:58.711692
2020-11-10 09:27:11,342 train INFO: [Epoch 2/400] [Batch 1746/4500] [psnr: -66.476873, tv: 0.005616, wnorm: 13220219.000000, mn: 0.041404] ETA: 15:12:59.880977
2020-11-10 09:27:11,373 train INFO: [Epoch 2/400] [Batch 1747/4500] [psnr: -66.476789, tv: 0.005617, wnorm: 13800442.000000, mn: 0.041409] ETA: 15:25:20.412611
2020-11-10 09:27:11,405 train INFO: [Epoch 2/400] [Batch 1748/4500] [psnr: -66.476983, tv: 0.005618, wnorm: 13888992.000000, mn: 0.041414] ETA: 15:45:25.927520
2020-11-10 09:27:11,438 train INFO: [Epoch 2/400] [Batch 1749/4500] [psnr: -66.477518, tv: 0.005619, wnorm: 13244171.000000, mn: 0.041418] ETA: 16:29:03.882826
2020-11-10 09:27:11,468 train INFO: [Epoch 2/400] [Batch 1750/4500] [psnr: -66.479966, tv: 0.005620, wnorm: 14453710.000000, mn: 0.041422] ETA: 15:00:37.917733
2020-11-10 09:27:11,501 train INFO: [Epoch 2/400] [Batch 1751/4500] [psnr: -66.479804, tv: 0.005620, wnorm: 13846912.000000, mn: 0.041427] ETA: 16:10:03.540876
2020-11-10 09:27:11,534 train INFO: [Epoch 2/400] [Batch 1752/4500] [psnr: -66.479726, tv: 0.005621, wnorm: 14591423.000000, mn: 0.041431] ETA: 16:40:38.271790
2020-11-10 09:27:11,563 train INFO: [Epoch 2/400] [Batch 1753/4500] [psnr: -66.478503, tv: 0.005622, wnorm: 14884492.000000, mn: 0.041435] ETA: 14:28:27.508612
2020-11-10 09:27:11,594 train INFO: [Epoch 2/400] [Batch 1754/4500] [psnr: -66.478010, tv: 0.005623, wnorm: 13539003.000000, mn: 0.041438] ETA: 15:17:59.528460
2020-11-10 09:27:11,625 train INFO: [Epoch 2/400] [Batch 1755/4500] [psnr: -66.480678, tv: 0.005623, wnorm: 14780592.000000, mn: 0.041441] ETA: 15:17:02.334716
2020-11-10 09:27:11,657 train INFO: [Epoch 2/400] [Batch 1756/4500] [psnr: -66.480658, tv: 0.005624, wnorm: 13367615.000000, mn: 0.041444] ETA: 15:51:05.665333
2020-11-10 09:27:11,690 train INFO: [Epoch 2/400] [Batch 1757/4500] [psnr: -66.479541, tv: 0.005624, wnorm: 13844442.000000, mn: 0.041447] ETA: 16:27:54.083521
2020-11-10 09:27:11,720 train INFO: [Epoch 2/400] [Batch 1758/4500] [psnr: -66.479929, tv: 0.005625, wnorm: 14087398.000000, mn: 0.041450] ETA: 14:59:46.912086
2020-11-10 09:27:11,753 train INFO: [Epoch 2/400] [Batch 1759/4500] [psnr: -66.478380, tv: 0.005625, wnorm: 14841076.000000, mn: 0.041452] ETA: 16:33:32.301784
2020-11-10 09:27:11,785 train INFO: [Epoch 2/400] [Batch 1760/4500] [psnr: -66.478203, tv: 0.005626, wnorm: 13629856.000000, mn: 0.041454] ETA: 15:39:58.780622
2020-11-10 09:27:11,819 train INFO: [Epoch 2/400] [Batch 1761/4500] [psnr: -66.477494, tv: 0.005626, wnorm: 14450051.000000, mn: 0.041456] ETA: 16:44:26.620866
2020-11-10 09:27:11,851 train INFO: [Epoch 2/400] [Batch 1762/4500] [psnr: -66.477274, tv: 0.005627, wnorm: 14596944.000000, mn: 0.041458] ETA: 16:10:22.806079
2020-11-10 09:27:11,882 train INFO: [Epoch 2/400] [Batch 1763/4500] [psnr: -66.478682, tv: 0.005627, wnorm: 14404615.000000, mn: 0.041460] ETA: 15:04:20.630295
2020-11-10 09:27:11,911 train INFO: [Epoch 2/400] [Batch 1764/4500] [psnr: -66.478084, tv: 0.005627, wnorm: 14862783.000000, mn: 0.041461] ETA: 14:50:47.098360
2020-11-10 09:27:11,943 train INFO: [Epoch 2/400] [Batch 1765/4500] [psnr: -66.478093, tv: 0.005628, wnorm: 13660739.000000, mn: 0.041463] ETA: 15:34:41.668975
2020-11-10 09:27:11,971 train INFO: [Epoch 2/400] [Batch 1766/4500] [psnr: -66.478094, tv: 0.005628, wnorm: 14588550.000000, mn: 0.041464] ETA: 14:08:38.659673
2020-11-10 09:27:12,002 train INFO: [Epoch 2/400] [Batch 1767/4500] [psnr: -66.477663, tv: 0.005628, wnorm: 14661858.000000, mn: 0.041466] ETA: 15:20:52.748933
2020-11-10 09:27:12,031 train INFO: [Epoch 2/400] [Batch 1768/4500] [psnr: -66.475148, tv: 0.005629, wnorm: 14658763.000000, mn: 0.041468] ETA: 14:29:40.018005
Have you tried to remove the randomness of your method? For example, fixing the random seed to obtain the same results for multiple training process with the same setting.
from image-adaptive-3dlut.
HuiZeng
commented on June 9, 2024
There may be some bugs in your training code.
Yes, I have tried. But it seems that there are some randomness that cannot be fixed in pytorch.
from image-adaptive-3dlut.
hkzhang95
commented on June 9, 2024
I re-write the codebase because of the code style. And maybe the problem is that I put the LUTs into a list and enumerate them in generator_train like this:
def generator_train(img, LUT_list):
pred = classifier(img).squeeze()
if len(pred.shape) == 1:
pred = pred.unsqueeze(0)
gens = []
for LUT in LUT_list:
gens.append(LUT(img))
weights_norm = torch.mean(pred**2)
combine_A = img.new(img.size())
for b in range(img.size(0)):
for i, gen in enumerate(gens):
combine_A[b, :, :, :] += pred[b, i] * gen[b, :, :, :]
return combine_A, weights_norm
When I remove the definition of the function and write the code in the main function, the psnr seems normal in the early epochs. I guess there may be some problems with the gradient? Not exactly sure about the reason now.
I will check the results tomorrow. Thanks a lot.
from image-adaptive-3dlut.
hkzhang95
commented on June 9, 2024
The new performance matches the paper result. I will think about the reason later and close this issue now.
from image-adaptive-3dlut.
Related Issues (20)
- 有能将3dlut可视化的脚本不 HOT 1
- about windows setup trilinear_cpp error
- Why does input image in XYZ while expertC image in RGB
- Specifications of the shared Pretrained Models
- For tone-mapping task, why did you use the expertC as the GT?
- Eval images corrupt when training using unpaired dataset HOT 1
- 你好！当我将3DLUT模型的pth模型转换到onnx模型的时候，出现trilinearInterpolationFunction报错，该怎么解决呢？
- python3 image_adaptive_lut_train_paired.py
- Segmentation fault while training HOT 6
- how to solve 3dlut banding HOT 1
- Fatal Error (Is it a LUT?) HOT 3
- 3DLUT 参数量计算 HOT 2
- LUT 学习方式 HOT 1
- CPU和GPU的运行结果不一致
- AttributeError: module 'trilinear' has no attribute 'forward' HOT 3
- 关于LUT的问题 HOT 1
- About TV_3D loss
- 關於kernel size
- 为什么 classifier 采用复杂网络，性能狂掉？
- 直接跑预训练模型 视频调色比较稳定，但是自己训练的模型无法复现预训练效果？
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from image-adaptive-3dlut.