Separate test from train

[jilu95]

Separate test from train

[Svito-zar]

I think it is an issue with Tensorflow model saving/loading.

To verify that I suggest evaluating a freshly trained model, then saving it, loading it from the checkpoint and evaluating the restored model again. If you get different results - then the issue is clearly in saving/loading.

[jilu95]

I have repeatedly trained to save the model several times and load the model, this problem will still appear.I can't find the error when saving and loading the model.The code of load the model comes from train.py.

[jilu95]

I run test.py twice,
The first result isbasketball: 0.8070919273342075 basketball: 1.0064784338799717
The second result isbasketball: 0.8070919273342075 basketball: 1.0064784338799717
The two results are the same.But,why the basketball different when tested together?I suspect that there is a problem in calling the test function. Is there any random value I have not found?but I have not found it yet.

[jilu95]

Mark points are randomly missing during training, but at the time of testing, the missing points are fixed.Or maybe there is a possibility that this deviation exists, but why does this deviation exist? I don't understand very well.

[Svito-zar]

@jilu95 , can you , please, provide, the code snippet you are using to get those results on the basketball sequence? (not the whole file, but just the code when you save, load an test the model)

[jilu95]

I have already sent you an email about code.Please let me know if something is not available.

[Svito-zar]

I think you did not understand what I mean.

What I suggested you try is to modify the code in train.py in the following way:

1. Train the model

2. Test the model (with your setting)

3. Save the model in the checkpoint

4. Load the model from this checkpoint

5. Test it again.

And that all should be in the same file. The results of step 2 and 5 should be the same, if saving/restoring work fine.

[Svito-zar]

@jilu95 , I had more time to look at your issue now and I think I found where the different results come from.

The function test has randomness in it, since it generates a random mask for the missing values using function cont_gap_mask. So every time you run test you are testing with different mask and hence getting different results.

Does it answer your question?

[jilu95]

Thank you for you help.It's really a matter of randomness.Thanks again!