google-research/fixmatch

A simple method to perform semi-supervised learning with limited data.

FixMatch

Code for the paper: "FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence" by Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel.

This is not an officially supported Google product.

Setup

Important: ML_DATA is a shell environment variable that should point to the location where the datasets are installed. See the Install datasets section for more details.

Install dependencies

sudo apt install python3-dev python3-virtualenv python3-tk imagemagick
virtualenv -p python3 --system-site-packages env3
. env3/bin/activate
pip install -r requirements.txt

Install datasets

export ML_DATA="path to where you want the datasets saved"
export PYTHONPATH=$PYTHONPATH:"path to the FixMatch"

# Download datasets
CUDA_VISIBLE_DEVICES= ./scripts/create_datasets.py
cp $ML_DATA/svhn-test.tfrecord $ML_DATA/svhn_noextra-test.tfrecord

# Create unlabeled datasets
CUDA_VISIBLE_DEVICES= scripts/create_unlabeled.py $ML_DATA/SSL2/svhn $ML_DATA/svhn-train.tfrecord $ML_DATA/svhn-extra.tfrecord &
CUDA_VISIBLE_DEVICES= scripts/create_unlabeled.py $ML_DATA/SSL2/svhn_noextra $ML_DATA/svhn-train.tfrecord &
CUDA_VISIBLE_DEVICES= scripts/create_unlabeled.py $ML_DATA/SSL2/cifar10 $ML_DATA/cifar10-train.tfrecord &
CUDA_VISIBLE_DEVICES= scripts/create_unlabeled.py $ML_DATA/SSL2/cifar100 $ML_DATA/cifar100-train.tfrecord &
CUDA_VISIBLE_DEVICES= scripts/create_unlabeled.py $ML_DATA/SSL2/stl10 $ML_DATA/stl10-train.tfrecord $ML_DATA/stl10-unlabeled.tfrecord &
wait

# Create semi-supervised subsets
for seed in 0 1 2 3 4 5; do
    for size in 10 20 30 40 100 250 1000 4000; do
        CUDA_VISIBLE_DEVICES= scripts/create_split.py --seed=$seed --size=$size $ML_DATA/SSL2/svhn $ML_DATA/svhn-train.tfrecord $ML_DATA/svhn-extra.tfrecord &
        CUDA_VISIBLE_DEVICES= scripts/create_split.py --seed=$seed --size=$size $ML_DATA/SSL2/svhn_noextra $ML_DATA/svhn-train.tfrecord &
        CUDA_VISIBLE_DEVICES= scripts/create_split.py --seed=$seed --size=$size $ML_DATA/SSL2/cifar10 $ML_DATA/cifar10-train.tfrecord &
    done
    for size in 400 1000 2500 10000; do
        CUDA_VISIBLE_DEVICES= scripts/create_split.py --seed=$seed --size=$size $ML_DATA/SSL2/cifar100 $ML_DATA/cifar100-train.tfrecord &
    done
    CUDA_VISIBLE_DEVICES= scripts/create_split.py --seed=$seed --size=1000 $ML_DATA/SSL2/stl10 $ML_DATA/stl10-train.tfrecord $ML_DATA/stl10-unlabeled.tfrecord &
    wait
done
CUDA_VISIBLE_DEVICES= scripts/create_split.py --seed=1 --size=5000 $ML_DATA/SSL2/stl10 $ML_DATA/stl10-train.tfrecord $ML_DATA/stl10-unlabeled.tfrecord

ImageNet

Codebase for ImageNet experiments located in the imagenet subdirectory.

Running

Setup

All commands must be ran from the project root. The following environment variables must be defined:

export ML_DATA="path to where you want the datasets saved"
export PYTHONPATH=$PYTHONPATH:.

Example

For example, training a FixMatch with 32 filters on cifar10 shuffled with seed=3, 40 labeled samples and 1 validation sample:

CUDA_VISIBLE_DEVICES=0 python fixmatch.py --filters=32 --dataset=cifar10.3@40-1 --train_dir ./experiments/fixmatch

Available labelled sizes are 10, 20, 30, 40, 100, 250, 1000, 4000. For validation, available sizes are 1, 5000. Possible shuffling seeds are 1, 2, 3, 4, 5 and 0 for no shuffling (0 is not used in practiced since data requires to be shuffled for gradient descent to work properly).

Multi-GPU training

Just pass more GPUs and fixmatch automatically scales to them, here we assign GPUs 4-7 to the program:

CUDA_VISIBLE_DEVICES=4,5,6,7 python fixmatch.py --filters=32 --dataset=cifar10.3@40-1 --train_dir ./experiments/fixmatch

Flags

python fixmatch.py --help
# The following option might be too slow to be really practical.
# python fixmatch.py --helpfull
# So instead I use this hack to find the flags:
fgrep -R flags.DEFINE libml fixmatch.py

The --augment flag can use a little more explanation. It is composed of 3 values, for example d.d.d (d=default augmentation, for example shift/mirror, x=identity, e.g. no augmentation, ra=rand-augment, rac=rand-augment + cutout):

• the first d refers to data augmentation to apply to the labeled example.
• the second d refers to data augmentation to apply to the weakly augmented unlabeled example.
• the third d refers to data augmentation to apply to the strongly augmented unlabeled example. For the strong augmentation, d is followed by CTAugment for fixmatch.py and code inside cta/ folder.

Valid dataset names

for dataset in cifar10 svhn svhn_noextra; do
for seed in 0 1 2 3 4 5; do
for valid in 1 5000; do
for size in 10 20 30 40 100 250 1000 4000; do
    echo "${dataset}.${seed}@${size}-${valid}"
done; done; done; done

for seed in 1 2 3 4 5; do
for valid in 1 5000; do
    echo "cifar100.${seed}@10000-${valid}"
done; done

for seed in 1 2 3 4 5; do
for valid in 1 5000; do
    echo "stl10.${seed}@1000-${valid}"
done; done
echo "stl10.1@5000-1"

Monitoring training progress

You can point tensorboard to the training folder (by default it is --train_dir=./experiments) to monitor the training process:

tensorboard.sh --port 6007 --logdir ./experiments

Checkpoint accuracy

We compute the median accuracy of the last 20 checkpoints in the paper, this is done through this code:

# Following the previous example in which we trained cifar10.3@250-5000, extracting accuracy:
./scripts/extract_accuracy.py ./experiments/fixmatch/cifar10.d.d.d.3@40-1/CTAugment_depth2_th0.80_decay0.990/FixMatch_archresnet_batch64_confidence0.95_filters32_lr0.03_nclass10_repeat4_scales3_uratio7_wd0.0005_wu1.0/

# The command above will create a stats/accuracy.json file in the model folder.
# The format is JSON so you can either see its content as a text file or process it to your liking.

Adding datasets

You can add custom datasets into the codebase by taking the following steps:

1. Add a function to acquire the dataset to scripts/create_datasets.py similar to the present ones, e.g. _load_cifar10. You need to call _encode_png to create encoded strings from the original images. The created function should return a dictionary of the format {'train' : {'images': <encoded 4D NHWC>, 'labels': <1D int array>}, 'test' : {'images': <encoded 4D NHWC>, 'labels': <1D int array>}} .
2. Add the dataset to the variable CONFIGS in scripts/create_datasets.py with the previous function as loader. You can now run the create_datasets script to obtain a tf record for it.
3. Use the create_unlabeled and create_split script to create unlabeled and differently split tf records as above in the Install Datasets section.
4. In libml/data.py add your dataset in the create_datasets function. The specified "label" for the dataset has to match the created splits for your dataset. You will need to specify the corresponding variables if your dataset has a different # of classes than 10 and different resolution and # of channels than 32x32x3
5. In libml/augment.py add your dataset to the DEFAULT_AUGMENT variable. Primitives "s", "m", "ms" represent mirror, shift and mirror+shift.

Citing this work

@article{sohn2020fixmatch,
    title={FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence},
    author={Kihyuk Sohn and David Berthelot and Chun-Liang Li and Zizhao Zhang and Nicholas Carlini and Ekin D. Cubuk and Alex Kurakin and Han Zhang and Colin Raffel},
    journal={arXiv preprint arXiv:2001.07685},
    year={2020},
}

fixmatch's Issues

Ids for the 1 samples per class runs

Hi guys,
Thanks for the great work and for the code ! I was wondering if you would make available the image ids or the dataset files for training with the eight "buckets" of 10 annotated images.

Thanks,
Paul

Accessing the pre-trained models

Hi there,
Where can I download the pre-trained model for this work?

training collapsed cifar10.3@40-1 with 4 GPUs

Dear authors,

any possible ideas result in following training collapsed?

docker environment: (TF 1.14.0) https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/rel_19.10.html#rel_19.10

CUDA_VISIBLE_DEVICES=0,1,2,3 python fixmatch.py --filters=32 --dataset=cifar10.3@40-1 --train_dir ./experiments/fixmatch

Thanks

Feature request: use this with structured data

As title suggests

Validation Data

Hello. First of all, thanks for the great study !!
I have a question about validation data.

To select an optimal model, how the validation data are selected?

• 5 different labeled dataset & compute minimum test error?
• Distinct validation dataset (from training & test dataset) for model selection? (then, how is it selected?)

After i read the paper and code, i think the former is the way you implement, but i want to check it.

Inputs to the layers that project to logits

Hello again,

You have provided some very good information in my previous post. But I am still eager to learn. I hope it is okay to bother you once more. I intend on accessing the inputs to the layers that project to logits. I want to save these tensors in a file. I was wondering if you can point me in the right direction in how to do this. So far I have tried "f.write(tf.strings.as_string(logits_x)"
but Im sure there must be a better way to do this. Thanks again

Question related to dataset and create_datasets.py

It seems that create_datasets.py only accepts certain options for online datasets.
Is there any way if we want to use our own data in our local folder?

How to implement WRN-37-2 in STL-10 experiment？

In your STL-10 experiment subsection, you said 'Following [3], we use a WRN-37-2 network (comprising 23.8M parameters).' However, in the paper of 'Wide Residual Networks' (official paper), the WRN-40-2 only has 2.2M parameters. So how to implement WRN-37-2 in STL-10 experiment？

Question about RandAugment Implementation

In the paper you mention that flips are applied with 50 percent probability. Is it also the case that each randaugment sample is also applied with 50 percent probability or are two randaugment choices always applied to each unlabelled image?

The reason I ask is that in the main pytorch reimplementation of your work there is a 50 percent chance for each randaugment policy to be applied but I didn't see this in your paper

Many Thanks

What is a typical wall-clock time for a FixMatch run?

Thanks for releasing this code and especially for such fine research behind it.

Could you please let me know what is the typical wall-clock time for a FixMatch run? My runs typically take 4 days, which seems a bit high to me.

Misrepresented WRNs

Hi there, I really enjoy your work. However, I feel a bit like standard WRNs have been misrepresented in a way throughout the _ixMatch lifespan. In MixMatch Section 4.2.2: '135 filters per layer' does not make sense to me. Firstly, something like '135 filters for the first residual block' would be more justifiable. Secondly, for a standard WRN starting with 16 filters, 135 filters would mean a k-value of 8.4375, as in WRN28-8.4375.

Furthermore, ReMixMatch refers to 'the same WRN-37-2-network' in Section 4.2, but in reality this is a different model that came about inbetween the papers. My problem lies not primarily in the fact that 135 was changes to 128 filters (the k-value of 8 seems more reasonable: e.g. WRN-28-8). I do however feel a bit uneasy that it was presented as a WRN-37-2 model. Moreover, FixMatch also refers to the same WRN-37-2 model. Indeed, it could have 37 conv layers and a similar structure to a WRN. But to my knowledge, such a model does not exist and the closest standard WRN models have either 34 or 40 conv layers (why not use these instead?). Using anything other than --scales=3 would deviate from the standard WRN model structure and the resulting model would be another creature.

I understand that the focus of this research is on the SSL-methods and the comparison thereof using the same models. It might also be that my comprehension in the above is lacking. I would however appreciate clarity surrounding issues like these, rather than avoiding or dragging along underlying deficiencies if any are known. Thanks.

How to reproduce the results of fully supervised learning with only 40 labels

Table 9 shows that you can achieve about 36% accuracy in a supervised manner by only using 40 labels. However, I didn't see the codes to reproduce it, can you share the command/codes you use? Plus, may I know the hyperparameters for the experiment because it is lacking in the paper. Thanks.

image layers

Hello Google-Research,

Amazing work! I'm just wondering if you know which layer contains the image layers or representations of images after the model is fully trained.

Thanks again for posting this. It has been very useful.

Training ImageNet dataset with multi nodes

Hello. I have a question.
When i train a model with the ImageNet dataset on "GPU",
how can i use the multi-node training?

I think the scripts in READ.md do not contain any information about it.

Replication of the barely supervised learning experiments

Hi, I am sorry to bother you but we met some problems to replicate the barely supervised learning experiments in your paper.
I wonder if you could provide me with the ordering of the CIFAR-10 training set in this experiment. And my email address is [email protected]. Thanks a lot.
Looking forward to your reply.

CIFAR-100 Model

In section 4.1 of the paper, you state:

To begin with, we compare FixMatch to various existing methods on the standard CIFAR-10, CIFAR-100, and SVHN benchmarks. As recommended by [31], we reimplemented all existing baselines and performed all experiments using the same codebase. In particular, we use the same network architecture (a Wide ResNet-28-2 [47] with 1.5M parameters) ...

Then in the Appendix, you state:

As mentioned in section 4, we used almost identical hyperparameters of FixMatch on CIFAR-10, CIFAR-100, SVHN and STL-10. Note that we used similar network architectures for these datasets, except that more convolution filters were used for CIFAR-100 to handle larger label space ...

I'm trying to reproduce this particular result, however "more convolution filters" isn't defined in the paper with a number. If my understanding is correct, the codebase is setup so that the default filters for the model is 32, and then you allow users to optionally configure this by passing a --filters argument to the main script. In other words, there's no single source of config files that we can review to understand how CIFAR-100 was treated differently.

Would you be able to provide these implementation details on CIFAR-100?

Error in ict.py

Hello, I want to run ict.py and learn this method, but I get the error as follows：

tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: You must feed a value for placeholder tensor 'x' with dtype float and shape [?,32,32,3]
	 [[{{node x}}]]
  (1) Invalid argument: You must feed a value for placeholder tensor 'x' with dtype float and shape [?,32,32,3]
	 [[{{node x}}]]
	 [[strided_slice_6/_1069]]

I found that in main, there is the following:

mix = tf.distributions.Beta(beta, beta).sample([tf.shape(x_in)[0], 1, 1, 1])
mix = tf.maximum(mix, 1 - mix)

but in train.py no input for x_in:

x, y = gen_labeled(), gen_unlabeled()
self.tmp.step = train_session.run([self.ops.train_op, self.ops.update_step],
                                    feed_dict={self.ops.y: y['image'],
                                                self.ops.xt: x['image'],
                                                self.ops.label: x['label']})[1]

So x_in in the main function should be replaced with xt_in, right?

no of epochs

How many epochs the model has been trained for results reported in the paper?

What is the meaning of expanding the dataset about labeled and unlabeled data?

Hi, I recently read FixMatch's paper and code. In the implementation of FixMatch, for example, 4000 labeled data; you need to copy the labeled data to 65536, and copy the remaining unlabeled to 65536*7. I don't quite understand why do this. Do you know why do this？ Or is it the default setting for semi-supervised fields?

Is BN updated for the whole 15 batches?

Hi,
Thanks for publishing your brilliant work! Recently I'm trying to reproducing this code by pytorch and I found some differences between fixmatch and mixmatch in the following lines:

1. The interleave function in fixmatch doesn't make any change to the input x, so the interleave function in fixmatch is different with that in mixmatch, am Icorrect?
2. In both your issue and code of mixmatch, you mention that you only update BN mean and variance using the first batch(labeled inputs), but in this fixmatch, it seems you update BN using the whole 15 batches inputs(1 labeled and 14 unlabeled) , do I understand it correctly? If so, why use all the batches?

Thanks in advance for your reply

How many epoch should be great for Training on ImageNet datasets?

Hi,
After implementing your brilliant work to PyTorch and testing it on Cifar10, recently when I try to reproduce the results on ImageNet. And I am puzzled with the setting of epochs. From your paper, I find that the number of epoch is 300 for unsupervised data, and we know that the amount of the unsupervised data is 10 times as large as that of supervised data, and in each batch the amount of unsupervised data is 5 times(ulratio=5) as large as supervised's, so I guess the number of epoch for supervised data should be 300*10/5 = 600？ However, in the Readme.txt of your code, I find that you set 3000 epochs for supervised data, why? BTW, I am wondering how long the training process will last? I guess 3000 epochs will cost really long time.

Thanks in advance for your reply

question about image whiten and model variance

Hi,

Thanks for the work and your patience of answering each of our questions !!!

I am confused about some details about the whiten mechanism. If I did not misunderstand the code, you switched off the usage of whiten by assigining default False value to FLAGS.whiten in the data.py script. But in the randaugment.py script, some augmentation operations need to call wrap_pil method which assumes the image is already normalized (divide 255 and whiten with mean/std). Is this the right behavior within your mean to implement ?

By the way, it seems that you used a fix dataset split (cifar-xx@40-5000 or so). Did you do experiements on randomly split the dataset each time you train the model ? I have did a little experiement myself (with pytorch), and found that the model variance is larger than expected. And since the wrongly gauessed labeled would let the model accumulate wrong "knowledge", sometimes the model would suffer from the problem of overfitting.

OutputStream realloc failed

Hi,
I am trying to run your code on ImageNet. And I begin to make the split by the command ./generate_ssl_imagenet.sh --runner=DirectRunner --direct_num_workers=16
I successfully make the labeled data split, but when it begins to make the unlabeled split (using the code:[ python -B save_sharded_data.py
--input_dir="${IMAGENET_DIR}"
--sharding_file="data_splits/files2shards_train.txt"
--output_file_prefix="${SSL_IMAGENET_DIR}/train"
-- "$@"]
), the error "AssertionError: OutputStream realloc failed." always occurs, so what can I do to fix it?

Thanks in advance for your help

When fix patch is applied to my dataset, the dataset is unbalanced. Why does the training set perform well, but the loss of verification set first decreases and then keeps rising?

"Aborted" while running multiple processes in train_session.run

The process is aborting right at the beginning of training. Is it because of the multiple processes running at a time? Can't figure out the issue.

Error in ./cta/cta_remixmatch.py

Hi,

Thanks for releasing the code on FixMatch, great to have it released so soon alongside the paper. I was able to run fixmatch.py on CIFAR10 after correcting line 22 in ./cta/cta_remixmatch.py

from: from remixmatch import ReMixMatch

to: from remixmatch_no_cta import ReMixMatch

Hope you make this change so others can run it successfully.

Leaky ReLU in ResNet

Standard ResNets are known to use ReLU activation function,
but i found that your implementation uses Leaky ReLU instead of ReLU.

Does replacing ReLU into Leaky ReLU affect the results?

`XLA_GPU` and no GPU utilization

When I run

CUDA_VISIBLE_DEVICES=0 python fixmatch.py --filters=32 --dataset=cifar10.0@40-1 --train_dir ./experiments/fixmatch

I see the process use GPU memory in nvidia-smi, but there is 0% GPU utilization and training is super slow. When I look at the devices returned by libml/utils.py:get_available_gpus, the local_device_protos are all XLA_GPU instead of GPU. Any ideas on what might be going on here and how to fix? Presumably this is some kind of version issue?

(Apologies that this is a more general TF question, but I wasn't able to find a working fix by Googling)

What's the args for remixmatch on cifar100?

Hi, thanks for your great job. It's amazing that the FixMatch can perform so well. And I also very curious about the reproduction results of ReMixMatch on cifar100.
We want to reproduce the results in table1. However, the results of ReMixMatch on cifar100 are significantly lower than the results in your paper. It's very grateful if you can tell me the details of ReMixMatch on cifar100.

FixMatch with MixUp

Thanks for the great research. Can I ask you a question? The paper said "One may replace strong augmentation in FixMatch modality-agnostic augmentation strategies, such as MixUp". I can't understand this part well. I am curious about the specific way MixUp was used in FixMatch.
For example, If there are two unlabeled data, FixMatch mixes up two data with 0.4:0.6 ratio. And FixMatch gives consistency loss between Pseudo-label and prediction. So, Ideal weakly Augmented model prediction should be [..., 0.4, 0.6, ...]. Am I right? Thanks for reading.

Wide resnet adopted here is different from the official one ??

Good work and thanks for your code.

I have one question that the wide resent adopted here seems different from the official one, besides the number of the filter ?? I believe the following codes give the implementation of the wide resnet, right?

fixmatch/libml/models.py

Line 62 in b41d9e3

class ResNet(ClassifySemi):

For example, the dropout is adopted in each block and the Relu activation is adopted in the official one, while the Dropout is only adopted before the final FC layer and the LeakyRelu activation is adopted here ??

Learning rate of ImageNet Training

I found that the lr is decayed at every 50 epoch.
However, it is different to the description in the paper (60, 120, 160, 200 epochs)

Motivation for EMA

Hi --

Are you able to provide a little insight into why you use an exponential moving average of the model weights? Do you have a sense of how important this is for good performance? I'm not super familiar w/ this technique, and didn't see a ton of discussion in the paper.

Thanks!

how to export model to savedmodel?

Hi, very appreciate to your job.
Could you pls share how to export model to savedmodel? I just know that with estimator, but I don't know how to implement it under your project.
Thanks

args to reproduce meanteacher, pimodel, etc

Hi, may I ask could you please tell me the args of all 7 algorithms to reproduce the reported results?
(For example, weight decay, consistency_weight, etc)

Total parameters of cifar100 model

Hello. Thanks for the awesome study.

Could you let me know the total number of parameters of cifar100 model (--filter 1280)??

How's FixMatch compared to supervised methods?

Thanks for sharing the great work.

It might be interesting to compare FixMatch with supervised learning methods, given the same labeled data and model structure, to further demonstrate the power of SSL.

I appreciate if you could comment on the idea or release experimental results on such comparison.

Regards,

Compatibility with MixMatch repo

Hi!

Congrats on the paper, I'm excited to try FixMatch on my own dataset. I actually have previously used your code for MixMatch and I was wondering if this new repo is fully compatible with it, i.e., have there been any changes in the implementation of the common files between the two repos?

Thanks! :)

Running fixmatch with RandAugment

First of all, thanks for publishing the code for this paper -- it makes reproducing your work much easier.

I am trying to run fixmatch with RandAugment+Cutout instead of CTAugment (as it's much easier to implement in practice and the performance is often comparable). I've read #4 which states that fixmatch always uses CTAugment (additionally to what's provided in the --augment flag). So if I understand this correctly, --augment d.d.rac does not achieve what I want since CTAugment is always added on top? I suspect that's what's happening since I still see the CTAugment-weight printout when running training with that argument.

If that's the case, is there an easy way to disable CTAugment?

accuracy drops at late stage of training

Hi,

When I was reproducing FixMatch with 40 labels, I encountered an accuracy drop with two different labeled dataset (seed=1, 2).
I wonder whether you have also faced this problem before or the problem is caused by my machine or environment.

(cifar10.1@40-1)

(cifar10.2@40-1)

I use this command to reproduce FixMatch.

export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64/
export ML_DATA="..."
export PYTHONPATH=$PYTHONPATH:.

CUDA_VISIBLE_DEVICES=0,1,2,3 python fixmatch.py \
--filters=32 \
--dataset=cifar10.3@40-1

Reproduce Fixmatch with RandAug

Thanks for the nice work and for releasing the code.

I am also trying to reproduce the CIFAR100 result of fixmatch with RandAugment+Cutout. In the paper, CIFAR100 with 10000 labels reaches 77.40% test accuracy. But I failed to get close to the number. Could you please enlighten me on what I might do wrong? I have checked #19 , and here is the command I used:
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python ablation/ab_fixmatch_ra.py --filters=32 --dataset=cifar100.1@10000-1 --train_dir ./experiments/fixmatch/

and used this to extract the accuracy:
./scripts/extract_accuracy.py ./experiments/fixmatch/cifar100.d.d.rac.1@10000-1/FixMatch_RA_archresnet_batch64_confidence0.95_filters32_lr0.03_nclass100_repeat4_scales3_uratio7_wd0.0005_wu1.0/

and here is the result:
"last01": 72.77999877929688, "last10": 72.43999862670898, "last20": 72.57499694824219, "last50": 72.69499969482422

If it is possible, could you maybe provide more details on the 77.40% experiments?

Thank you for your kind help in advance.

Model for CIFAR-100

In Appendix B.1 of the FixMatch paper, it says

Note that we used similar network architectures for these datasets, except that more convolution filters were used for CIFAR-100 to handle larger label space

Are you able to elaborate on this? I'm trying to get close to the CIFAR-100 results using my own FixMatch implementation, and want to make sure I'm being faithful to the experiments in the paper. I don't see any special model parameterization for CIFAR-100 in this repo -- so maybe it's not necessary?

Thanks!

How to reproduce the results of Table 11

Hi there, I really enjoy your work. However, I cannot reproduce the results of Table 11. Could you please share the code to reproduce it? Thanks.

encouter messy augmented images when the image shape is not square

Hi, I encouter autoaugmented images below, and i locate the problem is caused by that the pic_array shape used in augmentations is in wrong oder and my image shape is not square.

fixmatch/third_party/auto_augment/augmentations.py

Lines 160 to 161 in 08d9b83

	s = pil_img.size
	pic_array = (np.array(pil_img.getdata()).reshape((s[0], s[1], 4)) / 255.0)

1. make it correct
   the size of PIL img is（width,height)
   the shape of np.array which can be converted to PIL img should by (height,width,c), so the right way is
   s = pil_img.size
pic_array = (np.array(pil_img.getdata()).reshape((s[1], s[0], 4)) / 255.0)

2. make it faster
   I find np.array(pil_img.getdata()).reshape() is really slow, the much faster way is np.array(pil_img)

3. By the way, i have a question: why 'RGBA' format is used? Is 'RGBA' format image necessary? I checked the valued of 'A’ channel of my images are all 255.
   Thank you.

Did you use random augment as strong aug as claimed in the paper?

Hi,

Thanks for bring the work to us. I am going through the code you released, and I noticed that the augmentations are assigned with an argment of augment whose default value is d.d.d. Does it mean that in this repo you use default aug methods for both strong and weak augments for the unlabeled samples ?

How to include all labeled examples as part of unlabeled data?

The FixMatch paper said that "In practice, we include all labeled examples as part of unlabeled data without using their labels when constructing U". I'm curious about how to include all labeled examples as part of unlabeled data.

Did you repeat the labeled examples or just concatenate them with the unlabeled data?

In semi-supervised learning, if you just use 1% samples of ImageNet as labeled data and concatenate them with the remaining 99% samples as unlabeled data, would it really affect the final results?

commands to reproduce results in the paper

Hi
can you provide the commands (like you did in MixMatch repo, the runs/ folder) to reproduce the results in the paper?

Thanks a lot! and thanks for the great work!

Does CTAugment let each augment operation have 17 bins?

Hi,

I am learning about the details of the code, and I appreciate that you could help me with some puzzles.

I noticed that in the ctaugment.py each augment operator is registared to have 17 bins. Does this mean that the magnitude value of each operator will range from 0-17 before mapped to specific range of each augment operator ?

By the way, it seems that there are more augment operators in ctaugment than random augment(including smooth, bluring, cutout, etc), isn't it?

About the epochs and update iterations

Hi. Thanks for publishing this repo.
While I'm testing Fixmatch algorithm, I have several questions.

1. According to the experiment setup described in the paper, the K = 2^20 which is the total update iterations.
   From the codes below,
   

   fixmatch/mixmatch.py

   Line 174 in 08d9b83

   FLAGS.set_default('train_kimg', 1 << 16)

   
   

   fixmatch/mixmatch.py

   Line 158 in 08d9b83

   model.train(FLAGS.train_kimg << 10, FLAGS.report_kimg << 10)

   
   the total update iterations seem to be 2^26.
   Is it right?

2. Is there a special reason for setting to 1 epoch to be 2^16 iterations?

Thanks,

program frozen

Dear authors

Your work is very exciting and I want to try out your code. I followed the instructions on readme, and is trying to run this example

CUDA_VISIBLE_DEVICES=0 python fixmatch.py --filters=32 --dataset=cifar10.3@40-1 --train_dir ./experiments/fixmatch

however my program get stucked at self.train_step forever...

I did installed the required environments as you pointed out in the readme.

Do you have any idea what's going on?

I