FAQ

Does my model have to reconstruct an image or video in full resolution, or can it be cropped?
The decoder must produce images or videos with the same resolution or pixel size as the corresponding image or video in the validation or test set.

How is PSNR calculated? (image tracks)
We compute a single MSE value by averaging across all RGB channels of all pixels of the whole dataset, and from that calculate a PSNR value.

How is MS-SSIM calculated? (image tracks)
First, a single MS-SSIM value is calculated for each image. We then compute a weighted average where each image is weighted by the number of its pixels.

Why are you computing PSNR and MS-SSIM?
We chose two popular metrics to be computed for all methods. These gives some idea to participants on how they might stack up, but provides no guarantees on correlation with the final evaluation.

The evaluation server gives "ERROR: Missing image IMG_20170114_210112.png". What am I doing wrong?
The error means that the decoder failed and did not produce all required files. Make sure that the names of the files match the names of the target files exactly.

How can I see the decoding progress?
Make sure that your decoder outputs this information to the standard output. This information should be visible once you've submitted a decoder.

In which directory should the decoder save images?
The decoder can save images in the current working directory . or in any arbitrary subfolder such as images.

Does the model count towards the target bit-rate?
No. This year we don't do this.

What are the specifications of the decoder environment?
Each submission is provided 2 CPUs and 12 GB RAM. In the GPU tracks, submissions can additionally use a L4 GPU with 24 GB memory. There is a time limit of 5 hours for each submission. Note that to encourage development of low-complexity methods, the 25% slowest submissions will not be considered for a prize. This is independent of the time limit.

Can a single organization (company, university) participate with more than one team?
As long as the methods are sufficiently different, yes.

What are sufficiently different methods?
An insufficiently different pair of methods is, for example, the same neural network with different hyperparameters (i.e., learning rate, loss weights, etc.). A sufficiently different pair of methods should be such that if one were to envision submitting them at the same conference/workshop, they would be both accepted without any concerns of plagiarism.

Will the images/videos my method generates be made public?
Yes. In the spirit of being able to compare across the various submitted methods at any point in the future, we plan to make all participants' reconstructions from the test sets publicly available. These will follow the licensing terms of the respective source.

If I participate as part of a company's team, am I eligible for prize money?
No. You're eligible for an award and an award certificate/trophy, but we will be distributing the money to the next eligible student team. We're doing this to enable more students to participate in person.

How can I contact the organizers (as opposed to a specific organizer)?
Please email us at clic-organizers+questions@googlegroups.com. This should hopefully provide you with a faster reply than contacting a single individual organizer.

How can I request to remove an image from the dataset(s)?
Please email us at clic-organizers+request@googlegroups.com.

Can I request a custom docker image?
Before requesting one, please check whether one of the already available images would work for you. If that does not work, please email us at clic-list@googlegroups.com.

Can I store any image or video data in my Docker image?
No. Image or video data may not be stored in Docker images. In the submission page, you're required to provide two files: (1) decoder binary/code; and (2) data. Everything needed to decode the data must be present in the first part (1). We explicitly separate the decoder from image/video data, so that the decoder can be frozen after validation. That decoder will then be used to decode the data for the test set, which will be secret until the freeze happens.