serious artifact with sequences "foliage", "bridge" for 4x scale

[fujin678]

Please check the attachments.

[fujin678]

[fujin678]

[fujin678]

i followed official docs. THe command of my tests:

python demo/restoration_video_demo.py --start_idx 0001 --filename_tmpl "{:04d}.png" ./configs/restorers/basicvsr/basicvsr_reds4.py https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20120409-0e599677.pth /home/fjin2/workspace/tensorflow/TecoGAN/LR/foliage/ ./outputs/foliage_BasicVSR

[fujin678]

torch: 1.9.0
cuda: 11

[fujin678]

looks like basicVSR accumulate errors. In my test, first 60 frames look good. Then artifact start to show and become more serious with more frames processed

[ckkelvinchan]

In your case, I think the degradations in your test images are not the same as those during training. From what I know, TecoGAN did not use bicubic downsampling.

[fujin678]

thanks for your advice.
You are right, I used the test image from TecoGAN, which probably causes those artifact. Later on, i tried your matlab gaussian code to pre-process those test images, the artifact became much less serious.

BTW, can you share matlab code of bicubic downsampling?

[ckkelvinchan]

We simply use imresize(I, 0.25) in MATLAB. If you want to use a Python version, you can see #507 as a reference. But do note that there is a slight difference to the official MATLAB implementation.

[fujin678]

i see, thanks a lot.

[fujin678]

I used imresize('foliage", 0.25) to generate LQ foliage sequence. Then I ran below (this is reds4 model). I do not see serious artifact as above. Also, first 30 frames look reasonable. But artifact starts to show up after 30 frames. Please check attachement

python demo/restoration_video_demo.py --start_idx 0001 --filename_tmpl "{:04d}.png" ./configs/restorers/basicvsr/basicvsr_reds4.py https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20120409-0e599677.pth /home/fjin2/workspace/tensorflow/TecoGAN/LR/foliage_bicubic/ ./outputs/foliage_bicubic_BasicVSR

[fujin678]

for the same foliage seq, i also tried vimeo-bi config. It does not have the artifact from reds4.

[ckkelvinchan]

There could be a domain gap between Vimeo-90K and REDS.

[fujin678]

I see, thanks.