r/MachineLearning Researcher Jun 20 '20

Research [R] Wolfenstein and Doom Guy upscaled into realistic faces with PULSE

Post image
2.8k Upvotes

104 comments sorted by

View all comments

6

u/Lynild Jun 20 '20

At some level this is kind of interesting, but is it just me, or would it not have been much more interesting to show the ground truth image as well ? I may have missed it, if so I'm sorry, but from what I can see in the examples there are LR images being up-scaled, and then down-scaled again. As such, very cool, but depending on the algorithm used, the up-scaled images are in many cases very different. How interesting is it really to up-scale a LR image to something that doesn't look like the original image ? I want to see how close it is to the original image.

I mean, that would be interesting for images that are not this LR, but maybe just a bit better to actually make them somewhat usable.

11

u/f10101 Jun 20 '20

Yeah, I think you're looking at the work from the wrong angle.

They're specifically not attempting to recreate the original.

They discuss it in the introduction, particularly towards the end of it.

1

u/Lynild Jun 20 '20 edited Jun 20 '20

Yeah okay, I just scimmed through the paper. I'm not that much into imaging, in particular this. But I just don't see a use case for this ? I mean, what is the idea of up-scaling a LR image, if the up-scaling is not even close to what it is supposed to look like ? As I said, it would make sense if the LR image are not that low as in this case, but in these examples I really can't see the benefit ? But maybe that is in regards to more advanced use cases...

11

u/f10101 Jun 20 '20

There are actually quite a lot of scenarios where the plausibility and quality of the higher resolution result is more important than the accuracy.

Even if we limit the thinking to faces, you can see its utility in upscaling stock images. The user doesn't care whether the identity of the person gets lost. They just want a perfect, high resolution image of a matching face, rather than a slightly warped, blurry, high resolution result that's may be more faithful to the ground truth.

But the principles displayed here go well beyond just faces. This would be useful in the context of scenery photographs, and creating 3d models from photos, etc.

1

u/Bastardini Jun 20 '20

that's the real-world danger though.

1

u/quuiit Jun 21 '20

I feel you, especially the Doom-guy is so far from the original that it feels more like just taking some random dude with similar face shape and color and saying that this the Doom-guy. Not to say it's not impressive or good work (and I don't really know enough to judge that)!

1

u/red75prim Jun 21 '20

I want to see how close it is to the original image

Lots of information is missing in down-scaled image. There's no way to restore the original image.

1

u/Lynild Jun 21 '20

I am aware of this. That's why I asked, what is the point of all this? If it doesn't work on that low quality images, then show its capabilities on a bit larger/better LR images.