*sad update* ESRGAN AI Upscaling (ScummVM fork and tools pulled from github by author)

All the inane chatter goes in here. If you're curious about whether we will support a game, post HERE not in General Discussion :)

Moderator: ScummVM Team

Post Reply
nachazo
Posts: 4
Joined: Tue Dec 13, 2022 7:49 am

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by nachazo »

Thanks! I'm waiting for ScummVM devs opinion/help (bad dates for this things anyway! :D)
invwar
Posts: 67
Joined: Mon Jul 28, 2008 11:35 am

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by invwar »

NVidia made their RTX Technology now also available for Videos. Don't expect yet wonders as you can see with DLSS2, but DLSS1 was also back then just a simple super sampling, before it started to really shine with DLSS2. But as you might wonder, I made a [url=
first comparison video for you. The biggest difference here you see I think with the leather chair, but in general it is not MUCH better. But in general I have to say it feels better, but this might be just a feeling, as its harder to see in the video. But this might be also because of some effects lost in the process too.
I hope NVidia is working more on their models and it will in a few years maybe shine as DLSS2.

I have also checked with newer videos with a proper HD source, which I then set to the infamous 360p mode. They still look terrible, but you can see here much more details and formost typical high scaling effects like super smoothened faces.
Haravikk
Posts: 1
Joined: Sun Jul 23, 2023 11:44 am

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Haravikk »

invwar wrote: Tue Feb 28, 2023 10:34 pm NVidia made their RTX Technology now also available for Videos. Don't expect yet wonders as you can see with DLSS2, but DLSS1 was also back then just a simple super sampling, before it started to really shine with DLSS2. But as you might wonder, I made a [url=
first comparison video for you. The biggest difference here you see I think with the leather chair, but in general it is not MUCH better. But in general I have to say it feels better, but this might be just a feeling, as its harder to see in the video. But this might be also because of some effects lost in the process too.
I hope NVidia is working more on their models and it will in a few years maybe shine as DLSS2.

I have also checked with newer videos with a proper HD source, which I then set to the infamous 360p mode. They still look terrible, but you can see here much more details and formost typical high scaling effects like super smoothened faces.
Came by to the forums wondering about the same kind of thing; I use FSR (Fidelity FX Super Resolution) in newer video games like Cyberpunk 2077 because it lets me upscale from 720p or 1080p rendering to run at 1440p resolution and the result is generally very good, giving me a better overall picture on my limited hardware (not powerful enough to render natively at 1440p, at least not without a lot of heat and noise).

Obviously upscaling individual art assets and running games using more advanced AI upscaling will give better results, but that's a very time consuming process that has to be done for each game. While it might theoretically be possible to add a generic method to ScummVM to "pre-scale" assets for later use, it seems like more of a long term goal to me.

But if ScummVM could implement a basic form of FSR on final frames (which should work on all recent graphics hardware) it would allow many games to display at higher resolutions than they were originally capable of and look a bit better than basic "stretched" upscaling. FSR is designed around an adjustable scaling (up to 100%, so 540p becomes 1080p etc.) as well as adjustable sharpening of the result so we could still tweak for a sharper or softer look. The most complicated part would be feeding FSR with whatever motion data it expects, but I don't believe this is a necessary step (it will work without it, it just might suffer some ghosting on rapid motion). Once implemented users could then toggle this upscaling on or off for individual games, share what results they think work best for their favourite games, and maybe help inform some defaults for games that work well with it?

Once FSR is implemented it seems easy to also then enable DLSS for Nvidia hardware, because the API is essentially the same; many games with FSR or DLSS have DLL files available that allow switching FSR to DLSS or vice versa depending upon hardware (so DLSS-only games can run FSR on AMD hardware, FSR-only games can use DLSS on Nvidia where results are sometimes better etc.). But I think it makes sense to implement FSR first, since it's the most compatible, as DLSS is Nvidia only.

I'd love to try some of my favourite games like Zork Nemesis and Grand Inquisitor to see how they look with some modern upscaling; I'm not expecting miracles of course, but even a 50% increase in resolution could make them look a lot better on bigger screens.
User avatar
criezy
ScummVM Developer
Posts: 951
Joined: Sat Sep 23, 2006 10:41 am
Location: West Sussex, UK

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by criezy »

Haravikk wrote: Sun Jul 23, 2023 2:06 pm But if ScummVM could implement a basic form of FSR on final frames
This may already be available. As far as I know FSR is nothing more than a shader, and since the 2.7.0 release, shaders can be used in ScummVM to scale games.

However there are different types of shaders and different languages in which shaders can be written. The original AMD FSR is written as a compute shader, and ScumMVM does not support those I think. But it is Open Source and was for example ported as a pixel/vertex slang shader. ScummVM doesn't support that either and you would need a GLSL pixel/vertex shaders. But maybe somebody already ported it to GSLS as well.

So I don't know if FSR specifically exists in a form usable by ScummVM currently, but there are many other shaders you could try and may like.
User avatar
LogicDeLuxe
Posts: 435
Joined: Thu Nov 10, 2005 9:54 pm

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by LogicDeLuxe »

I played with ComfyUI. This took a lot of tweaking. This image was then upscaled and further enhanced from 1024x768 to 1440x1080 Full HD with Topaz Photo AI.
Enhanced Guybrush and Elaine finale
Enhanced Guybrush and Elaine finale
Guybrus_und_Elaine.png.jpg (353.39 KiB) Viewed 5970 times
User avatar
Raziel
ScummVM Porter
Posts: 1525
Joined: Tue Oct 25, 2005 8:27 am
Location: a dying planet
Contact:

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Raziel »

@LogicDeLuxe

ho.ly.dung.pile :shock:

now my imagination is running wild picturing the rest of the game

mind.blown
Kasenshlogga
Posts: 68
Joined: Thu Jul 18, 2019 1:31 pm

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Kasenshlogga »

LogicDeLuxe wrote: Thu Mar 28, 2024 10:04 pm I played with ComfyUI. This took a lot of tweaking. This image was then upscaled and further enhanced from 1024x768 to 1440x1080 Full HD with Topaz Photo AI.
Guybrus_und_Elaine.png.jpg
That's downright terrifying.
User avatar
Wanderer6
Posts: 2
Joined: Sat May 18, 2024 6:32 am
Location: Australia

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Wanderer6 »

The concept here looks very promising - but the links in the original post (to the ScummVM-X fork, SCI Companion X, etc) all appear to be dead and the content removed from the original GitHub account. Does anybody know what happened to this effort?
User avatar
MusicallyInspired
Posts: 1138
Joined: Fri Mar 02, 2007 8:03 am
Location: Manitoba, Canada
Contact:

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by MusicallyInspired »

Wanderer6 wrote: Sun May 19, 2024 4:11 am The concept here looks very promising - but the links in the original post (to the ScummVM-X fork, SCI Companion X, etc) all appear to be dead and the content removed from the original GitHub account. Does anybody know what happened to this effort?
When the original author of those modified tools and fork of ScummVM heard that Microsoft was buying ActiVision, he decided to pull everything off and send Microsoft an email asking them permission to continue what he was doing. I tried to talk him out of it but he was adamant. Ergo they sadly are no longer available.
Kasenshlogga
Posts: 68
Joined: Thu Jul 18, 2019 1:31 pm

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Kasenshlogga »

MusicallyInspired wrote: Mon May 27, 2024 10:22 pm
Wanderer6 wrote: Sun May 19, 2024 4:11 am The concept here looks very promising - but the links in the original post (to the ScummVM-X fork, SCI Companion X, etc) all appear to be dead and the content removed from the original GitHub account. Does anybody know what happened to this effort?
When the original author of those modified tools and fork of ScummVM heard that Microsoft was buying ActiVision, he decided to pull everything off and send Microsoft an email asking them permission to continue what he was doing. I tried to talk him out of it but he was adamant. Ergo they sadly are no longer available.
Seriously?! What a letdown.
I've been wondering what happened to this project, it seemed so promising in the beginning but nobody ever did anything with it.

Also what was even the point of that? He didn't even work on mods directly, he created a platform for people to create mods based on ScummVM, which makes this decision even more strange to me. Whatever people do with ScummVM is none of Microsoft's business.

Modding always was a grey zone but only in very rare cases do companies go after modders, besides I highly doubt Microsoft would go after modders of 20+ year old point and click adventure games.
User avatar
Wanderer6
Posts: 2
Joined: Sat May 18, 2024 6:32 am
Location: Australia

Re: *sad update* ESRGAN AI Upscaling (ScummVM fork and tools pulled from github by author)

Post by Wanderer6 »

Thanks for the update. Strange indeed.

Given the GPL licensing of ScummVM and therefore of derivative code as well - does anybody not have a fork of the ScummVM-X code which can be made available under the GPL, for example as reference for any future efforts in this space?
User avatar
MusicallyInspired
Posts: 1138
Joined: Fri Mar 02, 2007 8:03 am
Location: Manitoba, Canada
Contact:

Re: *sad update* ESRGAN AI Upscaling (ScummVM fork and tools pulled from github by author)

Post by MusicallyInspired »

Kasenshlogga wrote: Wed May 29, 2024 10:02 am Seriously?! What a letdown.
I've been wondering what happened to this project, it seemed so promising in the beginning but nobody ever did anything with it.

Also what was even the point of that? He didn't even work on mods directly, he created a platform for people to create mods based on ScummVM, which makes this decision even more strange to me. Whatever people do with ScummVM is none of Microsoft's business.

Modding always was a grey zone but only in very rare cases do companies go after modders, besides I highly doubt Microsoft would go after modders of 20+ year old point and click adventure games.
I agree. He was earnest about "doing the right thing" but his nobility was misguided and a little silly if you ask me. Very disappointing.
Wanderer6 wrote: Fri May 31, 2024 12:32 pm Thanks for the update. Strange indeed.

Given the GPL licensing of ScummVM and therefore of derivative code as well - does anybody not have a fork of the ScummVM-X code which can be made available under the GPL, for example as reference for any future efforts in this space?
If he would have been okay with that he would have left the code up in the first place. I do have the source from an older version without all the latest features. But despite how much I disagree with him, I don't want to publicly release his work without his consent (which he won't give).
Post Reply