*UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

All the inane chatter goes in here. If you're curious about whether we will support a game, post HERE not in General Discussion :)

Moderator: ScummVM Team

Post Reply
User avatar
criezy
ScummVM Developer
Posts: 891
Joined: Sat Sep 23, 2006 10:41 am
Location: West Sussex, UK

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by criezy »

That comment from Eugene refers to the possibility of using ESRGAN as a real time scaler to upscale the game to a higher resolution.

Another approach discussed in this thread is to use ESRGAN to generate a set of upscaled assets beforehand, and modify ScummVM to render the game directly at higher resolution using those assets (instead of using the original resolution and scaling the display on the fly). In term of development, this is a lot of work that needs to be done separately for each engine, and is unlikely to be integrated in the official ScummVM. And the user also has a lot of work to do as due to copyright reasons the distribution of such assets is not possible (unless the copyright holders agree to it), and thus each user would need to generate their own upscaled assets, for example using ESRGAN on assets extracted from their original copy of the game. But such an approach remain technically possible, even if not very practical, and for example work is being done in a separate fork (https://github.com/MiLO83/scummvmext) for the SCI engine.

Since Curse of Monkey Island is not an SCI game, that work will not help though and somebody would need to do similar work for the SCUMM engine. So don't expect it to be available any time soon, or even ever. It could happen but I would not hold my breath.
ZTylerDurden
Posts: 3
Joined: Thu Sep 09, 2021 2:51 pm

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by ZTylerDurden »

criezy wrote: Thu Sep 09, 2021 9:26 pm That comment from Eugene refers to the possibility of using ESRGAN as a real time scaler to upscale the game to a higher resolution.

Another approach discussed in this thread is to use ESRGAN to generate a set of upscaled assets beforehand, and modify ScummVM to render the game directly at higher resolution using those assets (instead of using the original resolution and scaling the display on the fly). In term of development, this is a lot of work that needs to be done separately for each engine, and is unlikely to be integrated in the official ScummVM. And the user also has a lot of work to do as due to copyright reasons the distribution of such assets is not possible (unless the copyright holders agree to it), and thus each user would need to generate their own upscaled assets, for example using ESRGAN on assets extracted from their original copy of the game. But such an approach remain technically possible, even if not very practical, and for example work is being done in a separate fork (https://github.com/MiLO83/scummvmext) for the SCI engine.

Since Curse of Monkey Island is not an SCI game, that work will not help though and somebody would need to do similar work for the SCUMM engine. So don't expect it to be available any time soon, or even ever. It could happen but I would not hold my breath.
Great explanation. Thank you!

Is there a scumm dev tool to extract/dump the game's textures?
User avatar
sev
ScummVM Lead
Posts: 2174
Joined: Wed Sep 21, 2005 1:06 pm
Contact:

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by sev »

ZTylerDurden wrote: Fri Sep 10, 2021 1:25 am Is there a scumm dev tool to extract/dump the game's textures?
There are no textures. There are pictures for backgrounds and sprites for objects, all are in a quite bizarre format. But for actors, there is a whole separate language that directs which parts of the actor to draw and at what position and time (see the list of opcodes here). To browse through those you may use tools like ScummEX.


Eugene
ZTylerDurden
Posts: 3
Joined: Thu Sep 09, 2021 2:51 pm

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by ZTylerDurden »

sev wrote: Fri Sep 10, 2021 12:36 pm
ZTylerDurden wrote: Fri Sep 10, 2021 1:25 am Is there a scumm dev tool to extract/dump the game's textures?
There are no textures. There are pictures for backgrounds and sprites for objects, all are in a quite bizarre format. But for actors, there is a whole separate language that directs which parts of the actor to draw and at what position and time (see the list of opcodes here). To browse through those you may use tools like ScummEX.


Eugene
Awesome thank you. I'm gonna take a look.
Kasenshlogga
Posts: 51
Joined: Thu Jul 18, 2019 1:31 pm

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Kasenshlogga »

So it's been like a almost a year, are there any completed Sierra High Res packs yet?

If there are any I'm not sure where to look for them. It's not like these games have Nexusmods or ModDB pages.
User avatar
MusicallyInspired
Posts: 1122
Joined: Fri Mar 02, 2007 8:03 am
Location: Manitoba, Canada
Contact:

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by MusicallyInspired »

I'm working on SQ4 and SQ5 at the moment but it's slow going because I've got a bunch of other things to do. It's a meticulous process because some models don't work well with some backgrounds. So I'm doing a lot of pre and post-upscale pixel modifications to backgrounds and sprites to make sure they come out properly. Also, some sprites are used for background animation so if you upscale these animation frames separately from the background they're meant for then once ingame they'll stick out like a sore thumb because of the different upscalings. So there's things like that to consider. Also, some screens (especially in SQ5) use paletted screens for colour cycling. ScummVM-ext also supports this but the resulting upscaled image has to have the exact same 8-bit palette as the original resource in order for the colour cycling to actually work. That's also a complicated (but doable) process. Ditto for sprites that colour cycle.

I have recreated (not upscaled) the SQ4 and SQ5 GUI interface though.
sq4interface.png
sq4interface.png (389.78 KiB) Viewed 1492 times
Kasenshlogga
Posts: 51
Joined: Thu Jul 18, 2019 1:31 pm

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Kasenshlogga »

MusicallyInspired wrote: Fri Oct 01, 2021 9:17 pm I'm working on SQ4 and SQ5 at the moment but it's slow going because I've got a bunch of other things to do. It's a meticulous process because some models don't work well with some backgrounds. So I'm doing a lot of pre and post-upscale pixel modifications to backgrounds and sprites to make sure they come out properly. Also, some sprites are used for background animation so if you upscale these animation frames separately from the background they're meant for then once ingame they'll stick out like a sore thumb because of the different upscalings. So there's things like that to consider. Also, some screens (especially in SQ5) use paletted screens for colour cycling. ScummVM-ext also supports this but the resulting upscaled image has to have the exact same 8-bit palette as the original resource in order for the colour cycling to actually work. That's also a complicated (but doable) process. Ditto for sprites that colour cycle.

I have recreated (not upscaled) the SQ4 and SQ5 GUI interface though.

sq4interface.png
That UI looks so clean, great job!

Yeah, I guess it makes sense that it's a meticulous process, if you want to do the artwork justice and not make it look like you just ran it through an algorithm.
Blakes7
Posts: 39
Joined: Sun Nov 03, 2019 9:44 am

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by Blakes7 »

MusicallyInspired wrote: Fri Oct 01, 2021 9:17 pm I'm working on SQ4 and SQ5 at the moment but it's slow going because I've got a bunch of other things to do. It's a meticulous process because some models don't work well with some backgrounds. So I'm doing a lot of pre and post-upscale pixel modifications to backgrounds and sprites to make sure they come out properly. Also, some sprites are used for background animation so if you upscale these animation frames separately from the background they're meant for then once ingame they'll stick out like a sore thumb because of the different upscalings. So there's things like that to consider. Also, some screens (especially in SQ5) use paletted screens for colour cycling. ScummVM-ext also supports this but the resulting upscaled image has to have the exact same 8-bit palette as the original resource in order for the colour cycling to actually work. That's also a complicated (but doable) process. Ditto for sprites that colour cycle.

I have recreated (not upscaled) the SQ4 and SQ5 GUI interface though.

sq4interface.png
I'm not sure if you are still working on this but any update?
User avatar
MusicallyInspired
Posts: 1122
Joined: Fri Mar 02, 2007 8:03 am
Location: Manitoba, Canada
Contact:

Re: *UPDATES!* ESRGAN AI Upscaling (HiRes packs now possible!)

Post by MusicallyInspired »

I haven't touched it in a while, no. Lots of stuff going on.
Post Reply