Do you think ScummVM release testing is cut too short?

General chat related to ScummVM, adventure gaming, and so on.

Moderator: ScummVM Team

Do you think ScummVM release testing is cut too short?

No, I think the team is right in being eager with release plans.
21
100%
 
Total votes: 21

joachimeberhard
ScummVM Team Member
Posts: 377
Joined: Sat Sep 24, 2005 12:25 pm
Location: Austria

Do you think ScummVM release testing is cut too short?

Post by joachimeberhard »

Dear ScummVM fans, users, testers and programmers!

I have been thorougly testing the upcoming ScummVM 0.8.0 release for some time now.

Although stability and performance has very much improved with every new CVS version, I encountered many new bugs introduced with updates of the CVS.

Now I want to know if you think that the release plans have been set to eager by setting the release testing phase to approximately 3-4 weeks.

What do you think?

:wink:

Best regards

Joachim Eberhard
Last edited by joachimeberhard on Fri Oct 28, 2005 1:55 pm, edited 4 times in total.
User avatar
sev
ScummVM Lead
Posts: 2273
Joined: Wed Sep 21, 2005 1:06 pm
Contact:

Post by sev »

No, I don't think so. All release critical games were tested already. As of FM-TOWNS games you submitted zillion bugs for, all of them go with lowest priority as we don't have enough game copies. I.e. no copies in hands of those capable to fix them.


Eugene
joachimeberhard
ScummVM Team Member
Posts: 377
Joined: Sat Sep 24, 2005 12:25 pm
Location: Austria

Post by joachimeberhard »

While I really admire the improvements introduced with 0.8.0, I have encountered many regressions in CVS introduced in the past few days.

Thats why I ask.

I really respect your opinion, and admire the good work you have done in the past few weeks in terms of bugfixing.

And, as I promised I will spend 20$ every 3 months from now for helping you buy some games.
I pay for games I like, so why shouldn't I pay for making it possible to play games I LOVE.

You have made me so very happy with ScummVM and I admire your work very much.

Best regards

Joachim Eberhard
Last edited by joachimeberhard on Tue Nov 01, 2005 1:40 pm, edited 1 time in total.
Ender
Retired
Posts: 106
Joined: Wed Sep 21, 2005 5:06 am
Location: Perth, Western Australia

Post by Ender »

To be honest (and I will reply to your e-mail soon :), if you are mainly donating $20/month for the purpose of better FM-TOWNS support... Your better off actually saving up and just buying the games for us :)

It's still quite hard to find most of these titles.
Last edited by Ender on Sun Oct 30, 2005 9:45 am, edited 2 times in total.
joachimeberhard
ScummVM Team Member
Posts: 377
Joined: Sat Sep 24, 2005 12:25 pm
Location: Austria

Post by joachimeberhard »

To be honest, one of my biggest problems is "saving up".

I have only been able to buy some of the rare games because I was surprisingly lucky enough to be the only bidder.

Yesterday I bought 4 different Mac-Versions for total of 35$.

Another problem is, I would pay shipment twice, one time to me and one time to you.

And I think you have a better idea which games you need.
But for helping me and some other possible donators, you could state somewhere which games are needed most (ie no-one in the team has a copy of a particular game.)

So, for now I just spend my 20 bucks every now and then just because I think you deserve it. If I ever manage to get another copy of any game cheap enough (Won't pay over 100$), I will donate it.

Best regards

Joachim
User avatar
eriktorbjorn
ScummVM Developer
Posts: 3523
Joined: Mon Oct 31, 2005 7:39 am

Post by eriktorbjorn »

At the risk of sounding flippant...


Given how rare and/or expensive the FM-Towns games tend to be, it might be both quicker and cheaper to invest the money in a good C++ book (or maybe borrow one at the library, if that's an option) and see if you can figure out any more details about the bugs yourself.

In some cases, it helps to compare the scripts of a version that works to the scripts of the one that misbehaves to see what they're doing differently. (In one case, a particularly amusing bug in the Indy 3 intro was tracked down by comparing the game's scripts to the ones from a demo.)
Last edited by eriktorbjorn on Tue Nov 01, 2005 7:39 pm, edited 1 time in total.
fingolfin
Retired
Posts: 1452
Joined: Wed Sep 21, 2005 4:12 pm

Post by fingolfin »

Regarding FM-TOWNS games: I own the Indy3 version for FM-TOWNS, and have access to ZakTowns. However, I never even saw the other games (Loom, Monkey, Monkey 2, Indy4 -- are there more?).

Please believe us: It's annoying to see that many bugs on our bug tracker, and it's made worse by the fact that I can't even do anything about it since I don't have those games and hence can't work on fixes.

However, considering how (comparatively) rare those games are, and considering that at least in the case of MI/MI2/Indy4 they don't seem to offer any advantage over the regular (and common) PC versions, the priority for fixing these issues is rather low. It's a bit different for Zak and Loom, since those are "unique".
User avatar
MetaFox
Posts: 54
Joined: Mon Oct 31, 2005 10:44 am

Post by MetaFox »

fingolfin wrote:Regarding FM-TOWNS games: I own the Indy3 version for FM-TOWNS, and have access to ZakTowns. However, I never even saw the other games (Loom, Monkey, Monkey 2, Indy4 -- are there more?).
That is all of them. The games were adapted to the FM-Towns Marty by James Leiterman. The two "unique" games (Loom and Zak) were adapted by a two man team - consisting of James and another team member. The other games were adapted to the FM-Towns Marty solely by Mr. Leiterman himself.

In a bit of unrelated info - James Leiterman was also responsible for localizing the NES version of Maniac Mansion to French, German, Italian, Spanish and Swedish. He also localized the PC version of Maniac Mansion to Italian.

http://66.15.18.176/leiterman/lfilm.html
Ender
Retired
Posts: 106
Joined: Wed Sep 21, 2005 5:06 am
Location: Perth, Western Australia

Post by Ender »

Jims a very friendly guy, we've had a few chats about the FM-TOWNS games :)
MetaFox wrote:The games were adapted to the FM-Towns Marty by James Leiterman. The two "unique" games (Loom and Zak) were adapted by a two man team - consisting of James and another team member. The other games were adapted to the FM-Towns Marty solely by Mr. Leiterman himself.
joachimeberhard
ScummVM Team Member
Posts: 377
Joined: Sat Sep 24, 2005 12:25 pm
Location: Austria

Post by joachimeberhard »

I know that it is hard for you to fix bugs for FM-Towns versions.

But the problem with the Kanji versions was introduced AFTER CVS 10/23/2005, short before release.

This is a very very sad regression of the final.

Considering that you were in feature freeze mode at that time, and I (hopefully) clearly statet in the bug-reports that this was introduced SHORT before release, this is a sad fact.

I hope you didn't ignore my bugreports just because I reported to many already and you didn't take me serious.

I am sorry that I am no programmer, so I just can't fix this bug myself.

And just to complete the list of unique FM-Towns-Games:

INDY3Towns is also much better than the standard PC version because of the CD-Soundtrack.

However, I DO understand your point about the difficulty to fix these bugs.

What can I do?

I already wrote to some good eBay sellers I know, if they could make me a special price if they find any FM-Towns games, for donating it to ScummVM.

But this could take some time.

Well, until then...

Best regards

Joachim
User avatar
eriktorbjorn
ScummVM Developer
Posts: 3523
Joined: Mon Oct 31, 2005 7:39 am

Post by eriktorbjorn »

joachimeberhard wrote:I hope you didn't ignore my bugreports just because I reported to many already and you didn't take me serious.
I doubt that. In fact, people are often encouraged to write individual bug reports rather than reporting several different bugs as one. It's frustrating to see so many bug reports, of course, but that's hardly a reason to ignore them. It's just that most ScummVM users and developers neither have the games, nor speak Japanese.
joachimeberhard wrote:INDY3Towns is also much better than the standard PC version because of the CD-Soundtrack.
I already have the movie soundtrack CD, so I don't think I'm missing that much. :wink:
User avatar
Lord Savage
Posts: 27
Joined: Thu Oct 27, 2005 9:41 pm
Location: Canada

I voted that the testing for 0.8.0 was too short

Post by Lord Savage »

First off I'd like to thank everyone involved with this release for their hard work. Please don't consider that I am slagging off on your hard work.

So, why did I vote that I felt that the testing was too short?

Because my primary (Mac OS X) and secondary (PalmOS) platforms did not offer test builds during the testing phase. The only platform that got a thorough testing workout this time around was the Windows one. The last two releases have had frequent official CVS builds of the Mac OS X port for me to test. That was definitely not the case this time around. The most recent official CVS build available on the site was dated June 24th.

To make matters worse, from my point of view, the PalmOS version isn't even ready yet. To me, this looks like this release was rushed out the door.

Any way that is why I voted that I felt that this version was rushed out the door.
User avatar
sev
ScummVM Lead
Posts: 2273
Joined: Wed Sep 21, 2005 1:06 pm
Contact:

Re: I voted that the testing for 0.8.0 was too short

Post by sev »

Lord Savage wrote: Because my primary (Mac OS X) and secondary (PalmOS) platforms did not offer test builds during the testing phase.
...
To make matters worse, from my point of view, the PalmOS version isn't even ready yet. To me, this looks like this release was rushed out the door.
Let me explain it as a 0.8.0 release coordinator.

MacOS X is one of the main development platforms in ScummVM. Several key developers and moreover our co-lead Fingolfin run it every day. So it is very tested. Moreover, there is no difference between MacOS, Windows, Linux, *nix, OS/2 ports. All they use exactly same codebase (except some Altivec code in MacOS and some audio drivers). That means that if there is a bug in MacOS, it will appear in Windows too (technically endianness could play a role here, but every engine was tested on both big- and little-endian platforms, and only sword2 had alignment ptoblems which are already fixed in 0.9.0). All of that makes MacOS test builds redundant.

Speaking of PalmOS, yes, it's a pity. The problem that the porter decided to redo whole backend and make it much more faster, he is just didn't came up to deadline, and he warned about it long beforehand. I'd suggest you to read scummvm-devel archives for details. So if we'd even postpone release because of PalmOS by 2 weeks, that wouldn't guarantee us presence of the port in final 0.8.0. All of us are volunteers, PalmOS porter included, and he just can't jump over his head.


Eugene
User avatar
Lord Savage
Posts: 27
Joined: Thu Oct 27, 2005 9:41 pm
Location: Canada

Re: I voted that the testing for 0.8.0 was too short

Post by Lord Savage »

sev wrote: Let me explain it as a 0.8.0 release coordinator.

MacOS X is one of the main development platforms in ScummVM. Several key developers and moreover our co-lead Fingolfin run it every day. So it is very tested. Moreover, there is no difference between MacOS, Windows, Linux, *nix, OS/2 ports. All they use exactly same codebase (except some Altivec code in MacOS and some audio drivers). That means that if there is a bug in MacOS, it will appear in Windows too (technically endianness could play a role here, but every engine was tested on both big- and little-endian platforms, and only sword2 had alignment ptoblems which are already fixed in 0.9.0). All of that makes MacOS test builds redundant.

Speaking of PalmOS, yes, it's a pity. The problem that the porter decided to redo whole backend and make it much more faster, he is just didn't came up to deadline, and he warned about it long beforehand. I'd suggest you to read scummvm-devel archives for details. So if we'd even postpone release because of PalmOS by 2 weeks, that wouldn't guarantee us presence of the port in final 0.8.0. All of us are volunteers, PalmOS porter included, and he just can't jump over his head.


Eugene
Thanks for taking the time to reply to my concerns. I do realize that this project is a completely volunteer based. I also find it to be one of the most professional volunteer based projects I have ever seen.

However, I still believe that 0.8.0 was rushed out the door. Using the Mac OS X build as example you stated that there are three areas where the Mac codebase differs the standard codebase. They are endianness, AltiVec and audio drivers.

You already mentioned that sword2 alignment problem (endianness) so, that is one out of three having an issue. Looking at the bug tracker shows priority 6 bug [ 1342732 ] MAC: hq2x/hq3x filters broken when altivec is enabled in which fingolfin had to disable the AltiVec code in 0.8.0 in order for the hq filters to work. That's two out of three having an issue. Further up the bug tracker shows two possibly related priority 5 issues (correct me if I'm wrong) [ 1344631 ] build fails on Mac OS X 10.2.8 when MT32 emulation is used and [ 1343971 ] MAC: ScummVM 0.8.0 fails to launch which might be related to a conflict between Mac OS X 10.2.8 and MT32 emulation through the audio drivers. No one is assigned to either of these cases so, there is no telling if they are duplicatable or related or if they actually is related to the audio drivers so, this one is just a potential at this point.

So definitively two of the three Mac specific areas have issues in the 0.8.0 release. Depending on the outcome 1344631 and 1343971 there may be an issue in the audio drivers as well making it three for three. Had there been at least one official CVS build for Mac OS X after the Oct 11 call for testers I truly believe that these would have been caught and corrected before 0.8.0 release. So, in my opinion an official CVS build for Mac OS X is not redundant.

Thanks for clarifying the hold up on the PalmOS port; it would have been nice to have been informed in the main page ahead time that the PalmOS port wasn't going to be included in the 0.8.0 release to temper our disappointment. Since my programming claim to fame is 10 PRINT "Hello World", I've never felt that I was worthy of reading to the scumm-devel archives ;)
User avatar
eriktorbjorn
ScummVM Developer
Posts: 3523
Joined: Mon Oct 31, 2005 7:39 am

Re: I voted that the testing for 0.8.0 was too short

Post by eriktorbjorn »

Lord Savage wrote: You already mentioned that sword2 alignment problem (endianness) so, that is one out of three having an issue.
Just to clarify, endianness and alignment are two completely different beasts. Endianness concerns in which order the bytes of a multi-byte value are stored, while alignment places restrictions on where in memory such values may be stored. E.g. for a 32-bit value the address may have to be dividable by four.

Broken Sword 2 should already be endian safe. I haven't heard any Mac-specific bug reports about it, either. Then again, it wasn't until recently that I heard there may be alignment problems in -- I think -- the PalmOS port. The alignment problems have hopefully been straightened out on the CVS trunk, but that was quite a large change so it didn't go into the 0.8 branch. Maybe if it had been reported earlier. Maybe.

Yet a third porting issue is whether or not the code is 64-bit clean. Particularly in Broken Sword 2, which used to do very naughty things with data pointers. The jury is still out on that one, but I hope it is safe.
Last edited by eriktorbjorn on Tue Nov 01, 2005 7:40 pm, edited 1 time in total.
Post Reply