I extracted a load of .7z files and two failed with CRC error in WinRAR. Ran a CLRMAMEPRO on all the .7z's to ensure integrity and there were no errors reported - only a single "7z: ERROR SZE_ERR appearing in the Warnings window with no file name. Has anyone else had a problem similiar to this?

Quite worrying...

Nothing is worrying at all. The default scan operation only reads zip/rar/7z headers. If you need a more in deep operation, either use

- Settings->Compressor->Full Zip Structure Scan
or
- Scanner->Checksums->decompress rom & check crc32 (or sha1/md5 check if the used datfile provides this data)

Both operations will slow down the processes.


Roman Scherzer

> Nothing is worrying at all. The default scan operation only reads zip/rar/7z
> headers. If you need a more in deep operation, either use
>
> - Settings->Compressor->Full Zip Structure Scan
> or
> - Scanner->Checksums->decompress rom & check crc32 (or sha1/md5 check if the
> used datfile provides this data)
>
> Both operations will slow down the processes.
>
>
> Roman Scherzer
>
Roman I do already have BOTH options on.

PS. I have already emailed you a screen shot with the problem.

> Roman I do already have BOTH options on.

The first option is zip only. If the second option doesn't report any error, the file (the data of the compressed file, not the archive) is valid.

You talk about 7z files here, not good old zip. 7zip (and esp. its SDK) is a lousy format when it comes to checking structures or caring for standards. You shouldn't check it with Winrar either. The sdk and so cmpro doesn't check for corrupt structures in 7zip files. If cmpro is able to decompress the file and check the crc and reports it as valid, the data is valid and rebuilding it to zip will create a fine file.

If you need 7z structure scans, complain to the sdk authors. You can send me one of the bad files and I may have a look at it if it's even possible to capture such things. To sum it up: nothing really to worry about.


Roman Scherzer

> > Roman I do already have BOTH options on.
>
> The first option is zip only. If the second option doesn't report any error, the
> file (the data of the compressed file, not the archive) is valid.
>
> You talk about 7z files here, not good old zip. 7zip (and esp. its SDK) is a
> lousy format when it comes to checking structures or caring for standards. You
> shouldn't check it with Winrar either. The sdk and so cmpro doesn't check for
> corrupt structures in 7zip files. If cmpro is able to decompress the file and
> check the crc and reports it as valid, the data is valid and rebuilding it to
> zip will create a fine file.
>
> If you need 7z structure scans, complain to the sdk authors. You can send me one
> of the bad files and I may have a look at it if it's even possible to capture
> such things. To sum it up: nothing really to worry about.
>
>
> Roman Scherzer
>
Hmm...
I don't know how cmpro reckons the "data" is valid (ie. able to decompress, check the crc and not report a problem) when I have tried a "Test" and also the "Extract" in WinRar and both options give me a CRC error and also 7z.exe reports an error...here's the output from a CMD prompt:
C:\>d:7z e "1894 - Koharu No DS Uchigohan. Shokuji Balance Guide Tsuki (J)(6rz).
7z"

7-Zip 4.57 Copyright (c) 1999-2007 Igor Pavlov 2007-12-06

Processing archive: 1894 - Koharu No DS Uchigohan. Shokuji Balance Guide Tsuki (
J)(6rz).7z

Extracting 1894 - Koharu No DS Uchigohan. Shokuji Balance Guide Tsuki (J)(6rz).
nds CRC Failed

Sub items Errors: 1


C:\>d:7z e "1936 - The Golden Compass (U)(Sir VG).7z"

7-Zip 4.57 Copyright (c) 1999-2007 Igor Pavlov 2007-12-06

Processing archive: 1936 - The Golden Compass (U)(Sir VG).7z

Extracting 1936 - The Golden Compass (U)(Sir VG).nds CRC Failed

Sub items Errors: 1

...but again cmpro didn't show any errors!
Does the "decompress and check CRC" actually decompress or does it just use the CRC value packed into the header?

I can email you the files but 1894 is ~24mb and 1936 is ~15mb.

I will try to extract (and keep the broken?! file) and re-pack it into a zip and re-test this...

[Edit]
I have used 7z.exe to extract each and re-packed them into separate zips and these fail the CRC check in cmamepro...so all I can deduce from this is it can't detect [particular] CRC errors in .7z files?

Thanks,
Azza

> I don't know how cmpro reckons the "data" is valid (ie. able to decompress,
> check the crc and not report a problem) when I have tried a "Test" and also the
> "Extract" in WinRar and both options give me a CRC error and also 7z.exe reports
> an error...here's the output from a CMD prompt:


Because cmpro isn't winrar and not 7z. cmpro uses the sdk to access the data and uses the 7z sdk again to decompress it and rechecks the checksums. Since the SDK doesn't return an error code, the sdk is either wrong or the data is ok.


> Does the "decompress and check CRC" actually decompress or does it just use the
> CRC value packed into the header?

It decompresses the data to memory, calculates the checksums over the decompressed data and compares it against the one stored in the datfile. Not against the one in the header, so maybe in your case the header is just wrong, while the data isn't.


> I can email you the files but 1894 is ~24mb and 1936 is ~15mb.

You can send me the smaller one to roman*dot*scherzer*at*gmail*dot*com


Roman Scherzer

after looking at the source I found out that the 'decompress and check crc32' option actually decompresses each file of an archive to memory. So it only checks the integrity of the data but it doesn't actually compare the hash against the datfile.

SHA1/MD5 checks do of course.

I will now add a real decompress and check crc32 and rename the old one to something like "test archive" or something.

In your case the data was successfully decompressed but it didn't most likely match the crc32 value of the datfile but it wasn't reported since you didn't scan for sha1 or md5 (if that's available in the datfile).

So...look out for the next version then...(don't ask me when though)


Roman Scherzer

> I will now add a real decompress and check crc32 and rename the old one to
> something like "test archive" or something.
Thanks - hopefully this will pick up corrupt "container files" (.7z or .zip)

I have mailed you the corrupt .7z for you to try out.

Next cmpro detects the bad one now.


Roman Scherzer