4.5.31 Bug: All mobile phone screenshots identified as duplicates

English support for the software AllDup
Post Reply
mkuser
Posts: 9
Joined: 24 Feb 2023, 20:43

4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by mkuser »

Dear author,

Using AllDup ver. 4.5.31, looks like found bug - have about 1000 _different_ screenshots (PNG files) from my mobile phone and when comparing them as Pictures regardless of comparing algorithm they all came as 100% duplicates.

File format is important for this bug to manifest - I have tried to convert my screenshots from PNG to JPG and they became different to AllDup. Only original PNG files came as 100% duplicates for _all _ picture comparison methods (literally all 10 of them - tested every one).

Original PNG files are big ones (1,5 - 6 MB), so couldn't attach samples to post but ready to provide by e-mail if needed to reproduce.

Could you please fix PNG files handling for picture duplicates search?

Thank you for your work!
Administrator
Site Admin
Posts: 4046
Joined: 04 Oct 2004, 18:38
Location: Thailand
Contact:

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by Administrator »

I cant reproduce this here using the search method "Find similar pictures" and different comparison methods.
I run the tests with 10 different png files in a folder. No dups found.
mkuser
Posts: 9
Joined: 24 Feb 2023, 20:43

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by mkuser »

Thank you for your answer!

Agree with your findings in general - it is not PNG problem, it has to be something special about my screenshots from phone. Have tested additionally - and with my other PNG files comparison works all right on my side also. But not with mobile screenshots - for them 100% identity comes again. On my next post screenshot ("4.5.31 Bug: Saved results comments overwritten") you could see 928 duplicates came for every comparison method.

Is it possible to send you two sample files to test on your side?

Below are log file from one of the passes:

24.02.2023 17:25:13 - INFO: Unable to detect VLC Media Player 32-bit version 3 on your system
24.02.2023 17:25:14 - --------------------------------------------------
24.02.2023 17:25:14 - AllDup 4.5.31
24.02.2023 17:25:14 - Search method: Find similar pictures
24.02.2023 17:25:14 - Comparison method: SHA-2 (512-Bit)
24.02.2023 17:25:14 - Image Formats: BMP, GIF, JPEG, JPG, PNG
24.02.2023 17:25:14 - Match: 100%
24.02.2023 17:25:14 - Picture area: entire picture
24.02.2023 17:25:14 - Comparison size: 100% Pixel
24.02.2023 17:25:14 - Checksum Strength: 512 Bit
24.02.2023 17:25:14 - Detect picture modifications (slower):
24.02.2023 17:25:14 - > Rotated 90° to the right
24.02.2023 17:25:14 - > Rotated 180° to the right
24.02.2023 17:25:14 - > Rotated 90° to the left
24.02.2023 17:25:14 - > Flipped horizontally
24.02.2023 17:25:14 - > Rotated 90° to the right and flipped horizontally
24.02.2023 17:25:14 - > Flipped vertically
24.02.2023 17:25:14 - > Rotated 90° to the left and flipped horizontally
24.02.2023 17:25:14 - Option: Use database
24.02.2023 17:25:14 - 1.Source folder: C:\_Photo
24.02.2023 17:25:14 - Option: Compare files from all source folders
24.02.2023 17:25:14 - Folder filter activated: 7
24.02.2023 17:25:14 - Filter type: Exclusive
24.02.2023 17:25:14 - 1.folder filter: c:\windows
24.02.2023 17:25:14 - 2.folder filter: c:\program files (x86)
24.02.2023 17:25:14 - 3.folder filter: c:\program files
24.02.2023 17:25:14 - 4.folder filter: ?:\system volume information
24.02.2023 17:25:14 - 5.folder filter: ?:\recycled
24.02.2023 17:25:14 - 6.folder filter: ?:\recycler
24.02.2023 17:25:14 - 7.folder filter: ?:\$recycle.bin
24.02.2023 17:25:14 - Determine file count of all source folders...
24.02.2023 17:25:14 - File count: 4 892
24.02.2023 17:25:14 - Scan: C:\_Photo
24.02.2023 17:26:02 - Files filtered: 1 008
24.02.2023 17:26:02 - Found 928 duplicates with a total of 2,99 GB inside folder 'C:\_Photo'
24.02.2023 17:26:02 - Scanned files: 4 892
24.02.2023 17:26:02 - Groups: 1
24.02.2023 17:26:02 - File comparison count: 45 821 519
24.02.2023 17:26:02 - Checksums created: 14
24.02.2023 17:26:02 - Database checksums used: 24 562
24.02.2023 17:26:02 - Database checksums stored: 14
24.02.2023 17:26:02 - Duplicates: 928 (18%) (2,99 GB)
24.02.2023 17:26:02 - Elapsed time: 00:00:48
therube
Posts: 322
Joined: 07 Nov 2012, 00:28

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by therube »

Is it possible to send you two sample files to test on your side?
Or you could zip them & upload them to a place like:
https://www.zippyshare.com/
or
https://www.upload.ee/

(and post the link here).
mkuser
Posts: 9
Joined: 24 Feb 2023, 20:43

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by mkuser »

Thank your for your reply!

Uploaded two sample files from 928 available here:

WrongDuplicates.zip
https://www66.zippyshare.com/v/r0ikDvi9/file.html

Waiting for your insights!

Below is screenshot from AllDup 4.5.31 comparing them:
Wrong duplicates.png
Comparison settings for screenshot above:
Search method.png
therube
Posts: 322
Joined: 07 Nov 2012, 00:28

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by therube »

Well color me confused?
(I don't know that I've particularly run an Image dup search before ? & I'll assume these are default values?)
Anyhow, showing as 100% match, much less that it found dups, seems... odd?
.
AllDup Image Dup Search - wrong, seemingly.png
Administrator
Site Admin
Posts: 4046
Joined: 04 Oct 2004, 18:38
Location: Thailand
Contact:

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by Administrator »

Your png files are *not* stored with the standard 24/32bit color model.
They are saved with a 64bit color model and this is not supported:
"PNG supports palette-based images (with palettes of 24-bit RGB or 32-bit RGBA colors)..."
https://en.wikipedia.org/wiki/PNG

Anyway, AllDup should not consider them as dups.

With the next update such images will be skipped and an error message will be displayed at the log:

ERROR: An error occurred while creating the checksum of the file 'C:\test\png\mobile png\IMG_4123.PNG': Unable to create the bitmap (FreeImage_GetBitmap error) (7)

Maybe i will find a workaround to convert the 64bit colors to the standard color model to support this image type...
Administrator
Site Admin
Posts: 4046
Joined: 04 Oct 2004, 18:38
Location: Thailand
Contact:

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by Administrator »

I found a workaround to convert the 64bit images to 32bit and now they can be used with the search method "Find similar pictures"!
Drop me a message if you want to test the workaround with all your mobile images and i will send you a new update.
mkuser
Posts: 9
Joined: 24 Feb 2023, 20:43

Re: 4.5.31 Bug: All mobile phone screenshots identified as duplicates

Post by mkuser »

Thank you for your help!
Post Reply