Page 1 of 1

Ignore metadata bug

Posted: 23 Jan 2020, 01:49
by Evds
Alldup 4.4.14

Download a file from a Sony A6500 (for example: https://3.img-dpreview.com/files/p/samp ... 051969.jpg).
Make a copy of that file in the same directory and add an IPTC keyword to one of the files.

Start Alldup and search for "file content" and "ignore metadata".
Alldup will not see them as duplicates.

Re: Ignore metadata bug

Posted: 23 Jan 2020, 09:17
by therube
Confirmed - with that file.

Other files I looked at (not from your source link, with purposely changed IPTC data) compared as expected.

(I used IrfanView to write IPTC title to the files I modified.)

Re: Ignore metadata bug

Posted: 23 Jan 2020, 09:34
by Administrator
We could reproduce and fix the problem using your example pic.
The problem are the many & unnecessary white spaces at the end of your file.
The current version of AllDup checks only the last 64 bytes of a jpeg file to find the end marker of the picture stream.
We have extended the search for the end marker to the last 4096 bytes.
Please send me an email if you need a new update with the fix.

Re: Ignore metadata bug

Posted: 24 Jan 2020, 15:43
by Evds
I know, I have no idea why Sony adds those trailing bytes.
I have photos (unedited, straight out of the camera) with more then 22000 bytes after the EOI marker.
In that case extending the search for the EOI marker to 4096 bytes isn't enough.
Is it possible to make that a configurable field (so I can increase it if necessary) ?

Thanks for investigating this problem !

Re: Ignore metadata bug

Posted: 24 Jan 2020, 20:48
by Administrator
We will extend the search range at the end of the file to 64k bytes.

Re: Ignore metadata bug

Posted: 01 Feb 2020, 00:58
by Evds
I noticed that after the scan data in the jpeg file (after the SOS and EOI marker) there are some trailing bytes and after those trailing bytes a new image starts (the preview image which is larger than the thumbnail) with a new SOI, SOS and EOI marker and then again some trailing bytes.

Re: Ignore metadata bug

Posted: 01 Feb 2020, 10:28
by Administrator
I guess thats the thumbnail data. See "File format structure" at https://en.wikipedia.org/wiki/JPEG_File ... nge_Format