Not a BUG 408:

[therube]

BUG 408:
XP SP3
File Contents, no other limitations I don't believe.


Something is definitely not kosher.

\\Nas\office\COPY\_pics\My Pictures2\Vacations
&
\\Nas\office\SPRING.CLEANING\PICS\Vacations

are identical - well once I get a duplicate cleaner program to tell me so .


Stats - per directory tree:
3254 files
137 folders
2840 MB

In addition to the two directory trees being identical, there are also DUPs within each tree too.

Code: Select all

11/18/2016 03:11:32 PM - --------------------------------------------------
11/18/2016 03:11:32 PM - Search Method: File content SHA-1 (160-Bit)
11/18/2016 03:11:32 PM - Determine the file count of all source folders...
11/18/2016 03:11:34 PM - File Count: 6,508
11/18/2016 03:11:34 PM - Scan: \\Nas\office\COPY\_pics\My Pictures2\Vacations
11/18/2016 03:11:46 PM - Scan: \\Nas\office\SPRING.CLEANING\PICS\Vacations
11/18/2016 03:11:58 PM - Found 101 duplicates with a total of 53.95 MB inside folder '\\Nas\office\COPY\_pics\My Pictures2\Vacations'
11/18/2016 03:11:58 PM - Found 101 duplicates with a total of 53.95 MB inside folder '\\Nas\office\SPRING.CLEANING\PICS\Vacations'
11/18/2016 03:11:58 PM - Groups: 34
11/18/2016 03:11:58 PM - File Comparison Count: 46
11/18/2016 03:11:58 PM - Checksums created: 92
11/18/2016 03:11:58 PM - Duplicates: 202 (3%) (107.90 MB)
11/18/2016 03:11:58 PM - Elapsed time: 00:00:26

Code: Select all

11/18/2016 03:24:36 PM - Non-duplicate files:

Has 6306 entries, wrong, it should have zero.


Trying to make sense of what /is/ shown as dups... well a picture might tell it as well as I.


Returned results:



I'll take it that the top portion, 32 entries if I counted correctly, is correct, that what it shows as dups are in fact so.

Then it jumps to a file size of -0-, showing 69 items, then -0- -again, also 69 items.

Size of 0 is certainly wrong.

Edit: not entirely, see below.

There should be ~1800 or so sets of duplicates.

If one were to select all but one... without fully reviewing... when it got to the "0-byte" files, well, you're going to loose quite a bit with that...

[therube]

(Separately...

I ran the same a second time, only takes a few seconds, but this time I told it to log all files.
So the scan itself finished speedily, but the writing of the log itself, well that was quite slow.
Many, many minutes, for 1.5 MB log file.

Not complaining about that, just saying.)

[therube]

Results from 3.9.24:

Code: Select all

11/18/2016 04:05:14 PM - --------------------------------------------------
11/18/2016 04:05:14 PM - Search: File content SHA-1 (160-Bit)
11/18/2016 04:05:14 PM - Determine the file count of all source folders...
11/18/2016 04:05:15 PM - File Count: 6508
11/18/2016 04:05:15 PM - Scan: \\Nas\office\COPY\_pics\My Pictures2\Vacations
11/18/2016 04:05:26 PM - Scan: \\Nas\office\SPRING.CLEANING\PICS\Vacations
11/18/2016 04:26:02 PM - Found 3253 duplicates with 2,976,814,273 Bytes in source folder '\\Nas\office\COPY\_pics\My Pictures2\Vacations'
11/18/2016 04:26:02 PM - Found 3253 duplicates with 2,976,814,273 Bytes in source folder '\\Nas\office\SPRING.CLEANING\PICS\Vacations'
11/18/2016 04:26:03 PM - Groups: 3,169
11/18/2016 04:26:03 PM - File Comparison Count: 3,215
11/18/2016 04:26:03 PM - Checksums created: 6,370
11/18/2016 04:26:03 PM - Duplicates: 6506 (99%) (5.54 GB)
11/18/2016 04:26:03 PM - Elapsed time: 00:20:49

[therube]

Ah, in the picture above, the -0- bytes files are in fact 0-bytes, so it is correct about that part of it.

(Didn't realize that there were [any] 0-byte files. Assumed that they were some of the actually meaningful duplicates, only being reported as 0-bytes, but that is not the case.)

[Administrator]

Ah, in the picture above, the -0- bytes files are in fact 0-bytes, so it is correct about that part of it.

something i dont understand about your picture:

how do you get 2 groups with 0-byte files?

there should always only 1 group with 0-byte files.

i cant reproduce this problem here. i always get only one group with 0-byte files.

[therube]

> how do you get 2 groups with 0-byte files?

Well, if I knew that, .

But that is (at least) part of the problem.


Note that while both may be show zero byte files (& I'm presuming & they most likely are), the two directory trees are also being identified individually rather then together as you would expect.

[Administrator]

But that is (at least) part of the problem.

I dont see any other failure. What do you think is AllDup doing wrong?

[therube]

4.0.8
101 dups & 53.95 MB

Code: Select all

11/18/2016 03:11:58 PM - Found 101 duplicates with a total of 53.95 MB inside folder '\\Nas\office\COPY\_pics\My Pictures2\Vacations'
11/18/2016 03:11:58 PM - Found 101 duplicates with a total of 53.95 MB inside folder '\\Nas\office\SPRING.CLEANING\PICS\Vacations'

3.9.24
3253 dups & 2976.81 MB

Code: Select all

11/18/2016 04:26:02 PM - Found 3253 duplicates with 2,976,814,273 Bytes in source folder '\\Nas\office\COPY\_pics\My Pictures2\Vacations'
11/18/2016 04:26:02 PM - Found 3253 duplicates with 2,976,814,273 Bytes in source folder '\\Nas\office\SPRING.CLEANING\PICS\Vacations'

Same set of files.
I believe I had all the same settings - as in only a content search, no other filters.
(Cannot verify till Monday or so.)

[Administrator]

everything runs fine here with v4.0.8:

Code: Select all

19.11.2016 20:10:59 - --------------------------------------------------
19.11.2016 20:10:59 - Search Method: File content SHA-1 (160-Bit)
19.11.2016 20:10:59 - Determine the file count of all source folders...
19.11.2016 20:10:59 - File Count: 198
19.11.2016 20:10:59 - Scan: C:\test2
19.11.2016 20:11:00 - Scan: C:\test3
19.11.2016 20:11:00 - Found 99 duplicates with a total of 37,85 MB inside folder 'C:\test2'
19.11.2016 20:11:00 - Found 99 duplicates with a total of 37,85 MB inside folder 'C:\test3'
19.11.2016 20:11:00 - Groups: 52
19.11.2016 20:11:00 - File Comparison Count: 159
19.11.2016 20:11:00 - Checksums created: 174
19.11.2016 20:11:00 - Duplicates: 198 (100%) (75,70 MB)
19.11.2016 20:11:00 - Elapsed time: 00:00:01

Tested with 2 source folders having each the same identical content and file count.
Verified with the windows explorer folder properties.

I dont know what you doing....

Unzip the AllDup v4.0.8 portable edition to a new folder to avoid a misconfiguration and run your test again with "File content SHA-1 (160-Bit)".

BTW: please dont compare the latest release with older beta versions...

[Administrator]

> how do you get 2 groups with 0-byte files?

Well, if I knew that, .

Showing 2 groups with 0-bytes files is possible if you have the option "Compare only files within the same source folder" activated!!

Maybe thats your problem...

[therube]

Compare only files within the same source folder

Ah, man, don't even tell me that's what I did!
(Though given the returned results, it appears that could be a possibility!)


And if that's the case... RFE: A Bit More Detail in Logs.


I'll bet you this is pointing something out to me, that I did in fact have some option set - that I didn't realize, that was affecting my results:

Unzip the AllDup v4.0.8 portable edition to a new folder to avoid a misconfiguration and run your test again with "File content SHA-1 (160-Bit)".

Yep that's what I should have done.
That is what I did with 3.9.24.

BTW: please dont compare the latest release with older beta versions...

I only did that because I thought I was seeing a breakage.


(Man, come Monday, when I check, & when I find, it goes to be like ... oh, I can't believe I...)

[therube]

(Man, come Monday, when I check, & when I find, it goes to be like ... oh, I can't believe I...)

I can't believe!...

As you knew, as I suspected after you told me, yep, you are right, my mistake.


I can't believe!..., sigh, .

[Administrator]

The next update will show more infos about important options at the log:

Code: Select all

21.11.2016 22:25:56 - --------------------------------------------------
21.11.2016 22:25:56 - Search Method: File content SHA-1 (160-Bit)
21.11.2016 22:25:56 - Option: Compare only files between different source folders
21.11.2016 22:25:56 - Option: Ignore ID3 Tags of MP3 files
21.11.2016 22:25:56 - Option: Ignore metadata of FLAC files
21.11.2016 22:25:56 - Option: Ignore Exif metadata of JPEG files
21.11.2016 22:25:56 - File filter activated: 1
21.11.2016 22:25:56 - Folder filter activated: 6
21.11.2016 22:25:56 - Determine the file count of all source folders...
21.11.2016 22:25:56 - File Count: 25
21.11.2016 22:25:56 - Scan: C:\test\AllDup2\A
21.11.2016 22:25:56 - Scan: C:\test\AllDup2\B
21.11.2016 22:25:56 - Scan: C:\test\AllDup2\C
21.11.2016 22:25:56 - Scan: C:\test\AllDup2\D
21.11.2016 22:25:56 - Found 3 duplicates with a total of 648,29 KB inside folder 'C:\test\AllDup2\A'
21.11.2016 22:25:56 - Found 3 duplicates with a total of 648,29 KB inside folder 'C:\test\AllDup2\B'
21.11.2016 22:25:56 - Found 3 duplicates with a total of 647,40 KB inside folder 'C:\test\AllDup2\C'
21.11.2016 22:25:56 - Found 3 duplicates with a total of 647,40 KB inside folder 'C:\test\AllDup2\D'
21.11.2016 22:25:56 - Groups: 4
21.11.2016 22:25:56 - File Comparison Count: 8
21.11.2016 22:25:56 - Checksums created: 12
21.11.2016 22:25:56 - Duplicates: 12 (48%) (2,53 MB)
21.11.2016 22:25:56 - Elapsed time: 00:00:00