The search was aborted

[OFX]

"The search was aborted because Windows cannot allocate any more memory for the search."

Just wondering if I am doing something wrong here?

Pc specc:
- AMD Ryzen 9 5950X 16-Core Processor 3.40 GHz
- RAM 128 GB
- Windows 10 Pro 64-bit

Logg:

11.12.2023 19:26:29 - AllDup 4.5.56
11.12.2023 19:26:29 - Search method: File content
11.12.2023 19:26:29 - Comparison method: SHA-2 (256-Bit)
11.12.2023 19:26:29 - Option: Use database
11.12.2023 19:26:29 - 1.Source folder: Y:\Folder1
11.12.2023 19:26:29 - 2.Source folder: Y:\Folder2
11.12.2023 19:26:29 - 3.Source folder: Y:\Folder3
11.12.2023 19:26:29 - 4.Source folder: Y:\Folder4
11.12.2023 19:26:29 - 5.Source folder: Y:\Folder5
11.12.2023 19:26:29 - 6.Source folder: Y:\Folder6
11.12.2023 19:26:29 - 7.Source folder: Y:\Folder7
11.12.2023 19:26:29 - Option: Compare files from all source folders
11.12.2023 19:26:29 - Folder filter activated: 7
11.12.2023 19:26:29 - Filter type: Exclusive
11.12.2023 19:26:29 - 1.folder filter: c:\windows
11.12.2023 19:26:29 - 2.folder filter: c:\program files (x86)
11.12.2023 19:26:29 - 3.folder filter: c:\program files
11.12.2023 19:26:29 - 4.folder filter: ?:\system volume information
11.12.2023 19:26:29 - 5.folder filter: ?:\recycled
11.12.2023 19:26:29 - 6.folder filter: ?:\recycler
11.12.2023 19:26:29 - 7.folder filter: ?:\$recycle.bin
11.12.2023 19:26:29 - Determine file count of all source folders...
11.12.2023 21:04:16 - File count: 74 422 194
11.12.2023 21:04:16 - Scan: Y:\Folder1
11.12.2023 23:38:06 - The search was aborted because Windows cannot allocate any more memory for the search.
11.12.2023 23:38:06 - Scanned files: 2 750 101 / 74 422 194
11.12.2023 23:38:06 - File comparisons: 893 442 235
11.12.2023 23:38:06 - Checksums created: 2 727 680
11.12.2023 23:38:06 - Database checksums stored: 2 727 680
11.12.2023 23:38:06 - 1784954 folders and 2750101 files were searched but no duplicates found.
11.12.2023 23:38:06 - Elapsed time: 04:11:37

[Administrator]

First of all, you did nothing wrong.

For the number of files to be checked, AllDup would have to save them in a database on the hard drive, which has the disadvantage that the search for duplicates would be significantly slowed down.

Currently AllDup keeps the information about the scanned files in the memory to compare them faster but there is a memory usage limit for 32bit application under windows and you reached this limit.

[OFX]

I see in task manager that i am running
- AllDup (32 bit)

Not sure why this is since im on a 64 bit system.

Thank you for clarifying the issue.

[Administrator]

Not sure why this is since im on a 64 bit system.

Because AllDup is a 32 bit application.

[therube]

Comparison method: SHA-2 (256-Bit)

Since an SHA-2 is a physically longer hash, then say a MD5 hash, would that make a difference in memory usage, simply because the hash is longer, & in that regard if you used a MD5 comparison instead, there would be a longer period of time before a (potential) OOM (out of memory) condition might (possibly) occur?

[Administrator]

This is no longer so important when you consider the total number of 74,422,194 files to be checked...

[therube]

Right. I understand that is a huge number of files.

But is my supposition correct, that using a hash that generates a physically small hash - number of bytes, will lessen memory requirements, thereby potentially allowing a longer run before an OOM condition may hit?

[Administrator]

That's right, MD5 has the lowest memory consumption.
MD5 needs only 16 bytes to store the hash value and SHA512 needs 64 bytes.
The order for the all hash array sizes are 16, 20, 32, 48 and 64 bytes.