Crash: Error 14 - Out of string space

[StuartP]

Running AllDup (4.5.54) checking on for duplicates on a mapped network drive. First time I ran it I used the default file name only option which completed successfully but realised the results were not what I needed, took around 6 hours to check all files. I have now modified the search to "filename, extension and size" but have as yet not managed to complete it. On two attempts carried out it reached 87% and displays a message with "Error 14 - Out of string space". Clicking on the message closes the application.

Running on HP 250 laptop, 64 bit Windows 10 Home (22H2) with 16Gb memory.

Is there any logging I can configure to capture the reason for the error, how do I fix it?

The previous search with only the filename match advises in the log:
Search aborted - the maximum available amount of memory for 32-bit applications has been reached
Found 735,279 duplicates with a total of 912.02 GB inside folder 'T'

There is no info in the log for the error 14 abort.

Stuart

[Administrator]

It looks like that the AllDup process was running out of memory and AllDup couldnt detect it before it was too late.

You can use the option "Log search statistics of the displayed progress info" to log the progress of the scan.

I guess the logged progress of your scan can help us to extend the internal low memory detection to abort the search before its too late.

[therube]

So AllDup is x86, so (the process is) limited to ~2GB (or possibly ~4GB, depending on "large address" - which seemingly it is) of RAM.
(I actually never even considered its' bit-ness before.)

Running a scan, some 400K files, AD is using ~750 MB of RAM.
And if RAM usage goes up (somewhat) linearly with the total number of files scanned, quit conceivable that (single process) RAM could be exhausted.

Would "just create an x64 version" alleviate part of that issue (for 64-bit systems, of course)?

[StuartP]

I tried running the search again, I'm getting a different error report that happens at exactly the same point.

When the error is displayed clicking save closes the dialog and the full application, I was able to copy and paste the window contents into a text file.

Now: 17/11/2023 21:10:41
App: AllDup 4.5.54
ExeInst: 12/11/2023 23:31:46
IniCreate: 12/11/2023 23:32:01
Modul: basMain
Func: LogProtokoll
Line: 130
Err: -2147417848 - Automation error
The object invoked has disconnected from its clients.
Path: C:\Program Files (x86)\AllDup\AllDup.exe
AppUp: 3:50:41
SysUp: 4 days 21:32:35
OS: Windows 10 (Build 19045) Version: 10.0 Build: 19045 ID: 2
64bit: True
IE: 9.11.19041
PhysRAM: 8,872.21 MB Free / 16,299.39 MB Total
MemLoad: 45%
RAM used: 1,752.89 MB
Pagefile: 10,802.35 MB Free / 18,731.39 MB Total
VirtMem: 13.30 MB Free / 2,047.88 MB Total
CPU: GetCPUstring2: -2147221020 - Automation error
Invalid syntax (Line: 20)
Screen: 1366x768xTrueColor (4294967296)
Portable: False
User: stuar
Comp: DESKTOP-QJRKFVO
FormCount: 3
FormNames: frmMain,frmDupSearch,frmPreview,
Apptitle: AllDup 4.5.54
Datapath: C:\Users\stuar\AppData\Roaming\AllDup
Admin: False
Errcount: 1
Lang: 9
Stop:0,0
Unicode:0
SM:1110000
Sb1:
Sb2:
Sb3: Groups: 0
Sb4: Files: 0 \ 0
Debug0: \\?\C:\Users\stuar\AppData\Roaming\AllDup\temp\409315453T9.txt
Debug1: Duration : 03:31:05, Progress : 86%, File : 911,646 / 1,059,423, 669,074 duplicates found, 202,262 groups created, 911,646 file comparison performed, - checksums created, - archives scanned, - files extracted, - files filtered, - folders filtered, RAM: 1,837,977,600

[Administrator]

Based on your bug report, we have made a few changes to detect the high memory usage earlier. Check your private messages to download an update for testing the changes.

[Administrator]

New improvement with the next update:

Under Windows 64-bit, AllDup can now use up to 4 GB of RAM to search for duplicate files (previously a maximum of 2 GB).

Example log of a scan with 3 million files:

Code: Select all

22.11.2023 19:25:26 - Search method: File name + File extension + File size
22.11.2023 19:25:26 - Comparison method: Compare all characters of a file name
22.11.2023 19:25:26 - 1.Source folder: E:\testfiles
22.11.2023 19:25:26 - 2.Source folder: E:\testfiles1
22.11.2023 19:25:26 - 3.Source folder: E:\testfiles2
22.11.2023 19:25:26 - Option: Compare files from all source folders
22.11.2023 19:25:26 - Determine file count of all source folders...
22.11.2023 19:25:27 - File count: 3.000.002
22.11.2023 19:25:27 - Duration: 00:00:00, Progress: 0%, File: 0 / 3.000.002, 0 duplicates found, 0 groups created, 0 file comparisons, RAM usage: 59,14 MB (62.017.536)
22.11.2023 19:25:27 - Scan: E:\testfiles
22.11.2023 19:25:39 - Scan: E:\testfiles1
22.11.2023 19:25:43 - Scan: E:\testfiles2
22.11.2023 19:26:27 - Duration: 00:01:01, Progress: 72%, File: 2.165.515 / 3.000.002, 331.028 duplicates found, 165.514 groups created, 2.165.515 file comparisons, RAM usage: 1,68 GB (1.802.637.312)
22.11.2023 19:27:27 - Duration: 00:02:01, Progress: 80%, File: 2.393.822 / 3.000.002, 787.642 duplicates found, 393.821 groups created, 2.393.822 file comparisons, RAM usage: 2,82 GB (3.033.317.376)
22.11.2023 19:27:50 - Duration: 00:02:24, Progress: 83%, File: 2.480.487 / 3.000.002, 960.972 duplicates found, 480.486 groups created, 2.480.487 file comparisons, RAM usage: 3,26 GB (3.500.449.792)
22.11.2023 19:27:50 - The search was canceled because the maximum memory usage for 32-bit applications under Windows was reached.
22.11.2023 19:27:50 - Found 480.487 duplicates with a total of 3,56 MB inside folder 'E:\testfiles'
22.11.2023 19:27:50 - Found 480.485 duplicates with a total of 3,56 MB inside folder 'E:\testfiles2'
22.11.2023 19:27:53 - Scanned files: 2.480.487 / 3.000.002
22.11.2023 19:27:53 - Groups: 480.486
22.11.2023 19:27:53 - File comparisons: 2.480.487
22.11.2023 19:27:53 - Duplicates: 960.972 (32%) (7,12 MB)
22.11.2023 19:27:53 - Elapsed time: 00:02:24

[therube]

AllDup can now use up to 4 GB of RAM

Just to note that is now included as of the 4.5.56 release.

App can handle >2GB addresses