Large Sheets of Merged Pages Freeze App

Forum for the PDF-XChange Editor - Free and Licensed Versions

Moderators: Tracker Support, TrackerSupp-Daniel, Sean - Tracker, Paul - Tracker Supp, Vasyl-Tracker Dev Team, Chris - Tracker Supp, Ivan - Tracker Software, Tracker Supp-Stefan

Post Reply
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Large Sheets of Merged Pages Freeze App

Post by PHK »

Why is it that PSF-XCE cannot really handle large sheets of merged pages? Particularly it seems if there have been a few page splitting and merging operations? That's when the app can freeze of many seconds or even minutes.

When I write "large" sheets, I mean it could be many columns of many A4/Letter pages and it might be a sheet +/- 100 inches wide by high.

Is there anything the user can do to mitigate? I just tried rasterizing a page but the process crashed out before completion. Optimizing doesn't help either.
All best,

FringePhil
User avatar
Paul - Tracker Supp
Site Admin
Posts: 6988
Joined: Wed Mar 25, 2009 10:37 pm
Location: Chemainus, Canada
Contact:

Re: Large Sheets of Merged Pages Freeze App

Post by Paul - Tracker Supp »

Hi PHK,

there are two approaches we can take,:

1. We can try and reproduce the issue here. That would mean you sending us the source documents and a description of the process used and us seeing if the same happens here.

2. We can look at a memory dump. If you take a memory dump while the Editor is frozen we may be able to glean some information from that about what is happening while it is frozen.

I would prefer the second option if you can manage it. Essentially you just right click the Editor in the Task Manager while it is "Not Responding" and select "Create dump file" and send it to us.
image.png
We have a file upload service if the dump is too large for email: https://www.pdf-xchange.com/knowle ... le-service
Best regards

Paul O'Rorke
Tracker Support North America
http://www.tracker-software.com
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Thanks, Paul. See below.
Paul - Tracker Supp wrote: Tue Jan 31, 2023 8:52 pm ...
1. We can try and reproduce the issue here. That would mean you sending us the source documents and a description of the process used and us seeing if the same happens here.
I don't really want to contaminate you with the sensitive file that is causing me the agita today so how about #2?
Paul - Tracker Supp wrote: Tue Jan 31, 2023 8:52 pm 2. We can look at a memory dump. If you take a memory dump while the Editor is frozen we may be able to glean some information from that about what is happening while it is frozen.

I would prefer the second option if you can manage it. Essentially you just right click the Editor in the Task Manager while it is "Not Responding" and select "Create dump file" and send it to us....
I put it in my OneDrive and I will send you a link via e-mail. I don't think the rest of the community would want to see it. If that doesn't work, I will send it on your site.

Thanking you in advance,

PHK
All best,

FringePhil
User avatar
Paul - Tracker Supp
Site Admin
Posts: 6988
Joined: Wed Mar 25, 2009 10:37 pm
Location: Chemainus, Canada
Contact:

Re: Large Sheets of Merged Pages Freeze App

Post by Paul - Tracker Supp »

I have it.

Thanks PHK.

Roman will look at this when he gets back from lunch. ;)
Best regards

Paul O'Rorke
Tracker Support North America
http://www.tracker-software.com
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

Hi PHK,

The dump you have provided does not show any signs of freeze - it looks like the Editor is in its normal idle state, waiting for user input.
Please open Windows Task Manager, go to Details tab, make sure the "CPU" column is visible, and locate the Editor process (PDFXEdit.exe) in the list. Then reproduce this freeze. During the freeze please monitor the "CPU" column value for the Editor process. What will be the min/max/average values (just a rough estimation)?

Also please try to disable table detection (Editor preferences->Page text->Detect tables by...):
image.png
As of the application crash during the rasterization - please provide us with a crash dump (see here for details - the section about crash dumps - not process dumps).

Thanks.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Tue Jan 31, 2023 10:40 pm ...
What will be the min/max/average values (just a rough estimation)?
9.5%/11.5%/10.1%. It hit the 11.5% immediately before coming out of the freeze.
Roman - Tracker Supp wrote: Tue Jan 31, 2023 10:40 pm Also please try to disable table detection
Table detection has been disabled throughout.
Roman - Tracker Supp wrote: Tue Jan 31, 2023 10:40 pm As of the application crash during the rasterization - please provide us with a crash dump ...
No time now; maybe later.
All best,

FringePhil
User avatar
Paul - Tracker Supp
Site Admin
Posts: 6988
Joined: Wed Mar 25, 2009 10:37 pm
Location: Chemainus, Canada
Contact:

Re: Large Sheets of Merged Pages Freeze App

Post by Paul - Tracker Supp »

At your convenience.

:)
Best regards

Paul O'Rorke
Tracker Support North America
http://www.tracker-software.com
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

P. S. Just to clarify the freezing does not happen when rasterizing; it is doing simple things like trying to move the document window focus to another part of the document, panning, etc.
All best,

FringePhil
User avatar
Paul - Tracker Supp
Site Admin
Posts: 6988
Joined: Wed Mar 25, 2009 10:37 pm
Location: Chemainus, Canada
Contact:

Re: Large Sheets of Merged Pages Freeze App

Post by Paul - Tracker Supp »

Got it. Roman is working on that right now...

:)
Best regards

Paul O'Rorke
Tracker Support North America
http://www.tracker-software.com
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

Thanks PHK,

Can you please roughly estimate the total number of merged pages per page of the final document in your case?
I was able to reproduce some freeze (around 20 seconds), in my case there were 600 merged pages (20x30 grid) per 100x100 inches final page.
We will discuss the issue here to figure out what we can do. However this is a very fringe case (huge amount of text per page), and other PDF editing applications (like Adobe Acrobat) also stumble badly when handling such documents.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Thank you for looking into this, Roman. In today's vexatious sheet, there are about 18 columns populated with about 100 Letter-size pages but not a huge amount of text, although roughly 1/3 have colored backgrounds. All are 1:1, not resized. The sheet is 144"x 88". Of course, I have many other files that are even more complex and dense and with which I have more freezing problems. This is what it looks like, zoomed out.
image.png
image.png (39.6 KiB) Viewed 3887 times
One other consideration: This page is one of over 500 pages in the file some of which are also large sheets. File size does seem to be an adverse factor. It might be that if the user isolates these monster sheets to single-sheet files the problem might be mitigated. Not a solution but a possible work-around.

I accept that my use might be a little "fringe" today but the future for digital document management has to migrate away from the wasteful legacy physical constraints of A4/Letter-sized document pages. Your platform is frustratingly close to providing the functionality for users to do that. If you don't embrace the possibilities, someone else will. And the problem with enlarging your customer base is that you always will risk riling up some "fringe" users. I, personally, enjoy being a fringe disrupter.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

Hi PHK,

By qualifying this case as "fringe" I did not mean you a "fringe user". :) On the contrary, we are grateful to you for revealing such a bottleneck in our product.
I have talked about this case with my colleagues and it appeared that the Editor component that is the main contributor to this freeze, is currently under full redesign. We expect the revamped code to be included in the next major version of the Editor (version 10). The release is expected this spring.
In the meantime I have added some minor optimizations to that code. This optimization is a candidate for the next maintenance build of version 9 of the Editor (the release is expected in a few weeks).
You can try it out right now - please download the minimal working set of pre-release binaries of the Editor here.
If you have installed the latest version of the Editor (9.5.366), you can replace the corresponding binaries in the Editor installation directory. Please make backups of the original binaries first.

Thanks again for being our customer. :)
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Sat Feb 04, 2023 3:31 am ...
By qualifying this case as "fringe" I did not mean you a "fringe user".
No offense taken; I am just trying, as an unsanctioned beta tester, to help you improve the product, albeit with a slightly selfish motive: I want it to work better FOR ME. Your acknowledgement of my efforts in that is appreciated. I am relieved that I have put my thumb on a legitimate weakness in the app and that it is being addressed.
Roman - Tracker Supp wrote: Sat Feb 04, 2023 3:31 am
You can try it out right now - please download the minimal working set of pre-release binaries of the Editor [url=https://files.tracker-software.support:8443/wl/?id=LPR89ADPTlZ0qTw8Ln4gKhyycB8o8SSK]here[/url].
If you have installed the latest version of the Editor (9.5.366), you can replace the corresponding binaries in the Editor installation directory.
Sorry, I don't know what to do with those files in the download zip file. Simply replacing the earlier files in the installation directory did not work for me.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

PHK wrote: Sat Feb 04, 2023 2:53 pm Simply replacing the earlier files in the installation directory did not work for me.
Have you installed the latest (9.5.366) version of the Editor?
If not, you can run this binary set as a standalone application (with limited functionality) - just unpack it to an empty directory and double-click the PDFXEdit.exe file. But make sure you have closed your regular Editor - otherwise that standalone version will not start but just switch you to the running Editor instance.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

It seems that I've got it working -- if it is 366.1...

Initial impression: Scrolling seems faster with less freeze. The troublesome page of my screenshot above (single page in the file) still freezes on loading for about 5 minutes.

I am not sure what I should be looking for in improvements. It may be too early for useful feedback. I will continue to use this 366.1 for a few days to see if anything pops.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

PHK wrote: Sat Feb 04, 2023 7:58 pm The troublesome page of my screenshot above (single page in the file) still freezes on loading for about 5 minutes.
5 mins! This is totally unacceptable... :( I now fully understand your disappointment.

Probably my assumption about the cause of this freeze is wrong. Can you please provide us with a test document that causes such a long freeze? With sensitive information removed of course.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Well, this problematic page has privacy information relating to hundreds of people besides myself. I am not sure if I can give you a non-sensitive example but I will think about how to do that.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

OK, thanks anyways.
Probably we can prepare a set of tools and a simple to use script for you to monitor the Editor during that freeze and collect performance data that we can analyze here locally. Hopefully next week.
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

Just to be sure - I assume you have this freezing issue on the same hardware you already reported to us in another topic (Intel i7 11700KF, 16 GB RAM).

Also please measure the opening time of this test pdf file, just to have something in common to compare. Please open this file from your local drive.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Sat Feb 04, 2023 10:07 pm Just to be sure - I assume you have this freezing issue on the same hardware you already reported to us in another topic (Intel i7 11700KF, 16 GB RAM).
Yes, same equipment throughout.
Roman - Tracker Supp wrote: Sat Feb 04, 2023 10:07 pm
Also please measure the opening time of [url=https://www.dropbox.com/s/34ezuns1gxgw0h1/Ecma-262-merged-100x100in.pdf?dl=0]this test pdf file[/url], just to have something in common to compare. Please open this file from your local drive.
This file opens in a thoroughly satisfactory time; too quick to bother timing. If all my files opened that well, I would have no issues.

However, there are a lot of differences between your reference file and the sort of file that cause my problems. My problem files tend to be with multi-page merge-ups and perhaps with page-splits and re-merges and from scanned sources, rather than down-loads. Down-load pages, too, but to a lesser extent. A very noticable difference is, whereas the reference file has a nice, simple, flat Content heirarchical structure, my bad files have a "deep" and complex structures where there are many levels of XForm elements. See below.
image.png
The next level of the heirarchy, or 'tree', is where images or text are.

I have no idea what goes into creating the Content layer or the significance of the heirarchical structure; this is just my simple observation.
All best,

FringePhil
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

There is one other problem with merging pages into a single sheet that is quite persistent: clicking on single lines in the Content panel to change the focus in the document view window to the location of the Content element fails after merging pages. That is, if you click on a line of text in the Content panel, the area in view in the document window will not reliably be where that text is.

It is not just my problem files, I see the same thing in the ECMA file you offered for download, Roman. If, for instance, I click on "238" the document shifts to an area of the document that has nothing to do with, nor is proximate to, any of the three locations on the sheet where that digit sequence can be found.
All best,

FringePhil
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Sat Feb 04, 2023 9:45 pm ...
Probably we can prepare a set of tools and a simple to use script for you to monitor the Editor during that freeze and collect performance data that we can analyze here locally. Hopefully next week.
I will be de-activating this particular PC tomorrow and, later in the week, activating a different PC in a different location for perhaps as long as three months. I will be accessing all my usual files via OneDrive in my alternate location and I fully expect the same issues. But you might want to wait until I am settled there before expending too much effort in preparing these tools. I can advise when appropriate.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

Thanks PHK.
Please let us know when ready.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Will do.

In the meantime, anything to say about what I wrote above?
PHK wrote: Sun Feb 05, 2023 3:14 pm There is one other problem with merging pages into a single sheet that is quite persistent: clicking on single lines in the Content panel to change the focus in the document view window to the location of the Content element fails after merging pages. That is, if you click on a line of text in the Content panel, the area in view in the document window will not reliably be where that text is.

It is not just my problem files, I see the same thing in the ECMA file you offered for download, Roman. If, for instance, I click on "238" the document shifts to an area of the document that has nothing to do with, nor is proximate to, any of the three locations on the sheet where that digit sequence can be found.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

As of the desynchronization issue between Content panel items and the document view after page merging - I passed this information to another developer who has more expertise in this.
As of the main issue (freezing) - I have some ideas, but it would be better to delay the conclusion until we can collect the actual performance data from your machine. The only thing I can be sure of so far is that my initial conclusion about the cause of the freezing was wrong.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Mon Feb 06, 2023 9:14 pm As of the desynchronization issue between Content panel items and the document view after page merging - I passed this information to another developer who has more expertise in this.
OK, good and thank you for the fancy terminology.
Roman - Tracker Supp wrote: Mon Feb 06, 2023 9:14 pm As of the main issue (freezing) - I have some ideas, but it would be better to delay the conclusion until we can collect the actual performance data from your machine. The only thing I can be sure of so far is that my initial conclusion about the cause of the freezing was wrong.
Nothing "wrong" about eliminating an invalid theory.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

The implemented optimizations will be useful for other cases (like the "ECMA-262-..." document above). But in your specific case the "bottleneck" is something else. Hopefully we will be able to identify and address it too. Thank you, PHK for your patience and useful feedback.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

If you want to stress-test a file, take your ECMA-262 file and duplicate that one page to four and then merge them into a 200"x200" single 4x4 sheet. It took a while to open on my low-cost laptop but everything is there and functional. Not the desynchronized Content; that is still out of whack.

But once loaded, it navigates fine unlike some of my Frankenstein files.
All best,

FringePhil
User avatar
Paul - Tracker Supp
Site Admin
Posts: 6988
Joined: Wed Mar 25, 2009 10:37 pm
Location: Chemainus, Canada
Contact:

Large Sheets of Merged Pages Freeze App

Post by Paul - Tracker Supp »

:)
Best regards

Paul O'Rorke
Tracker Support North America
http://www.tracker-software.com
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

And if you were thinking it might be overall file size, go even more perverse and duplicate that page until you have, say, 16 200x200 pages and it still navigates OK.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

We are focused on improving the Editor performance for "useful" cases. I mean documents created with a useful purpose in mind. It is no problem to create a 'strange' document that contains unbearable amount of text, images, etc. that will cause problems to any PDF software. Moreover, optimizing the Editor for such 'fringe' cases does not necessarily result in better performance in useful cases.

As I can understand, your documents that take a lot of time to open in the Editor, were created by another PDF management software for a practical purpose and are supposed to open in a reasonable time. And you are working with them on a daily basis. Am I correct?
If it is so, it makes sense for us to optimize the Editor for your cases.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Wed Feb 08, 2023 6:53 pm... your documents that take a lot of time to open in the Editor, were created by another PDF management software for a practical purpose and are supposed to open in a reasonable time. And you are working with them on a daily basis. Am I correct? ...
No, the documents were created all in PDF-XCEdit in the sense that most of the original content was downloaded files from trusted sources but all the merging and splitting was done in PDF-XCEdit. I stopped using inferior PDF management software a long time ago.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

So, I think it is possible that the problematic document you have created is naturally too complex to be opened in a reasonable time. At least we cannot be sure, as you did not try to work with it in another PDF editor, correct?
Regardless of the optimization efforts we could undertake, it will always be possible to create an unmanageable PDF document.

We have added a few more optimizations, you can download the updated version (366.2) here. Please try it with your problematic files.
However, I am not sure if this update addresses your case, before running some instrumented performance tests on your machine with your files.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Fri Feb 10, 2023 3:34 am ...as you did not try to work with it in another PDF editor, correct?
Correct.
Roman - Tracker Supp wrote: Fri Feb 10, 2023 3:34 am
We have added a few more optimizations, you can download the updated version (366.2) [url=https://files.tracker-software.support:8443/wl/?id=BZFCBthdBIQ25EUXSalCdbaIM5mYw7ci]here[/url]. Please try it with your problematic files.
...
I will try.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

Thanks PHK.
It appeared that the build 366.2 introduces a bug. Please download the updated build 366.3 here.
Please be aware this is also a pre-release build and can be unstable.
User avatar
PHK
User
Posts: 1028
Joined: Tue Nov 24, 2020 4:02 pm

Re: Large Sheets of Merged Pages Freeze App

Post by PHK »

Roman - Tracker Supp wrote: Sat Feb 11, 2023 12:36 am Thanks PHK.
It appeared that the build 366.2 introduces a bug. Please download the updated build 366.3 [url=https://files.tracker-software.support:8443/wl/?id=8kzSCnNJecluUyqzDkQokiIS9BCC4g8z]here[/url].
Please be aware this is also a pre-release build and can be unstable.
After taking about twenty minutes to open, the Frankestein page does perform appreciably better in 366.3 on my laptop, but still not 'perfectly.' So, directionally this represents real progress. This is not a huge file (single page, about 166k) and I cannot understand why it would take so long to open. I tried to duplicate the page by extracting it and that may have helped a tiny bit but it was still unacceptably slow to load. Note that I will not be using my main PC for perhaps several months but I don't think these problems I am experiencing are necessarily machine-related.

Thanks for your efforts, Roman, in trying to improve this.
All best,

FringePhil
User avatar
Roman - Tracker Supp
Site Admin
Posts: 312
Joined: Sun Nov 21, 2004 3:19 pm

Re: Large Sheets of Merged Pages Freeze App

Post by Roman - Tracker Supp »

This is terribly slow.
So we definitely need to run an instrumented performance test on your machine - there is no point to guess what is the bottleneck in your case.
I have emailed you the further instructions.
Post Reply