Clear OCR Results?

[PHK]

Is there a way to clear OCR results so that a new fresh start OCR session can be executed without residue from previous sessions?

[Paul - PDF-XChange]

Hi, PHK

what kind of "residue" are you referring to?

I was under the impression each time you start the tool it starts afresh.

[PHK]

...

what kind of "residue" are you referring to?

...

Hi, Paul. I am referring to ghost content. Earlier today, I downloaded five pages that I wanted to merge into one vertical page but I wanted to strip out redundant headers and footers. To do that, I split the page above the footers and below the headers and then deleted the unwanted header and footer pages. Then, I re-merge the remaining pages. But there are now five sets of duplicate content. In a way that doesn't matter too much until you do something like a search. Then, I get five times as many hits as I want, most unrelated to the image on the page.

So, how do I eliminate all that redundancy? A simple re-run of the OCR does not do much; certainly it doesn't clear the redundancies. If there were a way of clearing the text on the hidden content layer, I might be able to OCR and get a 'clean' content layer.

Any thoughts would be helpful.

[Daniel - PDF-XChange]

Hello, PHK

The only way this could be happening is if you are using the "searchable image" function, instead of the "Editable text" function (since editable text will replace any duplicates in that location), so all of the previous text information should be invisible at this time.

As such, simply right click on the page and choose "select > text" to select all instances of base content text on the page, then push the "delete" key.

image.png

If your documents were originally image based, this will leave the image layer perfectly intact. Do ensure you double check your docuemtns, on the off chance that one or more of them was not image based, as you may have just deleted actual visible text content, which will not reapper when performing OCR.

Alternatively, if you have our EOCR, you can use the editable image function to overwrite this text automatically.

Kind regards,

[PHK]

Thank you, Daniel.

You are exactly right. As the app evolved, I must have missed the significance of the introduction on EOCR and was basically ignoring the additional modes. So yes, I was using OCR in Searchable image mode rather than Editable text. Without delving into how these different modes functioned, I was inferring that the former was what I needed because what I was doing was not really editing text. In fact, just the opposite. I specifically did not want to edit the text, I just wanted to re-assembled pages without disturbing the text but being able to search and highlight text and create Bookmarks and Links. In other words, the short titles of these modes were counter-intuitive to me.

This will be very helpful to me and I can see how it will save me a great deal of aggro going forward.

[Stefan - PDF-XChange]

[PHK]

Having said all of the above, I now find that my problem is not exactly solved. I want an OCR result that has text content perfectly alligned with the visible image completely unchanged. None of the Output Options in EOCR seem to achieve that.

Here is an example. It is a large sheet (approximately 100"x45") that is an assemblage of 50 to 70 pages. Each column can be one or more pages, themselved the product of the Vertical Merge Pages function.

image.png

The content layer is an absolute mess.

image(3).png

Using the FIne Page Content Output Option, I get a decent-looking content layer

image(1).png

but the visible image has been altered in a way I find unacceptable.

image(2).png

Using the Editable Text and Images Output Option, I get a very weird and unusable content layer, although the visible image seems OK.

image(4).png

What is all that noise off the sheet and even some on the sheet?

Advice or comments?

[Daniel - PDF-XChange]

Hello, PHK

If the results are that out there... there may be something wrong with the document/page structure. Would it be at all possible to send us a copy of one of these pages?
Beyond that however, as you said you do not need to edit the files, only search them, since searchable image is working fine, that will likely be the best option for you, simply select and delete all of the existing invisible text content, and then run the OCR with searchable image again, resulting in only one layer of text.

Kind regards,

[PHK]

Is there a way to bulk-select invisible content for deletion?

[Daniel - PDF-XChange]

Hello, PHK

Unfortunately, not for base content... Perhaps we should consider adding some sorting options to the Content pane, similar to what the comments pane offers.
For now though, I suppose another option would be to use "rasterize pages" on the convert tab. That would have the side effect of recreating everything into a single large image however, and could cause your file size to increase notably.
Beyond that, a "blink test" by using "select text > delete" and then checking different areas of the document, and performing an undo/redo to confirm that nothing is missing could work?

Kind regards,

[PHK]

I think, Daniel, buried in your response above is the answer to the original question.

And that key word is "rasterize." Therefore, if I

* rasterize the page before I OCR it,
* copy the created image entry in the Content panel,
* EOCR with the third Output Option selected,
* paste the image onto the visible page, and
* make sure the image entry in the Content panel is a the top of the page stack of content.

Does this work?

[Daniel - PDF-XChange]

Hello, PHK

I suppose it would work, but you have a few unnecessary extra steps.

Your process could be as simple as
1. rasterize pages (with the option to flatten all text content enabled)
2. perform "searchable image" ocr.
There is no need for using the 3rd option in the OCR dialog, nor selecting and copy/pasting the images, infact that portion would likely cause more issues with file bloat.

The reason I did not recommend this at the beginning is because rasterize can impact your image quality, and vastly increase file size depending on the previous image compression methods that were used. Do ensure you are working with a duplicate, and that the output results are acceptable before you continue using this method.

Kind regards,

[PHK]

...
I suppose it would work, but you have a few unnecessary extra steps.

Well, it works best for me. I want EXACTLY the same Image layer in Content. See the comparison below.

Your process could be as simple as
1. rasterize pages (with the option to flatten all text content enabled)

I don't see this option

2. perform "searchable image" ocr.
There is no need for using the 3rd option in the OCR dialog, nor selecting and copy/pasting the images, infact that portion would likely cause more issues with flie bloat.

The reason I did not recommend this at the beginning is because rasterize can impact your image quality, and vastly increase file size depending on the previous image compression methods that were used. Do ensure you are working with a duplicate, and that the output results are acceptable before you continue using this method.

image.png

Your way makes subtle changes to fonts and spacing in the visible image that are unaccetable to me. An astute viewer may notice the difference and wonder "if this part of the visible content has been editted, what else?" That would undermine the integrity of the entire process. Focus on "J. Story Commentaries" in the middle of the screenshot. Granted, some users might find this acceptable depending on their objectives.

Also, the Searchable Image Output Option produces thoroughly unacceptible results on this document and, therefore, I do not trust it. The only option of the three that work for me on this document is Fine Page Content.

And I wonder why the Image of the page that is the first entry under Page 1 in the Content panel does not survive the EOCR process? I find it very useful to have that Image layer at the top of the Content stack for the page. Therefore, my copy and pasting steps that you consider redundant, Daniel. But I find that the safest way to do all this is to elect the Create a New Document Output Option. Then, I copy the Image Content entry in the pre-EOCR document and paste into into the post-EOCR document at the top of the page stack.

And if you are worried about "bloat," have no fears. As you can see from the respective Document Properties, doing this substantially reduces file size and, may I say, bloat.

Finally, I am not concerned about the effects of rasterization image quality. In these cases, the documents are 100% text which rasterization seems to handle well.

But all of this is learning. It is challenging to explore new uses of the PCF-XCE application and I still have not run out of new ways to use it.

[Stefan - PDF-XChange]

Hello FringePhil,

Happy to hear that you are still finding new ways to use our products!
Are you using the Enhanced OCR engine or the Tesseract one (Standard) - the Standard will have less options than the Enhanced one.

And if your approach is giving you the results you need - shall we consider this topic resolved or is there anything else we can assist with?

Kind regards,
Stefan

[Daniel - PDF-XChange]

Hello,

Your way makes subtle changes to fonts and spacing in the visible image that are unaccetable to me.

From your screenshot, you seem to still be using the "editable images" option, however you would want to use the "searchable image" option in this process. Searchable image will not make any changes to existing page content, this is why the "rasterize pages" function is needed manually, to remove the excess invisible text you have added from multiple re-applications of OCR.
And to answer your earlier question, the Rasterize pages command is located on the Convert tab as well:

image.png

Also, the Searchable Image Output Option produces thoroughly unacceptable results on this document and, therefore, I do not trust it. The only option of the three that work for me on this document is Fine Page Content.

I am not certain what you mean here, the EOCR engine uses the same logic for all three modes of performing OCR, the text layout and assigned font should be identical between them, fine page content simply strips out more of your document data compared to editable text, while searchable image leaves the original content intact, and places an invisible text layer in place (the root cause of your initial report was that you had 5 layers of text in place was it not? That is what I am trying to address with the steps above).
As for the Bloat, it is good to hear that in your case, that is not an issue. It was just something to keep in mind as not all documents handle the transition to image content quite so well.

Kind regards,

[PHK]

20240408141053810_23-939 bsac CREW Final Copy XCE.pdf

...
Are you using the Enhanced OCR engine or the Tesseract one (Standard) - the Standard will have less options than the Enhanced one.

Enhanced.

And if your approach is giving you the results you need - shall we consider this topic resolved or is there anything else we can assist with?
...

I don't really consider it "resolved." It is becoming increasingly clear to me that strange,non-sensical things are going on when I assemble a lot pages onto a single merged sheet, or page. This is something I have been working on this morning.

image.png

This is a zoomed-out screenshot of about 66 pages assembled onto one sheet. That is at the center. All that blue stuff is the result of selecting Text in the Content panel. Surely, there is something weird going on here. I think that the frequent use of the Split Pages Tool is partially responsible. Also, I find that my use of the Crop Pages Tool ruins whatever Bookmark links that may have pre-existed. My workaround is to avoid the Crop Pages Tool as much as I can and instead use the Resize Pages wherever I can. I find that much less destructive of invisible Bookmark links.

I'll attach the file for this screeshot and you can play with it.

None of the Enhanced OCR options work well for me. They all have drawbacks. I go back to my original question at the top of the thread. If users cannot have this right now, I think we should in the future.

[PHK]

Sorry, the file did not seem to attach.

20240408141053810_23-939 bsac CREW Final Copy XCE.pdf

[Daniel - PDF-XChange]

Hello, PHK

How does this version of the file match your needs?

raster-then-searchable-OCR.pdf

Is there anything missing, or extraneous you see here?

Kind regards,

[PHK]

Hello, PHK

How does this version of the file match your needs?
raster-then-searchable-OCR.pdf
Is there anything missing, or extraneous you see here?

Kind regards,

That looks great!

So, what am I doing wrong and/or what are you doing right? This MAJOR for me.

[Daniel - PDF-XChange]

Hello, PHK

All I did here was the same 2 step process I mentioned before. I will break it down with some screenshots to help (Since the file is large, and the OCR took a good 10 minutes the first time over, I am recording these with a blank document. The UI will of course remain the same):
First, open your document, head to the Convert tab, and click "rasterize pages", ensure you set only the "text" option to Rasterize, and leave links, form fields, and comments on "keep" so they are not affected by the process. (you can optionally increase or decrease the DPI, but 300-600 is the "sweetspot" so to speak, for the OCR process in the next step, I would not advise reducing the dpi here)

image.png

After that is completed, click the "OCR pages" button on the Convert tab, and ensure that you specify "Searchable image":

image(1).png

Now you should have your document just like I created above.

Kind regards,

[PHK]

Thank you, Daniel. Your screenshots seem to be from PDF-Tools V.10 where the "Actions with Objects" appears.

Certainly, the online Help resource does not show the presence of those options in Editor.

image.png

But, thankfully, I do have those options in my app so I will begin trying to see if that helps.

I must admit this is the first I was aware of these options or their use or signficance. But on the other hand, I am not sure how Users are supposed to intuit these undocumented features.

[Daniel - PDF-XChange]

Hello, PHK

No, those screenshots are from the Editor, as we have been discussing thus far, but text control is a fairly new addition to the software, and may not have been recorded to the manuals as of yet. I will let our writing team know about this change.

In any case, the options presented there are fairly barebones and should be fairly clear. Each of the 4 content types has the option to either "keep" it as it is without changes, or "Rasterize" it, ergo, turning it into a raster image. It should be fair to expect that people do read the options presented to the before proceeding to use a feature, and doing so here should result in a general understanding of what would happen, given the tool that was selected.

Kind regards,

[PHK]

.... those screenshots are from the Editor...

Well, yes it may be in Editor now. But the first reference I spotted was in Tools documentation.

... text control is a fairly new addition to the software...

Yeah, no kidding. I certainly missed its addition or significance. I had established a hot-key "Alt-R" for rasterizing a long time ago and haven't looked into its sub-options for years.

In any case, the options presented there are fairly barebones and should be fairly clear.

Sorry, but obviously it wasn't clear to me.

I have been experimenting with this set-up as you suggested above it seems to be exactly what I was looking for. Therefore, I accept that my original post in this thread has been satisfied and that the question is now truly Resolved. It just took a long time to get there.

[Stefan - PDF-XChange]

[PHK]

A sort-of new issue for me but I did not want to start a new thread: the use of the Enhanced Scanned Pages Tool.

Sometimes, I will have a file consisting of pages that were created as digital PDFs and others that are from a scanner without the benefit of OCRing. If the page image is a bit sketchy or 'thin,' the OCR output may not be ideal. For instance, on one page there was visaully "II." - Roman Numeral Two - that OCRed as "4." Enhancing and re-OCRing fixed that. But I am not sure what goes on 'behind the curtain' with that Tool. I assume the softare spots loose clusters of dots and then fills the cluster out with some more dots. Does this have anything to do with OCRing? If a file is a mix of good digital and unenhanced analogue-like pages, is any harm done (other than wasting time) by enhancing all pages? If I need to enhance some pages and rasterize some pages (not necessarilty the same ones), in what sequence should I do that?

Thank you, any and all, in advance for whatever advice you can provide.

[Daniel - PDF-XChange]

Hello, PHK

Your theory there is relatively close how it works. Enhance scans is essentially an automated "touch up" tool, it is designed to do tasks like remove extra "speckling" from the pages, or darken and sharpen text.
This helps the OCR process more accurately identify the content, instead of doing as computers tend to do, and mistakenly finding a "letter" made out of a coffee stain, because of an off pattern when the scanner captured it.

On an already "pristine" document (IE: one which never left the digital format), there should not be anything found by the enhance function (no skew would exist, the text would already be deepest black, there wouldnt be any speckling from a scanner processing the image), and so the enhance function would not be impeded by that, any more than the cursory scan attempt on the page leading to a green light to continue on its merry way.

Kind regards,

[PHK]

Thank you, Daniel.

Am I correct in assuming that the Enhance Scans Tool really gives the User the option of, in effect, adding on the full and regular OCR Pages Tool by selecting the "Recognize Text" "Text Recognition Option" thereby eliminating the need to OCR the pages following an Enhance Scans operation?

[Stefan - PDF-XChange]

Hello PHK,

If you have the "Recognize text" turned on - when you click on the Edit..." for that part of the settings - you will see a dialogue with the currently active OCR engine's options (enhanced on my shot) - so yes - you are effectively also able to run OCR on your file at the same time:

image.png

Kind regards,
Stefan

[PHK]

Well, yes, Daniel, but there is a slight difference. If the User does choose to use the Output Options as well as enhance in the Enhance Pages Tool, the third Output Option that is available in Enhanced OCR -- Fine Page Content -- is not available as a sub-option in Enhance.

[Daniel - PDF-XChange]

Hello, PHK

The "fine page content" option has very few differences when compared to the "Editable text and images" and most of those differences are alleviated by the other processes present in the enhance scans function. I am not 100% certain that is the sole reason for the lack of third option here, or if there is another technical difference that prevents it, but at this time, only those two options are presented.

Kind regards,

[PHK]

Noted with thanks!

[Daniel - PDF-XChange]