Extract highlighted text

[mihalvld]

Is any way to extract highlighted text from pdf with PDF-XChange Editor like in Sumnotes? I have no highlighted text copied to annotations.

[Daniel - PDF-XChange]

Hello, mihalvld

Unfortunately no, the trouble here is that in PDF the highlight annotation is an entirely separate object from the text on the page, they cannot be linked in any way. If you have configured your commenting settings in the editor to do the following:

image.png

Than, during placement of new highlights, the selected text will be copied into the highlight itself (note that future edits to the page text will not be mirrored into the highlight). You can use the "Export comments" tool, on the comment tab, to then export the text contained within the highlight.

That is about the best I can offer for this though.

Kind regards,

[mihalvld]

My pdfs already have highlighted text that not copied into pop-ups.

My current way (I have no Sumnotes subscription):
1. Select highlights, select corresponded pages.
2. Extract selected pages
3. Split document (max 50 highlights per part).
4. Use Sumnotes.

[Dimitar - PDF-XChange]

Hello, mihalvld,

Thank you for the feedback.

Could you please tell us what version is your PDF Editor?

You can see this information in the PDF Editor's Help -> About menu.

Regards.

[mihalvld]

Ver. 9.2 build 358

[Stefan - PDF-XChange]

Hello mihalvld,

Thanks for the additional detail!
How long is the file that you have with highlights in it that were created before you ticked that option in Dan's screenshot?
If there are only a few highlights that you need to do - then the quickest solution might be to just go through your document and remove each one of them and then recreate it with the option turned on.

If there are thousands of annotations - you might try to write some JS, that goes through the file, locates each annotation that is a highlight, gets it's coordinates, and then searches for words inside those coordinates, any that are found are then included in the "content" of the existing annotation. This JS is a bit complicated (and working with text through JS inside PDFs is not straight forward), so I can not help with writing such a script, but it should be technically possible to write such.

Kind regards,
Stefan

[mihalvld]

I use different pdf readers on android and pc to work with the same file and it usually has 100 - 200 highlights. As I'm complete zero in JS I'll prefer to use my method. It takes ~10 minutes to extract all highlighted text.

[Daniel - PDF-XChange]

Hello, mihalvld

Sorry that we could not offer a better solution for you.

Have a nice day

[XChangeGirl]

It would indeed be useful were there a command that would add or append highlighted text to the associated comment box. I have now set the option to copy selected text to popups. Wish I'd learned of it sooner, though.

[Stefan - PDF-XChange]

[PatrickOfLondon]

One thing that may help in this situation:

> Bookmarks tab (top menu bar)
> button on the Bookmarks tab's ribbon: "From Highlights" generates bookmarks from, and containing, the highlighted text
> button on the Bookmarks tab's ribbon: Export [the resulting bookmarks] to Text File

The exported text file will thus contain the text of the highlighted passages in the PDF.

Rudimentary, but might be useful to someone, somewhere, sometime.

[Stefan - PDF-XChange]

Hello PatrickOfLondon,

Many thanks for this!
It can indeed find it's uses in some specific cases so we are happy you shared that idea with everyone!

Kind regards,
Stefan

[Mathew]

In case someone comes here looking for a script as described by Stefan above

If there are thousands of annotations - you might try to write some JS, that goes through the file, locates each annotation that is a highlight, gets it's coordinates, and then searches for words inside those coordinates, any that are found are then included in the "content" of the existing annotation. This JS is a bit complicated (and working with text through JS inside PDFs is not straight forward), so I can not help with writing such a script, but it should be technically possible to write such.

Here's a script that does that. Run it in the console, or if you want a menu item you could use the snippet tool I posted: viewtopic.php?t=49412
Most of the code is to handle the dialog:

image.png

Code: Select all

/** update highlight text info
  * gets the quads of highlight annotations, and looks for all the text under it
  * based on suggestion in https://forum.pdf-xchange.com/viewtopic.php?t=38167
  *
  * @author Mathew Bittleston
  * @license MIT
  * @version 0.2
  *
  * @param {object} doc - the pdf in which the annotations should be processed
  * @param {array} [processAnns] - array of annotations to process. If supplied, then no dialog will be shown
  * @param {boolean} [shouldAppend] - true to append instead of replacing contents
**/


function updateHighlightText(doc, processAnns, shouldAppend = false) {
  if (!doc) {
    throw new Error("Update Highlight Text: Parameter Error: doc is nothing.");
  }
  
  /** get index(es)
    * @param {object} elements - object returned by dialog.store()
    * @returns {array} the index number of the item(s) with positive number value - always returns array
  **/
  function getIndex(elements) {
    return Object.values(elements).reduce( (a,e,i) => e>0 ? a.concat(i) : a , [] );
  }
  /** create object suitable for the listbox. allows multiple selections
    * @param {array|integer} selItem - selected index(es)
    * @returns {object} is {"Displayed option":-order,...}
  **/
  function getListboxArray(vals, selItem = 0) {
    let sub = {};
    const sel = new Set( Array.isArray(selItem) ? selItem : [selItem]);
    const counter = (c) => c ? '_' + c : '';
    vals.forEach( (v,i) => {
      if ('' === v) v = ' '; // empty key bug PXCE
      // if multiple duplicate names, add counter
      let j = 0;
      while (undefined !== sub[v + counter(j)]) j++;
      // positive number if selected
      sub[v + counter(j)] = (sel.has(i) ? 1 : -1) * ( i + 1);
    });
    return sub;
  }
  
  // function returns a check mark for true, long space for false
  function cMark(ck) {
    return ck ? '✔ ' : '\u2003'; // long space  \u2002\u2004
  }

  // these are the only annotation types that have quads
  const highlightTypes = ['Highlight', 'StrikeOut', 'Underline', 'Squiggly', 'Redact'];
  // filter for annotations with quads
  let hAnns = doc.selectedAnnots;

  
  const hDialog = {
    highlightTypes: ['< All >', 'Highlight', 'StrikeOut', 'Underline', 'Squiggly', 'Redact'],
    highlightOpts: [false, true, true, true, true, false],
    initialize(dialog) {
      dialog.load({
        cSel: !!hAnns?.length,
        allA: !hAnns?.length,
        hPrc: this.hTypes,
        tRep: !shouldAppend,
        tApp: shouldAppend
      });
      dialog.enable({ cSel: !!hAnns?.length});
    },
    get hTypes() {
      const hOpts = this.highlightTypes.map( (h,i) => cMark(this.highlightOpts[i]) + h);
      return getListboxArray(["Annotation types:"].concat( hOpts), 0);
    },
    commit(dialog) {
      this.results = dialog.store();
    },
    hPrc(dialog) {
      const [ix] = getIndex(dialog.store()['hPrc']);
      if (!ix) return;
      this.highlightOpts[ix - 1] = !this.highlightOpts[ix - 1];
      dialog.load({ hPrc: this.hTypes});
    },
    description: {
      name: "Update Highlight Text", elements: [
        { type: 'cluster', align_children: 'align_left', name: "Highlights to process", elements: [
          { type: 'radio', item_id: 'cSel', group_id: 'sOpt', name: "Current &Selection"},
          { type: 'radio', item_id: 'allA', group_id: 'sOpt', name: "&All Highlights in pdf"},
          { type: 'popup', item_id: 'hPrc', width: 150}
        ]},
        { type: 'cluster', align_children: 'align_left', name: "Text Handling", elements: [
          { type: 'radio', item_id: 'tRep', group_id: 'tOpt', name: "&Replace Contents"},
          { type: 'radio', item_id: 'tApp', group_id: 'tOpt', name: "A&ppend to Contents"}
        ]},
        { type: 'ok_cancel'}
    ]}
  };
  
  // get center of quad
  function centroid(quads) {
    let ctr = [0,0];
    let nqs = quads.length;
    for (let qs of quads) {
      let nq = qs.length / 2; // should be 4
      // center of these quads
      let qCtr = qs.reduce( function(acc, curr, i) {
        acc[i%2] += curr / nq;
        return acc;
      } , [0,0]);
      // add to ctr
      qCtr.forEach( (q,i) => ctr[i] += q / nqs);
    }
    return ctr;
  }
  
  // checks if all of quads are inside rect
  function inside(rect, quads) {
    // split rects to chunks 8 numbers long
    //const allChunks = Array.from(rect);
    //let chunk;
    let inside = true;
    //while ((chunk = allChunks.splice(0,8)) && chunk.length >= 4) {
    //for( chunk of allChunks) {
    //  const minMax = chunk.reduce((a,v,i) => {
    const minMax = rect.reduce((a,v,i) => {
      // min
      if (undefined === a[0][i%2] || a[0][i%2] > v) {
        a[0][i%2] = v;
      }
      // max
      if (undefined === a[1][i%2] || a[1][i%2] < v) {
        a[1][i%2] = v;
      }
      return a;
    }, [[],[]]);
    
    for (let qs of quads) {
      // work through flat array
      qs.forEach( (c, i) => {
        // min
        inside = inside && c >= minMax[0][i%2];
        // max
        inside = inside && c <= minMax[1][i%2];
      });
      if (!inside) return false;
    }
    //}
    return true;
  }
  

  // run the dialog
  if (!processAnns) {
    if ('ok' !== app.execDialog(hDialog)) return;
  
    const {allA, tApp} = hDialog.results;
    
    // could filter this by ANFB_ShouldExport etc.
    if (allA) hAnns = doc.getAnnots();
    
    shouldAppend = tApp;
  }
  
  // types to process
  const procTypes = hDialog.highlightTypes.filter((h,i) => hDialog.highlightOpts[i]);
  // filter selection
  hAnns = ( processAnns ?? hDialog.highlightOpts[0] ?
    hAnns :
    hAnns.filter(a => procTypes.includes(a.type))
  ).sort((a,b) => a.page - b.page);
    
  if (!hAnns?.length) { // no quads
    app.alert("No markups to process.");
    return;
  }

  // hold page words
  const pageWords = [];
  hAnns.forEach(ann => {
    const page = ann.page;
    if (!pageWords[page]) {
      const nWords = doc.getPageNumWords(page);
      pageWords[page] = [];
      for (let n = 0; n < nWords; n++) {
        const quads = doc.getPageNthWordQuads(page, n);
        pageWords[page].push({
          word: doc.getPageNthWord(page, n, false), // keep punctuation and white space
          quads,
          ctr: centroid(quads)
        });
      }
    }
    
    // try to build up words in this highlight
    let words = '';
    pageWords[page].forEach(({word, ctr}) => {
      const isInside = (ann.quads?.length ? ann.quads : [ann.rect])
        .some(q => inside(q, [ctr]));//inside(rect, allInside ? quads : [ctr])
      if (isInside) {
        //console.println(JSON.stringify({word,ctr,quads:ann.quads}, null, 1))
        words += word;
      }
    });
    // assign to the highlight
    ann.contents = (shouldAppend ? ann.contents : '') + words;
  });
  
  console.println(util.printf("Processed %s annotations.", hAnns.length));
}

updateHighlightText(this);

[Dimitar - PDF-XChange]