Turn words array from a document with their coordinates into sentences

Question

I have an array of words with their coordinates in the document, I want to turn them into sentences. For example this:

  [
    {
        "bounds": [
          {
            "x": 10,
            "y": 10
          },
          {
            "x": 15,
            "y": 10
          },
          {
            "x": 15,
            "y": 15
          },
          {
            "x": 10,
            "y": 15
          }
        ],
        "desc": "Hey"
      },
    {
        "bounds": [
          {
            "x": 18,
            "y": 10
          },
          {
            "x": 24,
            "y": 10
          },
          {
            "x": 24,
            "y": 15
          },
          {
            "x": 18,
            "y": 15
          }
        ],
        "desc": "Name"
      },
          {
        "bounds": [
          {
            "x": 18,
            "y": 20
          },
          {
            "x": 24,
            "y": 20
          },
          {
            "x": 24,
            "y": 25
          },
          {
            "x": 18,
            "y": 25
          }
        ],
        "desc": "What"
      },
    {
        "bounds": [
          {
            "x": 18,
            "y": 20
          },
          {
            "x": 24,
            "y": 20
          },
          {
            "x": 24,
            "y": 25
          },
          {
            "x": 18,
            "y": 25
          }
        ],
        "desc": "Sup"
      }
]

Should Have turned into this:

Hey Name
What Sup

The coordinates are not accurate just an example, also the algorithm needs to deal with words that are in the middle of sentences and other extreme cases.

What it the best way I can do it (Ideally implemented with JavaScript)?

Apiwat Chantawibul · Answer 1 · 2017-07-30T13:44:29.957

On this site, we only care about algorithm itself and not javascript. From what I gathered, your problem statement (without the javascript part) is:

Given a set of rectangles in described in Cartesian coordinate, (1) partition the set into "lines" (2) sort rectangles within each "line" by their position along the "line"

The reason I put the word "line" in quotation is because there are many ways to define it, and you didn't give enough information to decide which one.

Let's formulate this a bit: we are given

set $S$ of rectangles
four functions $t,b,l,r \colon S \rightarrow \mathbb{N}$ standing for top, bottom, left and right coordinate of the rectangle respectively.

We need to output:

ordered partition of $S$ such that "each partition is a line"

For convenience, I'm going to let $\sim \subseteq S \times S$ denotes the equivalence relation that partition $S$ into lines.

Here are three separate (not exhaustive) examples of definition of a line:

Two rectangle is in the same line if they have the same bottom $y$ value.$$x \sim y \iff bx = by$$
A rectangle is in the same line with its nearest neighbour in $y$ direction that is not further than some threshold. $$y = min_z |bx-bz| \implies (|bx-by| \leq T \implies x \sim y)$$ don't forget to make $\sim$ transitive.
Decide on the direction of a line first which may not be parallel to $x$ axis. Change the basis of coordinate. Then apply one of the above.

Decide on one you want according to the degree of sophistication needed to handle your input. Once you have the partition (supposed by $y$ value), you can then sort each by $x$ value.

Finally, there is also the alternative to do Hough Transform which is also generally applicable to bitmap images. In our case, our image is one of rectangles (or centre of rectangles, or corner of rectangles).

Turn words array from a document with their coordinates into sentences

1 Answers1