Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-15079

Block Collapse (faster collapse code when groups are co-located via Block Join style nested doc indexing)

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 9.0, 8.9
    • None
    • None

    Description

      A while back, JoelB had an idea for an optimized version of of the logic in the CollapsingQParserPlugin to take advantage of collapsing on fields where the user could knows that every doc with the same collapseKey were contiguous in the index - for example collapsing on the root field.

      Joel whipped up an initial PoC patch internally at lucidworks that only dealt with some limited cases (string field collapsing w/o any nulls, using default group head selection) to explain the idea, but other priorities prevented him from doing thorough benchmarking or flesh it out into "production ready" code.

      I took Joel's original PoC and fleshed it out with unit tests, fixed some bugs, and did some benchmarking against large indexes - the results look really good.

      I've since then beefed the code up more to include collapsing on numeric fields, and added support for all group head selector types, as well as adding support for nullPolicy=expand.

      Attachments

        1. SOLR-15079.patch
          89 kB
          Chris M. Hostetter

        Activity

          People

            hossman Chris M. Hostetter
            hossman Chris M. Hostetter
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: