Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9592

decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

    XMLWordPrintableJSON

Details

    Description

      I have serious performance issue using AtomicUpdate (and RealtimeGet) with non stored docValues.
      Because decorateDocValues try to merge each leafLeader on the fly via slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
      Simply access docValues via nonCompositeReader could resolve this issue.(patch)

      AtomicUpdate performance(or RealtimeGet performance)

      • Environment
        • solr version : 6.0.0
        • schema ~ 100 fields(90% docValues, some of those are multi valued)
        • index : 5,000,000
      • Performance
        • original : > 10sec per query
        • patched : at least 100msec per query

      This patch will also enhance search performance, because DocStreamer also fetch docValues via decorateDocValues.
      Though it depends on each environment, I could take 20% search performance gain.

      (This patch originally written for solr 6.0.0, and now rewritten for master)

      Attachments

        1. SOLR-9592_6x.patch
          34 kB
          Takahiro Ishikawa
        2. SOLR-9592.patch
          35 kB
          Takahiro Ishikawa
        3. SOLR-9592.patch
          3 kB
          Takahiro Ishikawa

        Activity

          People

            yseeley@gmail.com Yonik Seeley
            takaishi Takahiro Ishikawa
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: