Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7512

SolrOutputFormat creates an invalid solr.xml in the solr home zip for MapReduceIndexerTool

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 5.1
    • 5.2.1, 5.3, 6.0
    • contrib - MapReduce
    • None

    Description

      Sometime after Solr 4.9, the `MapReduceIndexerTool` got busted because invalid `solr.xml` contents were being written to the solr home dir zip. My guess is that a 5.0 change made the invalid file start to matter.

      The error manifests as:

      Error: java.lang.IllegalStateException: Failed to initialize record writer for org.apache.solr.hadoop.MapReduceIndexerTool/SolrMapper, attempt_1430953999892_0012_r_000001_1
              at org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:126)
              at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:163)
              at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:569)
              at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:643)
              at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:394)
              at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
              at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170)
      Caused by: org.apache.solr.common.SolrException: org.xml.sax.SAXParseException; Premature end of file.
              at org.apache.solr.core.Config.<init>(Config.java:156)
              at org.apache.solr.core.SolrXmlConfig.fromInputStream(SolrXmlConfig.java:127)
              at org.apache.solr.core.SolrXmlConfig.fromFile(SolrXmlConfig.java:110)
              at org.apache.solr.core.SolrXmlConfig.fromSolrHome(SolrXmlConfig.java:138)
              at org.apache.solr.core.CoreContainer.<init>(CoreContainer.java:142)
              at org.apache.solr.hadoop.SolrRecordWriter.createEmbeddedSolrServer(SolrRecordWriter.java:162)
              at org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:119)
              ... 9 more
      Caused by: org.xml.sax.SAXParseException; Premature end of file.
              at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
              at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
              at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
              at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
              at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
              at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
              at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
              at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
              at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
              at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
              at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
              at org.apache.solr.core.Config.<init>(Config.java:145)
              ... 15 more
      
      

      The last version that I've successfully used `MapReduceIndexerTool` was 4.9, and I verified that this patch resolves the issue for me (testing on 5.1). I spent a couple hours trying to write a simple test case to exhibit the error, but I haven't quite figured out how to deal with the

      java.security.AccessControlException: java.io.FilePermission ...

      errors.

      Pull request for bugfix here

      Attachments

        1. SOLR-7512.patch
          66 kB
          Mark Miller
        2. SOLR-7512.patch
          61 kB
          Mark Miller
        3. SOLR-7512.patch
          11 kB
          Adam McElwee
        4. SOLR-7512.patch
          12 kB
          Adam McElwee

        Activity

          People

            markrmiller@gmail.com Mark Miller
            amcelwee Adam McElwee
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: