Blocker Description
Hi,
Testing out branch_8x, we're randomly seeing the following errors on a simple 3 node cluster. It doesn't appear to affect replication (the cluster remains green).
They come in (mass, literally 1000s at a time) bulk.
There we no network issues at the time.
16:53:01.492 [updateExecutor-4-thread-34-processing-x:at-uk_shard1_replica_n1 r:core_node3 null n:solr-2.search-solr.preprod.k8.atcloud.io:80_solr c:at-uk s:shard1] ERROR org.apache.solr.client.solrj.impl.ConcurrentUpdateHttp2SolrClient - Error consuming and closing http response stream. java.nio.channels.AsynchronousCloseException: null at org.eclipse.jetty.client.util.InputStreamResponseListener$Input.read(InputStreamResponseListener.java:316) ~[jetty-client-9.4.14.v20181114.jar:9.4.14.v20181114] at java.io.InputStream.read(InputStream.java:101) ~[?:1.8.0_191] at org.eclipse.jetty.client.util.InputStreamResponseListener$Input.read(InputStreamResponseListener.java:287) ~[jetty-client-9.4.14.v20181114.jar:9.4.14.v20181114] at org.apache.solr.client.solrj.impl.ConcurrentUpdateHttp2SolrClient$Runner.sendUpdateStream(ConcurrentUpdateHttp2SolrClient.java:283) ~[solr-solrj-8.1.0-SNAPSHOT.jar:8.1.0-SNAPSHOT b14748e61fd147ea572f6545265b883fa69ed27f - root - 2019-03-04 16:30:04] at org.apache.solr.client.solrj.impl.ConcurrentUpdateHttp2SolrClient$Runner.run(ConcurrentUpdateHttp2SolrClient.java:176) ~[solr-solrj-8.1.0-SNAPSHOT.jar:8.1.0-SNAPSHOT b14748e61fd147ea572f6545265b883fa69ed27f - root - 2019-03-04 16:30:04] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6] at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[solr-solrj-8.1.0-SNAPSHOT.jar:8.1.0-SNAPSHOT b14748e61fd147ea572f6545265b883fa69ed27f - root - 2019-03-04 16:30:04] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_191] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]