Wikidata grafana dashboard for number of instances of various things is broken
Closed, ResolvedPublic5 Estimated Story Points
Actions

Assigned To

Authored By

	Addshore
	Mar 19 2019, 5:46 PM

Description

PHP Fatal error:  Uncaught RuntimeException: The SPARQL request failed! in /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php:78
Stack trace:
#0 /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php(59): WikidataInstanceOf->doSparqlQuery('PREFIX wd: <htt...')
#1 /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php(40): WikidataInstanceOf->getResult('Q11266439')
#2 /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php(14): WikidataInstanceOf->execute()
#3 {main}
  thrown in /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php on line 78

It looks like this is due to the hardcoded server wdqs1003 no longer existing.
https://github.com/wikimedia/analytics-wmde-scripts/blob/master/src/wikidata/sparql/instanceof.php#L74
perhaps wdqs1005 should be used?
See T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888 for why we can not just use the service alias.

Example broken dashboards

https://grafana.wikimedia.org/d/000000175/wikidata-datamodel-statements?refresh=30m&orgId=1

Details

Subject	Repo	Branch	Lines +/-
Use config for wdqs host name	analytics/wmde/scripts	production	+15 -2
Use config for wdqs host name	analytics/wmde/scripts	master	+15 -2
statistics: Add wdqs host to wmde statistcs configuration	operations/puppet	production	+1 -0

Customize query in gerrit

Related Objects

Mentioned In: rADES405b3b965c18: Use config for wdqs host name
T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888
Mentioned Here: T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888

Event Timeline

Addshore created this task.Mar 19 2019, 5:46 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 19 2019, 5:46 PM

It could be that the sparql query now takes too long and we perhaps need to think about a better way to get this data?

Hadoop perhaps? :)

Looks like this is because the host needs to be hard coded in the script.

https://github.com/wikimedia/analytics-wmde-scripts/blob/master/src/wikidata/sparql/instanceof.php#L74

Because T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888 can not be done.

So the host needs to be updated!
Perhaps to wdqs1005?

Addshore triaged this task as Low priority.Jun 22 2019, 10:12 PM

Addshore updated the task description. (Show Details)

Addshore moved this task from incoming to ready to go on the Wikidata board.

Addshore moved this task from Incoming to Ready to estimate on the Wikidata-Campsite board.

Addshore updated the task description. (Show Details)

Addshore mentioned this in T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888.Jun 22 2019, 10:17 PM

• alaa_wmde updated the task description. (Show Details)Jul 2 2019, 12:24 PM

• alaa_wmde updated the task description. (Show Details)

• alaa_wmde set the point value for this task to 5.Jul 2 2019, 12:27 PM

Change 520901 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/puppet@production] statistics: Add wdqs host to wmde statistcs configuration

https://gerrit.wikimedia.org/r/520901

Change 520903 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[analytics/wmde/scripts@master] Use config for wdqs host name

https://gerrit.wikimedia.org/r/520903

Ladsgroup claimed this task.Jul 5 2019, 3:18 PM

Ladsgroup edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJul 5 2019, 3:18 PM

Ladsgroup moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jul 5 2019, 3:18 PM

My patch fixes it, tested it on production.

Change 520901 merged by Filippo Giunchedi:
[operations/puppet@production] statistics: Add wdqs host to wmde statistcs configuration

https://gerrit.wikimedia.org/r/520901

Addshore moved this task from ready to go to in progress on the Wikidata board.Jul 9 2019, 4:41 PM

Ladsgroup merged a task: T227787: broken statements dashboard.Jul 12 2019, 8:32 PM

Ladsgroup added a subscriber: Lydia_Pintscher.

Change 520903 merged by jenkins-bot:
[analytics/wmde/scripts@master] Use config for wdqs host name

https://gerrit.wikimedia.org/r/520903

hoo moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jul 15 2019, 10:23 AM

Change 523131 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[analytics/wmde/scripts@production] Use config for wdqs host name

https://gerrit.wikimedia.org/r/523131

Change 523131 merged by jenkins-bot:
[analytics/wmde/scripts@production] Use config for wdqs host name