timebox: 2 days

That autocomplete uses the wbsearchentities API endpoint. As it was recently decided* that EntitySchema should not become Entities, they will not be available here without major changes.

‍* FTR: I favored a different decision that would have made this much simpler.

For Properties and Lexemes: Individually, these Entity types are already supported by the API:

• https://www.wikidata.org/wiki/Special:ApiSandbox#action=wbsearchentities&format=json&search=instance%20of&language=en&type=property&formatversion=2
• https://www.wikidata.org/wiki/Special:ApiSandbox#action=wbsearchentities&format=json&search=color&language=en&type=lexeme&formatversion=2

However, the wbsearchentities API currently requires exactly one type parameter (defaulting to item). Also, we need to figure out if CirrusSearch/Elastic is meaningfully able to search across entity types.

One possible approach could be to enable autocomplete / typeahead for other entity types based on a prefix:

• no prefix -> autocomplete for Items
• prefix P: -> autocomplete for Properties
• prefix L: -> autocomplete for Lexemes

With this approach, it would likely only be a moderate amount of work to also make it work for EntitySchema, probably with a different API endpoint.

In T327507#8648836, @Michael wrote:

One possible approach could be to enable autocomplete / typeahead for other entity types based on a prefix:

• no prefix -> autocomplete for Items
• prefix P: -> autocomplete for Properties
• prefix L: -> autocomplete for Lexemes

With this approach, it would likely only be a moderate amount of work to also make it work for EntitySchema, probably with a different API endpoint.

The underlying idea could also work when presenting the search results of the different Entity types / namespaces in separate lists below / next to each other. This would then be mainly a UX challenge.

That being said, if we want to show them in one single feed of suggestions, then this probably will need some dedicated work in CirrusSearch/Elastic. See for example the difference in results by just selecting the additional Lexeme namespace:

• Search for "color" in Items and Properties: https://www.wikidata.org/w/index.php?search=color&title=Special%3ASearch&profile=advanced&fulltext=1&ns0=1&ns120=1
• Search for "color" in Items and Properties and Lexemes: https://www.wikidata.org/w/index.php?search=color&title=Special%3ASearch&profile=advanced&fulltext=1&ns0=1&ns120=1&ns146=1

The CirrusSearch search callbacks for Items, Properties and Lexemes can be found at

• https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikibaseCirrusSearch/+/refs/heads/master/WikibaseSearch.entitytypes.php
• https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikibaseLexemeCirrusSearch/+/refs/heads/master/WikibaseSearch.entitytypes.repo.php

However, these are all specific to their respective entity type. I have yet to understand why Special:Search shows very different results depending on which namespace is included.

(timebox used so far: ~3:15)

Yeah when I discussed this with Stas from the Search team ages ago he basically said we can forget any meaningful ranking across entity types. So the designs so far all revolve around specifying the entity type or grouping by entity type. I think this is also better from a user PoV because the fact that there are very different types of entities is more obvious.

In T327507#8649433, @Lydia_Pintscher wrote:

Yeah when I discussed this with Stas from the Search team ages ago he basically said we can forget any meaningful ranking across entity types. [...]

Well, we do have meaningful ranking across Properties and Items on Special:Search, as shown above. However, I agree that this is probably tricky to achieve, and, currently, it breaks down when adding the Lexeme namespace.

I spontaneously joined the Search Team Office Hour today and asked about that and got a few pointers. (Thanks!)

• one can add &cirrusExplain=pretty&cirrusDumpResult to the search url, and it shows explanatory output
  • (verbose and json are other valid values for cirrusExplain)
  • one can see there is pretty different output based on whether Lexeme namespace is included or not:
    • explained search for "color" in Items and Properties
    • explained search for "color" in Items, Properties, and Lexemes
  • notice that the scores when searching for Lexemes are much simpler, probably that comes from Lexeme using a different SearchProfile or something than core Wikibase
    • probably that would be in Lexeme's handler for the onCirrusSearchProfileService hook
• one way to dig into this further and make this better comparable would be to look at the explained search results for a specific item under different conditions:
  • explained search for "Game Boy Color" in Items and Properties
  • explained search for "Game Boy Color" in Items, Properties, and Lexemes

But, as said above, getting this right across a diverse domain of namespaces is probably a pretty intricate challenge and maybe not the best thing to do right now.

In T327507#8649433, @Lydia_Pintscher wrote:

[...] So the designs so far all revolve around specifying the entity type or grouping by entity type. I think this is also better from a user PoV because the fact that there are very different types of entities is more obvious.

Ok, so if we split this by type, then it should only need minor adjustments for Properties and Lexemes (I think a search with wbsearchentities for Lexemes currently only displays one Lemma, and we might want to see all of them).

For EntitySchema, I think, we would need to create a new API endpoint, and there is already a feature request for that: T304070: API Endpoint to search for Schemas

(total timebox used so far: ~5:00)