Wikidata:Property proposal/georeferencing data

georeferencing control point data

Originally proposed at Wikidata:Property proposal/Sister projects

   Not done

Description	Tabular data file in the Commons Data: namespace containing georeferencing control points for all or part of a Commons old map image
Represents	georeferencing (Q772007)
Data type	Tabular data
Domain	Commons image
Allowed values	Data:.+\.gcps(\..*)?\.tab
Example 1	File:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.jpg → Data:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.gcps.tab
Example 2	MISSING
Example 3	MISSING
Source	user input via app; also existing data in Commons MapWarper, NYPL MapWarper, British Library Georeferencer, David Rumsey maps, other sites with georeferencing
Planned use	Movement of data from Commons MapWarper site to Commons Data namespace; import from external sites
See also	Wikidata:Property proposal/external georeferencer URL

•  Support I think this is an excellent idea. However we should have more examples. Are there any plans to use this data in some way?

georeferencing pixel mask data

Originally proposed at Wikidata:Property proposal/Sister projects

   Not done

Description	Tabular data file in the Commons Data: namespace containing the pixel coordinates of the boundary of the georeferenced area
Represents	georeferencing (Q772007)
Data type	Tabular data
Domain	Commons image
Allowed values	Data:.+\.mask(\..*)?\.tab
Example 1	File:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.jpg → Data:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.mask.tab
Example 2	MISSING
Example 3	MISSING
Source	user input via app; also existing data in Commons MapWarper, NYPL MapWarper, British Library Georeferencer, David Rumsey maps, other sites with georeferencing
Planned use	Movement of data from Commons MapWarper site to Commons Data namespace; import from external sites
See also	Wikidata:Property proposal/external georeferencer URL

•  Support --Jarekt (talk) 03:01, 1 July 2020 (UTC)

georeferencing mask geoshape

Originally proposed at Wikidata:Property proposal/Sister projects

   Not done

Description	Geoshape containing the georectified latitude and longitude coordinates of the boundary of the georeferenced area
Represents	georeferencing (Q772007)
Data type	Geographic shape
Domain	Commons image
Allowed values	Data:.+\.mask(\..*)?\.)\.map
Example 1	File:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.jpg → Data:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.mask.map
Example 2	MISSING
Example 3	MISSING
Source	computed from pixel mask using georectification based on a particular set of control points
Planned use	Movement of data from Commons MapWarper site to Commons Data namespace; import from external sites
See also	Wikidata:Property proposal/external georeferencer URL

Motivation

Moving Georeferencing data from its present silo in the MapWarper application into the Commons Data: namespace brings it much more naturally into the domain of the Commons community, and will make it much more accessible for use in gadgets or other reuse.

These SDC properties will enable apps to find what has been put where, for the two main types of information used as input for the process; and for queries and apps to access a geoshape showing the real-world position and characteristic boundary of the georectified map.

qualifiers, etc

Note that multiple sets of control-point (gcps) data may exist concurrently for the same image. Different sets of control points may relate to different maps contained in the image (eg multiple sub-maps; or main map/inset map(s) combinations); to sets of control point data obtained independently from different sources (eg Commons MapWarper / external source); or to control point data obtained from different features for different purposes (eg projection estimation [1] is often most effectively achieved based on control points representing a graticule of intersections of lines of latitude and longitude, whereas final map-warping often aims to distort the map to most accurately place depicted land-features and population centres).

Different sets of control points may be associated with the same or with different pixel masks.

The output mask geoshape will in turn relate to a particular pair of control-point and pixel-mask datafiles.

These potentially complicated relationships can be captured by qualifiers on the statements, in particular parallel applies to part, aspect, or form (P518) qualifiers on each of the statements to indicate a particular part of the image (eg main map / top-left map / inset map / second inset map etc), object of statement has role (P3831) to distinguish sets of control points relating to different features; and referencing to distinguish sets of control points with different provenances.

The control-point and pixel-mask statements should also be qualified to associate them with a particular revision-id of the file (Wikidata:Property proposal/image revision-id), in case it may subsequently have been cropped/modified.

The georectified mask should be linked to the inputs it is derived from (Wikidata:Property proposal/based on tabular data). It should also have, for convenience, a P518 to indicate the particular map it relates to, where the image contains multiple maps. Jheald (talk) 06:01, 17 August 2019 (UTC) (revised proposal)

Discussion

• Proposed. Jheald (talk) 06:01, 17 August 2019 (UTC)

•  Support I've just watched the summary of what is possible at min 22.30 of this presentation at Wikimania. The potential for this looks amazing. - Ambrosia10 (talk) 20:58, 18 August 2019 (UTC)
•  Support I also saw this demoed at the Wikimania hackathon showcase and think it makes a lot of sense. --Daniel Mietchen (talk) 00:31, 19 August 2019 (UTC)
•  Support Thisismattmiller (talk) 13:20, 19 August 2019 (UTC)
•  Support --Sabas88 (talk) 16:12, 19 August 2019 (UTC)
•  Support Buccalon (talk) 17:54, 19 August 2019 (UTC)
•  Support link Wikimania 2019: Hackathon Showcase 22:05 --Salgo60 (talk) 06:33, 22 August 2019 (UTC)
•  Support Never heard about this, but it looks like it could be covered by Wikidata. --Juandev (talk) 20:15, 29 September 2019 (UTC)
•  Conditional support @Jheald: Hello, Could you include other examples so that future properties don't start production with "warnings"? This will also verify if the RegEx are compliant. —Eihel (talk) 06:45, 20 January 2020 (UTC)
•  Support Susanna Ånäs (Susannaanas) (talk) 10:12, 17 May 2020 (UTC)

I agree that we need to get the data out of the silo, but this proposal seems to make things more complex than needed. I've been poking around a bit today based on https://github.com/bertspaan/wikimania-hackathon-2019 . This is what we want to store and how it's done in the example:

• Map image: Commons:File:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.jpg
• GCPs: Commons:Data:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.gcps.tab
• Pixel mask: Commons:Data:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.mask.tab
• Map mask: Commons:Data:Pigot_and_Co_(1842)_p2.138_-_Map_of_Lancashire.mask.map

I would like to have just one (geojson) file to contain all this data. To test this I'm using Commons:Data:Georectification.example.geojson.map which is a json file with a geojson part in the data field (see also mw:Help:Map Data). Above I read the motivation for multiple pieces. Just create a new map data file if you have another source or another view. Mixing everything makes it extremely complicated. As for the specific revision. These maps shouldn't be overwritten at all per overwrite policy. So one file per warping. Containing the points mentioned above. I had a look at the RFC and Geojson allows foreign members. In the example I made use of that to have several parts in the file:

• A feature of type "mask" contain the polygon for the mask. You can see the shape on the map
• A feature of type "gcp" for every geo control point containing the pixel x and y and a point with the lat and long. I would like to group these and maybe add numbering to them. Not sure what is possible here and I have to poke a bit more (maybe GeometryCollection)
• A feature of type "pixelmask" containing the file name, the entity id and all the pixels (x and y) for the pixel mask

The exact format is open for improvements. This way we can have everything in just one data file making it much easier to work with. Note that the data file contains what file it applies to so it doesn't even have to be linked for the file to work. I would prefer to go down this path and only have one property to link the image file with the geojson file. Multichill (talk) 11:05, 17 May 2020 (UTC)

• I  Support this new proposal. I cannot see any obstacles in putting the different parts together.
  • It makes great sense in putting the two different parts of the mask in the same file.
  • Bundling all parts of one warping reduces complexity, and makes versioning easier. – Susanna Ånäs (Susannaanas) (talk) 15:29, 17 May 2020 (UTC)
• Looks reasonable to me. I'd may have suggestions for small tweaks, I'll have to think some things through. But it is a good start. I'd suggest adding a 'sha1' or 'sha256' field to pixelmask, to identify and track the exact version of an image that the mapping was made from, should be easy for mapwarper to add that. TheDJ (talk) 11:09, 18 May 2020 (UTC)
  • @TheDJ: thanks for you input. My main point is to use one Geojson file with everything in it. I made an example to show it's possible. The format is in no way final and we should probably think it through properly before mass deploying it. I like the hash suggestion. Let's just do the sha1 of the file because that's already exposed in the api anyway and saves the hassle of calculating it ourselves. Multichill (talk) 20:37, 18 May 2020 (UTC)
•  Support  Don't ping me NMaia (talk) 01:29, 11 November 2020 (UTC)

• Looks like this (like the other Commons property proposals) fell thru the cracks; there is clear support for the three proposals on this page, so I've marked them ready. JesseW (talk) 21:56, 30 March 2021 (UTC)
  • Actually, looking further, it looks like they are superseded (maybe) by @Multichill:'s proposal. @Jheald:, what are your thoughts? JesseW (talk) 21:59, 30 March 2021 (UTC)
    • @JesseW: Thanks for going through these proposals. Having talked about it with User:Multichill since first proposing this, I agree that it does make sense to keep all the data together in one file, as sketched out at c:User_talk:Multichill/Map_warping_format, so I think that probably is the best way forward. The separate files have a certain amount to be said for them in terms of transparency, but I agree that is outwieghed by the convenience of just having to deal with one file, and not having to worry about any versioning issues. So there would then be one property "georeferencing data" pointing to a 'geographic shape' object, ie a .map file in the Data: namespace on Commons.

I am not sure whether one could go ahead and create such a property based on the support already shown on this page, or whether it would need a new proposal. But we are probably quite close to wanting to deploy at least some test examples of this, and IMO it would be useful for development to have statements on SDC pointing to which the relevant georeferencing data file was. Jheald (talk) 11:31, 31 March 2021 (UTC)

I think the given support for Multichill's proposal is sufficient, but it'd be nice to have the proper template set up to make it clear what exactly is being proposed. Once you do that, I think it's fine to mark it as ready. JesseW (talk) 14:20, 31 March 2021 (UTC)

@Jheald, JesseW: I realize this has had a long wait for various reasons, but I think it could be dealt with quickly (i.e. in about a week) as a new proposal, assuming nobody objects. I think that would be cleaner than trying to rewrite the proposals on this page for the new approach. ArthurPSmith (talk) 17:36, 31 March 2021 (UTC)

@MasterRus21thCentury: why did you mark these as ready? This proposal is stale and unclear how to go next. If these properties are created now, we'll just end up with a bunch of unused properties that end up getting deleted. Multichill (talk) 21:11, 14 April 2022 (UTC)

 Question: So, the discussion should be closed due to inactivity? MasterRus21thCentury (talk) 21:18, 14 April 2022 (UTC)

• Jheald I'm closing this proposal as  Not done, no consensus of the proposed property at this time based on the above discussion. Please open a new property proposal with the proper template setup. Regards, ZI Jony ^(Talk) 06:29, 28 May 2023 (UTC)