Module:scripts
- The following documentation is located at Module:scripts/documentation. [edit]
- Useful links: subpage list • links • transclusions • testcases • sandbox
This module is used to retrieve and manage Wiktionary's various writing systems and the information associated with them. See Wiktionary:Scripts for more information.
The information itself is stored in Module:scripts/data. The data module should not be used directly by any other module, the data should only be accessed through the functions provided by Module:scripts.
For functions that allow templates to use this module, see Module:scripts/templates.
Finding and retrieving scripts
The module exports a number of functions that are used to find scripts.
export.makeObject
function export.makeObject(code, raw_data)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
export.getByCode
function export.getByCode(code, paramForError, disallowNil)
Finds the script whose code matches the one provided. If it exists, it returns a Script
object representing the script. Otherwise, it returns nil
, unless paramForError is given, in which case an error is generated. If paramForError
is true
, a generic error message mentioning the bad code is generated; otherwise paramForError
should be a string or number specifying the parameter that the code came from, and this parameter will be mentioned in the error message along with the bad code.
export.getByCanonicalName
function export.getByCanonicalName(name)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
export.charToScript
function export.charToScript(char)
Takes a codepoint or a character and finds the script code (if any) that is appropriate for it based on the codepoint, using the data module Module:scripts/recognition data. The data module was generated from the patterns in Module:scripts/data using Module:User:Erutuon/script recognition.
Converts the character to a codepoint. Returns a script code if the codepoint is in the list of individual characters, or if it is in one of the defined ranges in the 4096-character block that it belongs to, else returns "None".
export.findBestScriptWithoutLang
function export.findBestScriptWithoutLang(text, none_is_last_resort_only)
Returns the code for the script that has the greatest number of characters in text
. Useful for script tagging text that is unspecified for language. Uses Module:scripts/recognition data to determine a script code for a character language-agnostically. Specifically, it works as follows: Convert each character to a codepoint. Increment the counter for the script code if the codepoint is in the list of individual characters, or if it is in one of the defined ranges in the 4096-character block that it belongs to. Each script has a two-part counter, for primary and secondary matches. Primary matches are when the script is the first one listed; otherwise, it's a secondary match. When comparing scripts, first the total of both are compared (i.e. the overall number of matches). If these are the same, the number of primary and then secondary matches are used as tiebreakers. For example, this is used to ensure that Grek
takes priority over Polyt
if no characters which exclusively match Polyt
are found, as Grek
is a subset of Polyt
. If none_is_last_resort_only
is specified, this will never return "None"
if any characters in text
belong to a script. Otherwise, it will return "None"
if there are more characters that don't belong to a script than belong to any individual script. (FIXME: This behavior is probably wrong, and none_is_last_resort_only
should probably become the default.)
Script objects
A Script
object is returned from one of the functions above. It is a Lua representation of a script and the data associated with it. It has a number of methods that can be called on it, using the :
syntax. For example:
local m_scripts = require("Module:scripts")
local sc = m_scripts.getByCode("Latn")
local name = sc:getCanonicalName()
-- "name" will now be "Latin"
Script:getCode
function Script:getCode()
Returns the script code of the script. Example: "Cyrl"
for Cyrillic.
Script:getCanonicalName
function Script:getCanonicalName()
Returns the canonical name of the script. This is the name used to represent that script on Wiktionary. Example: "Cyrillic"
for Cyrillic.
Script:getDisplayForm
function Script:getDisplayForm()
Returns the display form of the script. For scripts, this is the same as the value returned by :getCategoryName("nocap")
, i.e. it reads "NAME script" (e.g. "Arabic script"
). The displayed text used in :makeCategoryLink
is always the same as the display form.
Script:getOtherNames
function Script:getOtherNames(onlyOtherNames)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:getAliases
function Script:getAliases()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:getVarieties
function Script:getVarieties(flatten)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:getIETFSubtag
function Script:getIETFSubtag()
Returns the IETF subtag used for the script, which should always be a valid ISO 15924 script code. This is used when constructing HTML lang=
tags. The ietf_subtag
value from the script's data file is used, if present; otherwise, the script code is used. For script codes which contain a hyphen, only the part after the hyphen is used (e.g. "fa-Arab"
becomes "Arab"
).
Script:getParent
function Script:getParent()
Returns the parent of the script. Example: "Arab"
for "fa-Arab"
. It returns "top"
for scripts without a parent, like "Latn"
, "Grek"
, etc.
Script:getSystemCodes
function Script:getSystemCodes()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:getSystems
function Script:getSystems()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:isSystem
function Script:isSystem(...)
Check whether the script is of type system
, which can be a writing system code or object. If multiple systems are passed, return true if the script is any of the specified systems.
Script:hasType
function Script:hasType(...)
Given a list of types as strings, returns true if the script has all of them.
Currently the only possible type is script
; use hasType("script")
to determine if an object that may be a language, family or script is a script.
Script:getCategoryName
function Script:getCategoryName(nocap)
Returns the name of the main category of that script. Example: "Cyrillic script"
for Cyrillic, whose category is at Category:Cyrillic script. Unless optional argument nocap
is given, the script name at the beginning of the returned value will be capitalized. This capitalization is correct for category names, but not if the script name is lowercase and the returned value of this function is used in the middle of a sentence. (For example, the script with the code Semap
has the name "flag semaphore"
, which should remain lowercase when used as part of the category name Category:Translingual letters in flag semaphore but should be capitalized in Category:Flag semaphore templates.) If you are considering using getCategoryName("nocap")
, use getDisplayForm()
instead.
Script:makeCategoryLink
function Script:makeCategoryLink()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:getWikidataItem
function Script:getWikidataItem()
Returns the Wikidata item id for the script or nil
. This corresponds to the the second field in the data modules.
Script:getWikipediaArticle
function Script:getWikipediaArticle(noCategoryFallback, project)
Returns the name of the Wikipedia article for the script. project
specifies the language and project to retrieve the article from, defaulting to "enwiki"
for the English Wikipedia. Normally if specified it should be the project code for a specific-language Wikipedia e.g. "zhwiki" for the Chinese Wikipedia, but it can be any project, including non-Wikipedia ones. If the project is the English Wikipedia and the property wikipedia_article
is present in the data module it will be used first. In all other cases, a sitelink will be generated from :getWikidataItem
(if set). The resulting value (or lack of value) is cached so that subsequent calls are fast. If no value could be determined, and noCategoryFallback
is false
, :getCategoryName
is used as fallback; otherwise, nil
is returned. Note that if noCategoryFallback
is nil
or omitted, it defaults to false
if the project is the English Wikipedia, otherwise to true
. In other words, under normal circumstances, if the English Wikipedia article couldn't be retrieved, the return value will fall back to a link to the script's category, but this won't normally happen for any other project.
Script:getCommonsCategory
function Script:getCommonsCategory()
Returns the name of the Wikimedia Commons category page for the script.
Script:getCharacters
function Script:getCharacters()
Returns the charset defining the script's characters from the script's data file. This can be used to search for words consisting only of this script, but see the warning above.
Script:countCharacters
function Script:countCharacters(text)
Returns the number of characters in the text that are part of this script. Note: You should never assume that text consists entirely of the same script. Strings may contain spaces, punctuation and even wiki markup or HTML tags. HTML tags will skew the counts, as they contain Latin-script characters. So it's best to avoid them.
Script:hasCapitalization
function Script:hasCapitalization()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:hasSpaces
function Script:hasSpaces()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:isTransliterated
function Script:isTransliterated()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:sortByScraping
function Script:sortByScraping()
Returns true if the script is (sometimes) sorted by scraping page content, meaning that it is sensitive to changes in capitalization during sorting.
Script:getDirection
function Script:getDirection()
Returns the text direction. Horizontal scripts return "ltr"
(left-to-right) or "rtl"
(right-to-left), while vertical scripts return "vertical-ltr"
(vertical left-to-right) or "vertical-rtl"
(vertical right-to-left).
Script:getRawData
function Script:getRawData()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:hasNormalizationFixes
function Script:hasNormalizationFixes()
Returns true
if the script contains characters that require fixes to Unicode normalization under certain circumstances, false
if it doesn't.
Script:fixDiscouragedSequences
function Script:fixDiscouragedSequences(text)
Corrects discouraged sequences of Unicode characters to the encouraged equivalents.
Script:toFixedNFC
function Script:toFixedNFC(text)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:toFixedNFD
function Script:toFixedNFD(text)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:toFixedNFKC
function Script:toFixedNFKC(text)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:toFixedNFKD
function Script:toFixedNFKD(text)
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Script:toJSON
function Script:toJSON()
This function lacks documentation. Please add a description of its usages, inputs and outputs, or its difference from similar functions, or make it local to remove it from the function list.
Subpages
See also
local export = {}
local combining_classes_module = "Module:Unicode data/combining classes"
local data_module = "Module:scripts/data"
local debug_track_module = "Module:debug/track"
local json_module = "Module:JSON"
local language_like_module = "Module:language-like"
local languages_error_module = "Module:languages/error"
local scripts_by_name_module = "Module:scripts/by name"
local scripts_chartoscript_module = "Module:scripts/charToScript"
local string_utilities_module = "Module:string utilities"
local writing_systems_module = "Module:writing systems"
local concat = table.concat
local insert = table.insert
local load_data = mw.loadData
local match = string.match
local select = select
local toNFC = mw.ustring.toNFC
local toNFD = mw.ustring.toNFD
local toNFKC = mw.ustring.toNFKC
local toNFKD = mw.ustring.toNFKD
local type = type
--[==[
Loaders for functions in other modules, which overwrite themselves with the target function when called. This ensures modules are only loaded when needed, retains the speed/convenience of locally-declared pre-loaded functions, and has no overhead after the first call, since the target functions are called directly in any subsequent calls.]==]
local function explode(...)
explode = require(string_utilities_module).explode_utf8
return explode(...)
end
local function get_writing_system(...)
get_writing_system = require(writing_systems_module).getByCode
return get_writing_system(...)
end
local function gsplit(...)
gsplit = require(string_utilities_module).gsplit
return gsplit(...)
end
local function languages_error(...)
languages_error = require(languages_error_module)
return languages_error(...)
end
local function split(...)
split = require(string_utilities_module).split
return split(...)
end
local function to_json(...)
to_json = require(json_module).toJSON
return to_json(...)
end
local function track(...)
track = require(debug_track_module)
return track(...)
end
local function ugsub(...)
ugsub = require(string_utilities_module).gsub
return ugsub(...)
end
local function umatch(...)
umatch = require(string_utilities_module).match
return umatch(...)
end
--[==[
Loaders for objects, which load data (or some other object) into some variable, which can then be accessed as "foo or get_foo()", where the function get_foo sets the object to "foo" and then returns it. This ensures they are only loaded when needed, and avoids the need to check for the existence of the object each time, since once "foo" has been set, "get_foo" will not be called again.]==]
local m_data
local function get_data()
m_data, get_data = load_data(data_module), nil
return m_data
end
local scripts_by_name
local function get_scripts_by_name()
scripts_by_name, get_scripts_by_name = load_data(scripts_by_name_module), nil
return scripts_by_name
end
local Script = {}
--[==[Returns the script code of the script. Example: {{lua|"Cyrl"}} for Cyrillic.]==]
function Script:getCode()
return self._code
end
--[==[Returns the canonical name of the script. This is the name used to represent that script on Wiktionary. Example: {{lua|"Cyrillic"}} for Cyrillic.]==]
function Script:getCanonicalName()
return self._rawData[1] or self._rawData.canonicalName
end
--[==[Returns the display form of the script. For scripts, this is the same as the value returned by <code>:getCategoryName("nocap")</code>, i.e. it reads "NAME script" (e.g. {{lua|"Arabic script"}}). The displayed text used in <code>:makeCategoryLink</code> is always the same as the display form.]==]
function Script:getDisplayForm()
return self:getCategoryName("nocap")
end
function Script:getOtherNames(onlyOtherNames)
Script.getOtherNames = require(language_like_module).getOtherNames
return self:getOtherNames(onlyOtherNames)
end
function Script:getAliases()
return self._rawData.aliases or {}
end
function Script:getVarieties(flatten)
Script.getVarieties = require(language_like_module).getVarieties
return self:getVarieties(flatten)
end
--[==[Returns the {{w|IETF language tag#Syntax of language tags|IETF subtag}} used for the script, which should always be a valid {{w|ISO 15924}} script code. This is used when constructing HTML {{code|html|lang{{=}}}} tags. The {{lua|ietf_subtag}} value from the script's data file is used, if present; otherwise, the script code is used. For script codes which contain a hyphen, only the part after the hyphen is used (e.g. {{lua|"fa-Arab"}} becomes {{lua|"Arab"}}).]==]
function Script:getIETFSubtag()
local code = self._ietf_subtag
if code == nil then
code = self._rawData.ietf_subtag or match(self._code, "[^%-]+$")
self._ietf_subtag = code
end
return code
end
--[==[Returns the parent of the script. Example: {{lua|"Arab"}} for {{lua|"fa-Arab"}}. It returns {{lua|"top"}} for scripts without a parent, like {{lua|"Latn"}}, {{lua|"Grek"}}, etc.]==]
function Script:getParent()
return self._rawData.parent
end
function Script:getSystemCodes()
if not self._systemCodes then
local system_codes = self._rawData[3]
if type(system_codes) == "table" then
self._systemCodes = system_codes
elseif type(system_codes) == "string" then
self._systemCodes = split(system_codes, "%s*,%s*", true)
else
self._systemCodes = {}
end
end
return self._systemCodes
end
function Script:getSystems()
if not self._systemObjects then
self._systemObjects = {}
for _, ws in ipairs(self:getSystemCodes()) do
insert(self._systemObjects, get_writing_system(ws))
end
end
return self._systemObjects
end
--[==[Check whether the script is of type `system`, which can be a writing system code or object. If multiple systems are passed, return true if the script is any of the specified systems.]==]
function Script:isSystem(...)
for _, system in ipairs{...} do
if type(system) == "table" then
system = system:getCode()
end
for _, s in ipairs(self:getSystemCodes()) do
if system == s then
return true
end
end
end
return false
end
--function Script:getAllNames()
-- return self._rawData.names
--end
--[==[Given a list of types as strings, returns true if the script has all of them.
Currently the only possible type is {script}; use {{lua|hasType("script")}} to determine if an object that
may be a language, family or script is a script.
]==]
function Script:hasType(...)
local types = self._types
if types == nil then
types = {script = true}
local rawtypes = self._rawData.type
if rawtypes then
for rawtype in gsplit(rawtypes, "%s*,%s*", true) do
types[rawtype] = true
end
end
self._types = types
end
for i = 1, arg.n do
if not types[arg[i]] then
return false
end
end
return true
end
--[==[Returns the name of the main category of that script. Example: {{lua|"Cyrillic script"}} for Cyrillic, whose category is at [[:Category:Cyrillic script]].
Unless optional argument <code>nocap</code> is given, the script name at the beginning of the returned value will be capitalized. This capitalization is correct for category names, but not if the script name is lowercase and the returned value of this function is used in the middle of a sentence. (For example, the script with the code <code>Semap</code> has the name <code>"flag semaphore"</code>, which should remain lowercase when used as part of the category name [[:Category:Translingual letters in flag semaphore]] but should be capitalized in [[:Category:Flag semaphore templates]].) If you are considering using <code>getCategoryName("nocap")</code>, use <code>getDisplayForm()</code> instead.]==]
function Script:getCategoryName(nocap)
local name = self:getCanonicalName()
-- If the name already has "script", "code" or "semaphore" at the end, don't add it.
if not (
name:find("[ %-][Ss]cript$") or
name:find("[ %-][Cc]ode$") or
name:find("[ %-][Ss]emaphore$")
) then
name = name .. " script"
end
if not nocap then
name = mw.getContentLanguage():ucfirst(name)
end
return name
end
function Script:makeCategoryLink()
return "[[:Category:" .. self:getCategoryName() .. "|" .. self:getDisplayForm() .. "]]"
end
--[==[Returns the Wikidata item id for the script or <code>nil</code>. This corresponds to the the second field in the data modules.]==]
function Script:getWikidataItem()
Script.getWikidataItem = require(language_like_module).getWikidataItem
return self:getWikidataItem()
end
--[==[
Returns the name of the Wikipedia article for the script. `project` specifies the language and project to retrieve
the article from, defaulting to {"enwiki"} for the English Wikipedia. Normally if specified it should be the project
code for a specific-language Wikipedia e.g. "zhwiki" for the Chinese Wikipedia, but it can be any project, including
non-Wikipedia ones. If the project is the English Wikipedia and the property {wikipedia_article} is present in the data
module it will be used first. In all other cases, a sitelink will be generated from {:getWikidataItem} (if set). The
resulting value (or lack of value) is cached so that subsequent calls are fast. If no value could be determined, and
`noCategoryFallback` is {false}, {:getCategoryName} is used as fallback; otherwise, {nil} is returned. Note that if
`noCategoryFallback` is {nil} or omitted, it defaults to {false} if the project is the English Wikipedia, otherwise
to {true}. In other words, under normal circumstances, if the English Wikipedia article couldn't be retrieved, the
return value will fall back to a link to the script's category, but this won't normally happen for any other project.
]==]
function Script:getWikipediaArticle(noCategoryFallback, project)
Script.getWikipediaArticle = require(language_like_module).getWikipediaArticle
return self:getWikipediaArticle(noCategoryFallback, project)
end
--[==[Returns the name of the Wikimedia Commons category page for the script.]==]
function Script:getCommonsCategory()
Script.getCommonsCategory = require(language_like_module).getCommonsCategory
return self:getCommonsCategory()
end
--[==[Returns the charset defining the script's characters from the script's data file.
This can be used to search for words consisting only of this script, but see the warning above.]==]
function Script:getCharacters()
return self.characters or nil
end
--[==[Returns the number of characters in the text that are part of this script.
'''Note:''' You should never assume that text consists entirely of the same script. Strings may contain spaces, punctuation and even wiki markup or HTML tags. HTML tags will skew the counts, as they contain Latin-script characters. So it's best to avoid them.]==]
function Script:countCharacters(text)
local charset = self._rawData.characters
if charset == nil then
return 0
end
return select(2, ugsub(text, "[" .. charset .. "]", ""))
end
function Script:hasCapitalization()
return not not self._rawData.capitalized
end
function Script:hasSpaces()
return self._rawData.spaces ~= false
end
function Script:isTransliterated()
return self._rawData.translit ~= false
end
--[==[Returns true if the script is (sometimes) sorted by scraping page content, meaning that it is sensitive to changes in capitalization during sorting.]==]
function Script:sortByScraping()
return not not self._rawData.sort_by_scraping
end
--[==[Returns the text direction. Horizontal scripts return {{lua|"ltr"}} (left-to-right) or {{lua|"rtl"}} (right-to-left), while vertical scripts return {{lua|"vertical-ltr"}} (vertical left-to-right) or {{lua|"vertical-rtl"}} (vertical right-to-left).]==]
function Script:getDirection()
return self._rawData.direction or "ltr"
end
function Script:getRawData()
return self._rawData
end
--[==[Returns {{lua|true}} if the script contains characters that require fixes to Unicode normalization under certain circumstances, {{lua|false}} if it doesn't.]==]
function Script:hasNormalizationFixes()
return not not self._rawData.normalizationFixes
end
--[==[Corrects discouraged sequences of Unicode characters to the encouraged equivalents.]==]
function Script:fixDiscouragedSequences(text)
if self:hasNormalizationFixes() then
local norm_fixes = self._rawData.normalizationFixes
local to = norm_fixes.to
if to then
for i, v in ipairs(norm_fixes.from) do
text = ugsub(text, v, to[i] or "")
end
end
end
return text
end
do
local combining_classes
-- Obtain the list of default combining classes.
local function get_combining_classes()
combining_classes, get_combining_classes = load_data(combining_classes_module), nil
return combining_classes
end
-- Implements a modified form of Unicode normalization for instances where there are identified deficiencies in the default Unicode combining classes.
local function fixNormalization(text, self)
if not self:hasNormalizationFixes() then
return text
end
local norm_fixes = self._rawData.normalizationFixes
local new_classes = norm_fixes.combiningClasses
if not (new_classes and umatch(text, "[" .. norm_fixes.combiningClassCharacters .. "]")) then
return text
end
text = explode(text)
-- Manual sort based on new combining classes.
-- We can't use table.sort, as it compares the first/last values in an array as a shortcut, which messes things up.
for i = 2, #text do
local char = text[i]
local class = new_classes[char] or (combining_classes or get_combining_classes())[char]
if class then
repeat
i = i - 1
local prev = text[i]
if (new_classes[prev] or (combining_classes or get_combining_classes())[prev] or 0) < class then
break
end
text[i], text[i + 1] = char, prev
until i == 1
end
end
return concat(text)
end
function Script:toFixedNFC(text)
return fixNormalization(toNFC(text), self)
end
function Script:toFixedNFD(text)
return fixNormalization(toNFD(text), self)
end
function Script:toFixedNFKC(text)
return fixNormalization(toNFKC(text), self)
end
function Script:toFixedNFKD(text)
return fixNormalization(toNFKD(text), self)
end
end
function Script:toJSON()
if not self._types then
self:hasType()
end
local types = {}
for type in pairs(self._types) do
insert(types, type)
end
local ret = {
canonicalName = self:getCanonicalName(),
categoryName = self:getCategoryName("nocap"),
code = self._code,
otherNames = self:getOtherNames(true),
aliases = self:getAliases(),
varieties = self:getVarieties(),
type = types,
direction = self:getDirection(),
characters = self:getCharacters(),
parent = self:getParent(),
systems = self:getSystemCodes(),
wikipediaArticle = self._rawData.wikipedia_article,
}
return to_json(ret)
end
Script.__index = Script
function export.makeObject(code, raw_data)
return raw_data and setmetatable({
_rawData = raw_data,
_code = code,
characters = raw_data.characters
}, Script) or nil
end
--[==[Finds the script whose code matches the one provided. If it exists, it returns a {{lua|Script}} object representing the script. Otherwise, it returns {{lua|nil}}, unless <span class="n">paramForError</span> is given, in which case an error is generated. If <code class="n">paramForError</code> is {{lua|true}}, a generic error message mentioning the bad code is generated; otherwise <code class="n">paramForError</code> should be a string or number specifying the parameter that the code came from, and this parameter will be mentioned in the error message along with the bad code.]==]
function export.getByCode(code, paramForError, disallowNil)
-- Track uses of paramForError, ultimately so it can be removed, as error-handling should be done by [[Module:parameters]], not here.
if paramForError ~= nil then
track("scripts/paramForError")
end
if code == nil and not disallowNil then
return nil
end
local retval = export.makeObject(code, (m_data or get_data())[code])
if not retval and paramForError then
languages_error(code, paramForError, "script code", nil, "not real lang")
end
return retval
end
function export.getByCanonicalName(name)
return export.getByCode((scripts_by_name or get_scripts_by_name())[name])
end
--[==[
Takes a codepoint or a character and finds the script code (if any) that is
appropriate for it based on the codepoint, using the data module
[[Module:scripts/recognition data]]. The data module was generated from the
patterns in [[Module:scripts/data]] using [[Module:User:Erutuon/script recognition]].
Converts the character to a codepoint. Returns a script code if the codepoint
is in the list of individual characters, or if it is in one of the defined
ranges in the 4096-character block that it belongs to, else returns "None".
]==]
function export.charToScript(char)
export.charToScript = require(scripts_chartoscript_module).charToScript
return export.charToScript(char)
end
--[==[
Returns the code for the script that has the greatest number of characters in `text`. Useful for script tagging text
that is unspecified for language. Uses [[Module:scripts/recognition data]] to determine a script code for a character
language-agnostically. Specifically, it works as follows:
Convert each character to a codepoint. Increment the counter for the script code if the codepoint is in the list
of individual characters, or if it is in one of the defined ranges in the 4096-character block that it belongs to.
Each script has a two-part counter, for primary and secondary matches. Primary matches are when the script is the
first one listed; otherwise, it's a secondary match. When comparing scripts, first the total of both are compared
(i.e. the overall number of matches). If these are the same, the number of primary and then secondary matches are
used as tiebreakers. For example, this is used to ensure that `Grek` takes priority over `Polyt` if no characters
which exclusively match `Polyt` are found, as `Grek` is a subset of `Polyt`.
If `none_is_last_resort_only` is specified, this will never return {"None"} if any characters in `text` belong to a
script. Otherwise, it will return {"None"} if there are more characters that don't belong to a script than belong to
any individual script. (FIXME: This behavior is probably wrong, and `none_is_last_resort_only` should probably
become the default.)
]==]
function export.findBestScriptWithoutLang(text, none_is_last_resort_only)
export.findBestScriptWithoutLang = require(scripts_chartoscript_module).findBestScriptWithoutLang
return export.findBestScriptWithoutLang(text, none_is_last_resort_only)
end
return export