A computational framework for colour metrics and colour space transforms

Transformation of colour data

Transformations between different colour spaces can in general take the shape of a function, $\bar{x}=\bar{x}\left(x\right)$, where x = (x₁, x₂, x₃)^T represents a colour, i.e., a point in a colour space. Fortunately, most common colour space conversions are made up of a small set of relatively simple mathematical operations. The linear transformation is a very common ingredient in the transforms. Some colour spaces, such as, e.g., the CIECAT02 colour adaptation space (Moroney et al., 2002), are even defined simply by a linear transformation from some other colour space: (1)${\displaystyle \bar{x}=Ax,}$ where A is a 3 × 3 constant matrix. Combined with the so-called gamma correction, which is applied channel-wise, most RGB type colour spaces, and also, e.g., the IPT (Ebner & Fairchild, 1998) colour space can be construced (2)${\displaystyle {\bar{x}}_i=\mathrm{sgn}\left(x_i\right)|x_i{|}^{\gamma },}$ where γ > 0 is the constant exponent.

For many perceptual colour spaces such as CIELAB, both Cartesian and cylindrical coordinates are commonly used for describing the chromatic plane. The transformation from Cartesian to polar is (3)${\displaystyle \begin{array}{c}{\bar{x}}_1=x_1,\hfill \\ {\bar{x}}_2=\sqrt{x_2^2+x_3^2},\hfill \\ {\bar{x}}_3=\mathrm{atan2}\left(x_3,x_2\right),\hfill \end{array}}$ with the corresponding inverse transform (4)${\displaystyle \begin{array}{c}{\bar{x}}_1=x_1,\hfill \\ {\bar{x}}_2=x_{2}cos\left(x_3\right),\hfill \\ {\bar{x}}_3=x_{2}sin\left(x_3\right).\hfill \end{array}}$ Chromaticities and luminances are often represented in projective spaces such as xyY, (5)${\displaystyle \begin{array}{c}{\bar{x}}_1=\frac{x_1}{x_1+x_2+x_3},\hfill \\ {\bar{x}}_2=\frac{x_2}{x_1+x_2+x_3},\hfill \\ {\bar{x}}_3=x_2.\hfill \end{array}}$ Colour spaces used for colour metrics such as ΔE_E (Oleari, Melgosa & Huertas, 2009) and the various DIN99 metrics (Cui et al., 2002) often include a logarithmic compression of some or all of the channels such as lightness and chroma (radius in polar coordinates): (6)${\displaystyle {\bar{x}}_i=a_{i}ln\left(1+b_i{x}_i\right),}$ where a_i and b_i are the parameters of the transform. Recently, the Poincaré disk representation of the hyperbolic plane has been used for representing the chromatic plane (Lenz, Carmona & Meer, 2007; Farup, 2014). The chroma-preserving mapping to the Poincaré disk can be written as a mapping of the radius in polar coordinates as (7)${\displaystyle {\bar{x}}_2=tanh\left(\frac{x_2}{2R}\right),}$ where R > 0 is the radius of curvature. Besides these more generic transformations, various non-linear transformation functions specific to individual colour spaces are used in such cases as sRGB, CIELAB, CIELUV, the underlying colour space of the CIEDE2000 metric, etc.

Transformation of tensorial data

Most colour metrics can be represented in the form of a line element, or a differential quadratic form (Wyszecki & Stiles, 1982, Chapter 8.4), as (8)${\displaystyle d{s}^2=d{x}^T\mathrm{G}dx.}$ Here, G is the metric tensor—a function of the coordinates. For metrics defined as Euclidean distances in a given colour space, the metric tensor is the identity tensor, I, in the given space. Some colour metrics, like, e.g., CIEDE2000, cannot be written in this form, but can be linearised—or Riemannised—to a good approximation (Pant & Farup, 2012).

Under a coordinate transformation, $\bar{x}=\bar{x}\left(x\right)$, this metric transforms according to (9)${\displaystyle d{s}^2=d{x}^T\mathrm{G}dx=d{\bar{x}}^T{\frac{\partial x}{\partial \bar{x}}}^T\mathrm{G}\frac{\partial x}{\partial \bar{x}}d\bar{x}=d{\bar{x}}^T\bar{\mathrm{G}}d\bar{x},}$ where $\partial x∕\partial \bar{x}$ is the Jacobian matrix of the coordinate transform with componentns $\partial x_i∕\partial {\bar{x}}_j$. In other words, the metric tensor transforms according to (10)${\displaystyle \bar{\mathrm{G}}={\frac{\partial x}{\partial \bar{x}}}^T\mathrm{G}\frac{\partial x}{\partial \bar{x}}.}$ Under composition of several coordinate transformations, $\tilde{x}=\tilde{x}\left(\bar{x}\right)=\tilde{x}\left(\bar{x}\left(x\right)\right)$, the process is nested, (11)${\displaystyle d{s}^2=d{\tilde{x}}^T{\frac{\partial \bar{x}}{\partial \tilde{x}}}^T{\frac{\partial x}{\partial \bar{x}}}^T\mathrm{G}\frac{\partial x}{\partial \bar{x}}\frac{\partial \bar{x}}{\partial \tilde{x}}d\tilde{x},}$ (12)${\displaystyle \tilde{\mathrm{G}}={\frac{\partial \bar{x}}{\partial \tilde{x}}}^T{\frac{\partial x}{\partial \bar{x}}}^T\mathrm{G}\frac{\partial x}{\partial \bar{x}}\frac{\partial \bar{x}}{\partial \tilde{x}},}$ which can also be seen directly from the chain rule for the Jacobian matrices, (13)${\displaystyle \frac{\partial x}{\partial \tilde{x}}=\frac{\partial x}{\partial \bar{x}}\frac{\partial \bar{x}}{\partial \tilde{x}}.}$

All the points with unit distance from a given central point—a unit ball—constitute an ellipsoid (14)${\displaystyle \Delta x^T\mathrm{G}\Delta x=1.}$ The cross section of this ellipsoid with a principal plane in a given coordinate is obtained by setting the corresponding Δx_i = 0, reducing the ellipsoid to an ellipsis (Pant & Farup, 2012), (15)${\displaystyle \left(\begin{array}{cc}\hfill \Delta x_1\hfill & \hfill \Delta x_2\hfill \end{array}\right)\left(\begin{array}{cc}\hfill g_{11}\hfill & \hfill g_{12}\hfill \\ \hfill g_{21}\hfill & \hfill g_{22}\hfill \end{array}\right)\left(\begin{array}{c}\hfill \Delta x_1\hfill \\ \hfill \Delta x_2\hfill \end{array}\right)=1,}$ with angle θ and semi-axes a and b given by (16)${\displaystyle tan\left(2\theta \right)=\frac{2g_{12}}{g_{11}-g_{22}},}$ (17)${\displaystyle a=\frac{1}{\sqrt{g_{22}+g_{12}cot\theta }},}$ (18)${\displaystyle b=\frac{1}{\sqrt{g_{11}-g_{12}cot\theta }}.}$

Representing colour spaces

The core functionality of the colour space and colour metric transforms is found in the space module. The basic idea in designing the object oriented framework is to realise a colour space as an object, and to facilitate the construction of new such objects by providing classes for transforming new colour spaces from already existing ones. The class hierarchy which constistutes the core of the of the module, is shown in Fig. 2. All boxes represent classes, and the arrows denote class inheritance. Italicized method names indicate methods that should be overridden in a subclass. Details about attributes and auxiliary methods etc. have been left out for readability.

All colour space objects must derive from the abstract Space class, and as such implement the methods to_XYZ and from_XYZ for converting colour data between the XYZ colour space and the colour space represented by the object, and the methods jacobian_XYZ and inv_jacobian_XYZ for computing the corresponding Jacobian matrix and its inverse. The two latter methods are implemented in Space as inverses of each other, so the subclasses only need to implement one of them—the other one can be inherited. The colour data is represented as N × 3 NumPy (Oliphant, 2007) ndarray s, and the Jacobian matrices as N × 3 × 3 ndarray s.

All transformations between colour spaces must go through XYZ, which thus serves a special role, and has a separate class of its own. Here, the transformations are simply the identity transform, and the Jacobian matrices are identity matrices. All other colour spaces will be built by transforming colour data from an already existing colour space, starting from XYZ. To facilitate making the specific transforms, an abstract class Transform is provided. During instantiation of a transformed colour space object, the base space of the transformation has to be set. The virtual methods to_base and from_base for converting colour data to and from the base base, and jacobian_base and inv_jacobian_base for computing the Jacobian matrix and its inverse between the current space and the base space must be provided in the derived classes. The methods to_XYZ, from_XYZ, jacobian_XYZ, and inv_jacobian_XYZ are implemented in the base class Transform using the transformation between the current space and the base space (provided by derived classes) and the corresponding transformations in the base class, see Eq. (12). Hence, there is no need to reimplement these in the derived classes. Finally, the concrete colour space tranforms are implemented as classes TransformXXX derived from Transform. They must all implement the methods to_base, from_base and either jacobian_base or inv_jacobian_base. The remaining methods will be inferred by inheritance. In some cases, though, it is more efficient to provide more methods in order to reduce the computational cost. For example, in TransformLinear, both methods jacobian_base and inv_jacobian_base are provided in order to avoid inverting every single Jacobian matrix for large data sets.

Representing colour and metric data

The colour space objects constructed by the method described above, will convert colour data represented as N × 3 ndarray s (N colour data points). For real-life applications, colours some times come as single data points (3-vectors), some times as lists of colour data (N × 3 matrices), and some times as images (M × N × 3 arrays). In order for the user not having to deal with converting back and forth between these formats, as well as remembering in which colour space all the data is given, a separate class Data for storing colour data has been implemented as part of the data module, cf. Fig. 3. Again, the boxes denote classes. In this module, there are no inheritance relationship between the classes, but they are related by the TensorData class having an attribute of the Data type.

A colour Data object can be instantiated with single colour data, lists of colour data, or colour images in any implemented colour space. The Data object takes the colour space (object) of the data as an argument, and keeps a dictionary of the colour spaces in which the data in question has been computed. When the colour data in a given space is requested (using the get method), it first checks the dictionary whether it has already been computed. If not, it is computed, stored in the dictionary and returned. All the actual computations are taken care of by the hierarcy of colour space objects representing the transforms necessary for building the colour space.

A similar approach is taken for colour metric data in the form of colour tensors, see Fig. 3. In this case, both the locations of the colour metrics (as colour Data), and the metrics themselves are represented in the class. Like for the colour data, a dictionary of the computed tensors is maintained. For the conversion between the different colour spaces, the Jacobian matrices are applied according to Eq. (10). The nested tensor transforms, Eq. (12), are implicitly taken care of by the colour space class hierarchy without the user having to interfere.

Colour metrics, tensors and statistics

The four remaining modules, metric, tensor, statistics, and misc contain separate functions (not part of the class hierarchy) for computing various properties of colour data, colour transforms, and sets of these. The metric module has functions for computing the most common colour metrics, such as the standard CIE ΔE_ab and ΔE_uv metrics, CIEDE2000 (Luo, Cui & Rigg, 2001), the different versions of the DIN99 metric as described by Cui et al. (2002), the log-compressed OSA-UCS metric ΔE_E by Oleari, Melgosa & Huertas (2009), as well as a general Euclidean distance and the Poincare disk metric developed in Reference (Farup, 2014) in any colour space. All these functions take two colour data objects of the same size as arguments, and return an N-vector of colour differences.

The tensor module has functions for computing the metric tensors corresponding to the metrics in the metric package. The functions take one colour Data object as argument, and returns the corresponding TensorData object. The statistics module contains functions for calculating various statistics of colour metric data, such as the STRESS measure (García et al., 2007) and Pant’s R-values (Pant & Farup, 2012; Pant, Farup & Melgosa, 2013). The misc module contains miscellaneous supporting functions.

Computational complexity

The framework has been implemented using NumPy (Oliphant, 2007) ndarray s, and all operations for colour and metric conversion are vectorised. Thus, all the real computations take place using the highly optimised underlying libraries for matrices, such as LAPACK (Anderson et al., 1999) etc. No loops over individual colour data are implemented in the high-level language. Thus, there are only two sources of computational overhead by using the framework.

First, there are the function calls associated with computing the transformations between a given colour space and its bases all the way back to the CIEXYZ colour space. These function calls will only take place once per transformation call. This can be a significant overhead when the data set consists of only one or very few data points, but then the computation is very quick anyway. For bigger colour data sets, such as images, this will represent only very few function calls (given by the number of steps in the transformation from CIEXYZ to the given space), and thus be negligible in comparison with the real computation, which takes place at the highly optimised low-level code.

Secondly, all colour and metric conversions go through the CIEXYZ colour space. When converting between two colour spaces based on a common basis other than CIEXYZ, an unneccesary conversion back and forth between this common basis and CIEXYZ will inevitably take place. It would, in principle, be possible to eliminate this by advanced optimisation techniques, but since the computations are already fast (fractions of a second even for quite large images), and the goal of the framework has been ease of implementation rather than computational efficiency, this has not been prioritized.

Not all of the operations in the statistics module are vectorised, although this would in principle also be possible. The reason for this is that they are mainly meant for colour research applications, and as such, they are not expected to be used in production. For the relevant use in research, data collection etc. will be much more time consuming than the actual computations, so computational efficiency has not been emphasized in this part of the framework.