Studio 3 Precision Values

Understanding Single and Extended Precision

An overview of the way precision values are handled in Studio 3

PRO-ST3-0001

James Newland
Datamine Software Limited
Technical Author
2 St Cuthbert Street
Wells, Somerset, United Kingdom
Datamine Software Limited
Tel: +44 1749 679299
Fax: +44 1749 670290

This documentation is confidential and may not be reproduced or shown to third parties
without the prior written permission of Datamine Corporate Limited.
© Datamine Corporate Limited
EXECUTIVE SUMMARY
This document outlines the differences between the use of single and extended precision files
in Studio 3.

Studio 3 has taken the concept of mixed precision data management from Studio 2, and
provides a single precision environment by default, with optional support for extended
precision files.

There are implications surrounding the choice of precision value, and for this reason it is
important that a full knowledge of the impact of that choice can be provided. This document
is one resource available to explain the concepts and processes behind precision
management from both a file- and project-perspective.

Studio 3 Precision Settings 1

Contents
1 Overview 3
Purpose of this document 3
Prerequisites 3
Acronyms and Abbreviations 3
More information 3
2 Introduction 4
What is meant by ‘precision’? 4
Which Precision is Correct? 4
Studio 2 Precision Settings 4
Possible Conflicts – Projects and Files 5
3 Studio 3 and Data Precision 6
Choosing Your Precision 6
Studio 3 Default Precision Settings 6
Precision Selection Guidelines 6
Setting Your Project’s Initial Precision 7
Managing Single Precision Projects 8
Managing Extended Precision Projects 8
Going from Single to Extended Precision 9
4 Studio 2 Projects in Studio 3 10
Using Studio 2 and Studio 3 10
5 Summary of SP and EP 11

Appendix A: Precision Technical Information 12

Floating Point Representation 12
Significant Digits 12
Datamine Extended Precision format 12
Precision and Block Models 12
Datamine Special Values 13

Studio 3 Precision Settings 2

1 OVERVIEW
Purpose of this document

This document is intended as a reference guide to enable you to make fully informed
decisions regarding the use of single and/or extended precision projects and/or files in Studio
3.

Prerequisites

You should have some working knowledge of the file structures associated with Datamine
data types. The basic concepts of ‘precision’ are explained, but it would be useful if you also
had an appreciation of how these settings are managed in Studio 2.

Acronyms and Abbreviations

Acronym Description

EP Extended Precision
SP Single Precision
S3 Studio 3
DM Datamine

More information

Other sources of information relating to precision settings are:

 Your Studio 3 online Help system (particularly Studio 3 Help | Using Studio 3 |
General Concepts, and Single and Extended Precision)

 The Studio 3 Migration presentation describes single and extended precision file
implications in detail.

Studio 3 Precision Settings 3

2 INTRODUCTION
What is meant by ‘precision’?

Studio 2 and Studio 3 use the concept of ‘precision’ to determine the resolution of data held
within the underlying Datamine format files associated with data objects. This resolution
comes in one of two forms: single or extended.

Single precision (SP) files are of a lower resolution than extended precision files. This means
that whereas they are generally smaller in size than an equivalent EP version, the reduction
in precision means that they are not as exact, from a numerical accuracy perspective.
However, this may or may not be an implication for the data files you are using.

Extended precision (EP) files (sometimes referred to as ‘double precision’ files), introduced in
Datamine Studio version 2.1, provide a higher level of accuracy than their single precision
equivalents, but are also associated with larger file sizes. Again, this may or may not have an
adverse affect on your operations in Datamine Studio products.

The choice of which precision you use is entirely up to you; generally it is a choice you would
make at the advent of a project, although it is possible to alter the precision values of files
using Studio’s various utilities and dialogs for some (but not all) scenarios.

Being more specific; the difference between SP and EP is in the way numeric values are
stored. Both store numbers as floating point values, but in SP files, this is restricted to 24 bits
of precision, representing 6/7 significant digits, whilst in EP files, this is extended to 53 bits,
representing 16 significant digits.

To put this into context consider that 16 significant figures is sufficient in metres to compare
the circumference of the earth (4e7 metres) to a human hair (1e-4 metres), and still have a
few significant figures left over.

Single precision files permit a maximum of 64 data columns, whereas EP files extend this to
256 possible columns.

For a more technical comparison of SP and EP files, see Appendix A.

Which Precision is Correct?

In short, either. The choice of precision will depend on the type of data you wish to represent,
and your decision may also be based on the processing power you have available in order to
manipulate data files. Just as importantly, if you are intending to use existing data files within
your current project, the precision of these files may determine the type of project you
create. For example, if you are intending to use extended precision files, you will need to
create an extended precision project.

Conversely, if only single precision files are relevant to your project, you should create a
single precision (default) project type. This can be upgraded at a later time (to extended
precision) if required.

Studio 2 Precision Settings

Studio 3 has adopted a simple and user-friendly approach to controlling the precision settings
for projects (and subsequently files), which is shown in Chapter 3, however, as a comparison,
it is worth mentioning that Studio 2 precision was set using environment values. A manual

Studio 3 Precision Settings 4

alteration of the Datamine Environment file (datamine.env, or local.env) to be altered to
force all subsequent actions to be taken place according to the new precision file.

Possible Conflicts – Projects and Files

The existence of two separate precision values, although allowing for a flexible approach to
data management, and easier optimization of working processes, it does raise issues when
mixed precision files exist; to execute Datamine processes and commands, your system
needs to ‘know’ which precision values are being dealt with at any given time, and there are
other implications when adding files to a project.

When a precision setting is made, Datamine effectively ‘swaps’ to that setting for subsequent
operations. In Datamine terms, this means that a different Software Server is used for each
situation. So, if you are working in single precision mode, and you wish to read a file in
extended precision, the active server will not be able to process that resolution of data.

So, it is necessary for Studio 3 to recognize the precision values of each and every project
file, and also to check for the precision values of all incoming files, making sure that no data
resolution conflicts occur in any subsequent operations.

In other words, it is vital that projects are labelled with the relevant precision and all
associated files’ precision values are ‘in agreement’ with that setting.

The way that Studio 3 achieves this is explained in the following section.

Studio 3 Precision Settings 5

3 STUDIO 3 AND DATA PRECISION
Choosing Your Precision

Ideally, you should decide on the resolution of your project when it is created, this will ensure
that all generated files are in tune with the precision of the project in question.

Whereas it is possible to change the resolution of a project from single to extended precision
at any time, you will be unable to ‘downgrade’ the resolution of an extended precision project
to a single precision project using Studio 3 alone (strictly speaking, it is possible to alter the
precision of any file, in any ‘direction’, using the Studio 3 Table Editor, although this practice
should be avoided as it leads to a degradation of data – project files amended in this way
may not provide the required resolution for required results).

Studio 3 Default Precision Settings

By default, all Studio 3 projects provide support for single precision files only to ensure
maximum compatibility with existing Studio 3 projects, and Studio 2 projects, but this can be
changed when a new project is created (see “Creating Projects”, below). You can also create
an extended precision project from a single precision project, but not vice versa. For this
reason, single precision projects are the default setting in Studio 3, as it is with Studio 2.

Precision Selection Guidelines

Before deciding on a precision setting for your project, you should be aware of the following:

 Extended precision projects will support the inclusion of both SP and EP files, however
only extended precision output files will be generated by internal processes.

 Single precision projects will not support the addition of extended precision files, and
will only generate single precision files.

 You can upgrade your project from single to extended precision using Studio 3’s
proprietary functions (although the project will then have to be closed, and re-
opened), but not the other way round.

This situation can be summarised using the following table:

EXTENDED PRECISION PROJECT

Read Write
EP Files Yes Yes
SP Files Yes No
SINGLE PRECISION PROJECT
Read Write
EP Files No No
SP Files Yes Yes

The arrows at the side of the table represent the possible project ‘upgrade’ route.

If an attempt is made to load in a conflicting data file, the appropriate messages will be
issued by Studio 3.

Studio 3 Precision Settings 6

Setting Your Project’s Initial Precision

The Project Wizard controls which precision setting your new project will adopt, with the
addition of a new check box on the first screen:

Fig 1. The Project Properties screen

The default setting for all new Studio 3 projects is for single precision projects. You
must select the Created Extended precision project check box in order to provide
initial support for EP files.

The next screen, Project Files, will only show files that can be loaded into the selected
project type. For example, if you have elected to create a single precision project, you will not
see any extended precision files listed on the Project Files screen:

Fig 2. The Project Files screen

Studio 3 Precision Settings 7

Note that you will also be unable to add any EP files to the list using the Add Files... button if
a single precision project is being created. Click Next and OK to create your project in the
selected precision.

Managing Single Precision Projects

If you have created a single precision project, you will be unable to add EP files to it, using
either the Datamine file loading functions (under Data | Load in the Design window) or by
attempting to use the Data Source Drivers to create extended precision files.

For example, if you have created a single precision project, and use the Data Source Drivers
approach to import data from an Excel® spreadsheet (using File | Add Files to Project |
Imported from Data Source, for example), and then import a text file, you will not have
the opportunity to generate an extended precision file as this option will be disabled:

Fig 3. The Import Files dialog example for a single precision project
Similarly, if you try to load a file from an area outside of the current project directory (using
Data | Load | External Datamine File options), any attempt to load an EP file will result in
a warning message being shown, and the load operation being cancelled:

This dialog will be shown whenever an attempt is made to load an EP file into an SP project.

Managing Extended Precision Projects

If you have created an extended precision project, you can load or import single or extended
precision files. All files generated in EP projects will be extended precision.

Recognising File Precision Types

There are two different levels at which precision can be identified; project and file.

In an extended precision project, different icons are used to represent the

precision of loaded files.

This differentiation can be found in the Project Files control bar. Single
precision files are recognizable by their Datamine diamond logo with an ‘SP’
overlay, as shown in the bottom image on the left. All extended precision
files are shown, in iconic format, as a diamond without an overlay, as shown
in the top image on the left.

Studio 3 Precision Settings 8

In addition to the file differentiation, you can also see the precision of the Project in a similar
manner – single precision projects will have an ‘SP’ overlay icon (and will only contain single-
precision files, whereas extended precision projects will not have an overlay, and may contain
a combination of single and extended precision project files associated – although only
extended precision files will be generated during subsequent operations..

Once you have created an extended precision project, it is strongly recommended that you
leave this precision for the duration of the project. Whereas it is possible to manually alter
the precision value of Datamine files (as opposed to project files) from EP to SP format, this
process will result in the degradation of data, and may cause accuracy levels to fall outside of
acceptable limits.

It is not possible to ‘downgrade’ an EP project file to SP in Studio 3.

Going from Single to Extended Precision

Single precision files can be enhanced to included support for extended precision files.

To do this:

1. Load your single precision project

2. Select File | Settings and on the General screen (shown by default, select the
Support Datamine extended precision files check box. Note that this option is only
available for single precision files, and is shown as selected and disabled for extended
precision files, i.e.:

Single Precision Project Files Option

Extended Precision Project Files Option

3. Select File | Save followed by File | Close.

4. Reopen the project. This closing and reopening will ensure that the software 'engine'
dealing with extended-precision files (known as the ‘Precision Server') is re-started.
Support for extended-precision files will then be available.

Studio 3 Precision Settings 9

4 STUDIO 2 PROJECTS IN STUDIO 3
As with Studio 3, Studio 2 also provides support for extended and single precision files.

When a Studio 2 project is loaded into Studio 3, the type of project that is being imported is
detected automatically, and subsequent work within the project will be on the basis of the
imported precision status; for example, if a single precision Studio 2 file is loaded, you will
not initially have support for extended files, although this can be provided using the
instructions shown in the previous section.

Using Studio 2 and Studio 3

If you are using Studio 2 and Studio 3 to manage your projects, you will need to be aware of
the compatibility issues between the two programs.

 Single precision files can be created in either application, and loaded into either
application, regardless of the project precision setting (Studio 3) or the environment
variable settings (Studio 2).

 You can only load either a single or extended precision file from Studio 2 in Studio 3,
and the project’s precision status will be detected on loading. The relevant Precision
Server will then be activated, and your Studio 3 application will then react accordingly
to attempts to import files of each precision.

 If you create an extended precision file in Studio 2, you will not be able to load it into
a single precision Studio 3 project, but it can be loaded into a Studio 3 extended
precision project.

In brief, Studio 2 files and projects are treated in a similar manner to their Studio 3
equivalents.

Studio 3 Precision Settings 10

5 SUMMARY OF SP AND EP
The following summary diagram outlines the points raised by this document (document taken
from ‘Studio 3 Client Migration’ presentation):

 There are two paths shown in the diagram: the top set of arrows (above the dashed
line) represents the creation of a single precision project (the default option).

 The lower arrow ‘path’ (below the dashed line) represents the creation of an extended
precision project.

 The area in the green rectangle represents precision alteration options and limitations
for existing projects.]

Studio 3 Precision Settings 11

Appendix A: PRECISION TECHNICAL INFORMATION
The following information outlines technical information relating to how floating point values
are stored in both single and extended precision files:

Floating Point Representation

The IEEE-754 standard lays down the following definition of floating point formats:

Floating Lowest value Highest value Mantissa Signific

Point ant
Format Bits
Digits

IEEE 1.17549435E–38 3.402823E38 24 6-7

Single
Precision

IEEE 2.2250738585072014E– 1.797693134862316E 53 16

Double 308 308
Precision

All Datamine software conforms to these representations of floating point numbers as

implemented by the compilers which it uses to create the processor executables.

Significant Digits
As can be seen from the table above the mantissa of a single precision float has 24 bits of
precision while a double has 53 bits. Converting from bits to decimal digits gives 7 decimal
digits for floats, 16 for doubles. For example 24 bits leads to a decimal accuracy of 1/224, or
6x10-8, while 53 bits leads to a decimal accuracy of 1/253, or 1x10-16. While the binary
precision is exact, the decimal ‘mileage’ will vary slightly depending on the actual number
being considered.

Datamine Extended Precision format

The term “Extended Precision” format as used in Datamine Studio version 2.1 relates to the
format used to store floating point values in the Datamine binary file. In the past all floating
point operations were carried out in single precision to reduce memory and speed overheads
of the computations. The original binary format of Datamine files was designed to store single
precision values as efficiently as possible. As processors have evolved the benefits of using
double precision have become available and the format of the Datamine files has been
adjusted to allow the storage of these higher precision numbers. By choosing the Extended
precision server in the Datamine environment the user is assured of using Double precision
accuracy for all internal calculations. By setting the file format to extended precision as well
the user is able to store the results in double precision format without losing any resolution

Precision and Block Models

Block models are indexed by an encoding method that relies on storing the index within a
single floating point value within the Datamine file format. This is the IJK field. This field value
is calculated by the following expression:

IJK = (k - 1) + (j - 1)*Nz + (i - 1)*Nz*Ny

Studio 3 Precision Settings 12

Where Nx,Ny,Nz are the number of cells in each of the directions X,Y,Z and I, j and k the
local model coordinates.

Although this value is an integer it is stored as a floating point value in the Datamine file. The
maximum values that can be used are shown in the table below:

Precision Maximum IJK Value

Single 9,999,999

Extended 2147483646 *

* Note that this is a theoretical limit – Datamine process boundary limits will restrict this.
Under 32-bit Windows, the total memory address space is 2GB.

Datamine Special Values

Datamine has a number of fixed values that are used to represent specific values within the
software. The value of 1.0e+30 was originally chosen to match standards other than IEEE
and is used in both single and extended precision Datamine files.

Symbol/Name Representation Value

+ Top ( ceiling value) 1.0e+30

- Absent (smallest value) -1.0e30

tr Trace (small value not zero) 1.0e-30

Studio 3 Precision Settings 13

 Datamine Software Limited
2 St Cuthbert Street
Wells, Somerset, United Kingdom

Tel: +44 1749 679299

Fax: +44 1749 670290
www.datamine.co.uk

Studio 3 Precision Settings 14