Consolidate Block-Compressed Index: Purpose

Download as pdf or txt
Download as pdf or txt
You are on page 1of 2

Co>Operating System Help > Developer reference > Components > Component reference > CONSOLIDATE BLOCK-

COMPRESSED INDEX

CONSOLIDATE BLOCK-COMPRESSED INDEX

Purpose
CONSOLIDATE BLOCK-COMPRESSED INDEX consolidates the many index generations contained in an ICFF index file. It does not affect the
block-compressed data itself.

The component produces a master index for all the generations of input ICFF data.

Location in the Component Organizer


Datasets/Lookup utilities folder

Requirements
CONSOLIDATE BLOCK-COMPRESSED INDEX has two input ports:

datafile must be connected to the existing ICFF data file

indexfile must be connected to the existing ICFF index file

Recommendations
Information stored in compressed data blocks and their associated indexes can be concatenated as successive generations are added.
Performance decreases as the number of concatenated index generations grows.

On when to use various block compress components


CONSOLIDATE BLOCK-COMPRESSED INDEX builds a single index by combining the existing concatenated index generations in an ICFF index
file. Because this component does not process the data, it is more efficient than MERGE BLOCK-COMPRESSED DATA.

Therefore, you should use CONSOLIDATE BLOCK-COMPRESSED INDEX on block-compressed files only when you can safely assume that
the data is already sorted not only within generations but from each generation to the next. This is the case when the lookup key has
been sequentially assigned.

In the general case, where you cannot assume data is sorted from each generation to the next, use MERGE BLOCK-COMPRESSED
DATA instead.

For best performance


You should reindex concatenated block-compressed data periodically — by consolidating if possible; or by merging the data and rewriting
the ICFF if necessary.
Parameters for CONSOLIDATE BLOCK-COMPRESSED INDEX

key
(key specifier, required)

Describes how the input records are indexed for later use as a lookup file.

additional_options
(unquoted string, optional)

No additional options are recommended at this time.

Runtime behavior of CONSOLIDATE BLOCK-COMPRESSED INDEX


CONSOLIDATE BLOCK-COMPRESSED INDEX takes the several separate index generations residing in an ICFF index file and consolidates them
into a master index:

1. Reads a number of index generations from the index file at its indexfile input port.

2. Reads the data record format from the file at its datafile input port.

3. Bases a key on that data record format.

4. Merges the index information together.

5. Writes the consolidated index to its output port.

NOTE: For sake of efficiency, the consolidated index produced by the CONSOLIDATE BLOCK-COMPRESSED INDEX component does not
retain any of the bitmap information that might have been present in the separate index generations.

Related topics

An inside view of CONSOLIDATE BLOCK-COMPRESSED INDEX

Copyright © 1997-2011, Ab Initio, Confidential and Proprietary. All rights reserved.

You might also like