Ab Initio - V1.3

Ab Initio Training
Data Manipulation Language

Objectives
• Introduction to Data Manipulation
Language
• Simple and Compound Data Types
• Key Specifiers
• Expressions
• Transformation Functions
• In-built Functions
Data Manipulation Language
What is a DML?
DML is the language that is used by Ab Initio to
describe, interpret and manipulate data.
What is the use of a DML
• Interpreting Data – Types
• Grouping and ordering – Key Specifiers
• Computations of a single values – Expressions
• Transformations – Transform computations
• Collections of related types, variables or

transform functions - Packages.
Types
• Types are used to interpret the data from the

staging area.
• Data types that are supported in Ab Initio
– Base Types
The base types are atomic types which can used to
store data.
- Compound Types
The compound types are made up of atomic types
or Base types.
Hierarchy of types
Void Types
• Used to store data which has internal

structure that is unnecessary to mention.
• Examples: A compressed file
• Syntax: void (integer constant)
• Example: void(10)
Integer Types
• Interprets the data as an integer
• Syntax: [signed|unsigned][little endian |big endian]

integer(integer- constant)
• Endian – The way in which the values are stored in the

computer.
• Signed – Includes a sign bit
• Unsigned – Includes no signed bytes
• Example: integer(10)
Decimal Types
• SYNTAX
[character set|packed] [decimal(size)|
decimal(size,zoned) |
decimal(size,position) |
decimal(size,position,zoned)]
• Examples:
decimal(8)
decimal(‘,’)
packed decimal(2)
decimal(3,0,zoned)
• Punctuation marks are not considered
String Types
• Syntax: [character set] string(length)
• Fixed String Examples
– String(20) var_name;
– Ebcdic string(10) var_name;
• Delimited Strings
– Ascii string(‘,’) var_name;
• Varying Length Strings

– Data is : 04abcd
– DML is : string(decimal(2)) str;

Date Types
• Date Syntax :
[Character Set] date(format string[,split = split_string])

[(delimiter_string)]
• Examples
date(“YYYYMMDD”); - 19060902
date(YYYY/MM/DD); - 1998/02/01
date(YYMMDD,split = “500101”);
• Split – This is used to modify the century window.

Date-time types
• Syntax: [character set] datetime

(format_string[,split=split-string][(delimiter string)]
• Examples:
ebcdic string(“YYYY-MMM-
DD:hh:mi:ss”)(“,”) timestamp;
• The split string is used to specify the century

window.
Vectors
• Vectors are equivalent to arrays
• Syntax : element_type [expression]
• Expressions – can be integer constants
• Expressions – can be delimiters
• Examples
– Unsigned integer(4) [7][3]
Record Types
• Record type is a compound type
• Simple record types
record
field_type field1;
field_type field2;
end
• Default Character types for Records

EBCDIC record
decimal(9) id;
string (10) first_name, last_name;
end
Record Types …..
• Record wise Delimiters
– Syntax : record … end (Delimiter string)
– Rules that are to be followed
• All the records must end with a delimiter string

• A delimiter string must not occur within a record
– Example
Record
string(“,”) first_name,last_name;
decimal(3) age;
End(“\n”)
Record Types
• Null Default values

record
integer a;
integer b = NULL(-1);
string(4) d = NULL(“ “);
end
Record Types …
• Function fields in record types
Record
string(6) item_name;
decimal(7,2) price;
decimal(3,0) quantity;
decimal(7,2) total() = price* quantity;
end
Field-validity
checking functions
• This is used to check the validity of the variables
present in the record
• Rules:
– Must begin with a is_valid;
– Must not have any arguments
– This must have a return type which is int
– Must be defined within the record .. End pair
• Example
Record
string(3) a;
int is_valid_a() = ! Is_blank(a);
End
Record types .. ..
• Conditional fields
– The structure of the data is defined according to the
– Example
Record
decimal(1) type_of_rec;
if (type_of_rec == 1)
string(10) name;
else
string(20) item_name;
end
Unions
• Unions are used to share the same part of the

memory
• Same as unions in C language
• All the field names must be of fixed types
– Example
Union
ascii string(3) str;
record
ascii string(1) a,b,c;
end
end
Used defined types
• Used to define a type with the other base and
compound types
• Syntax: type type_name =type_specifier
• Example
type address_type =
record
string(10) street1,street2;
string(12) city;
string(9) state;
end;
Key Specifiers
What are Key Specifiers
• Key specifiers are used to order, partion or group
records
• Used in the components that process records in

groups
• Three types of key specifiers

– Key Specifier for fields
– Direction modifiers
– Sequence modifiers
– Advanced Look up modifiers
Key Specifiers for Files
• Specifies the field name alone
• Example
{age;firstname}
Used in the SORT components

Directional Specifiers
• Directional Specifiers are used to specify the order of

the fields for processing
• Examples:
{ last_name; first_name;age descending}
Sequence Specifiers
• Specifies the priority
• Two types of Sequence Specifiers

– Pre-defined Sequence Specifiers
• Phonebook
• Index
• Machine
– User defined sequence specifiers

• Example
– {address alpha “,.” digit}
• Ignoring Characters
– {address alpha “,.” digit ignore whitespaces}
• Making Case insensitive
– {address alpha “,.” digit foldcase}
Expressions
• An expression is a simple computation using values

from constants , variables or other expressions to
form a new value
• An Expression can result in

– A new value
– An error
– NULL
Simple Expressions
• Constants
– Integer Constants
• 1,+12,-123,0x012,0o12,0b1101101
– Real Constants
• +12,-45.6,2E6
– Decimal Constants
• 12.3d
• +0.01D
• 11d
– String Constants
• “Characters”
• “Hi Hutch”
• “(‘)This is abInitio(‘)”
Simple Expressions ….
• Variable References
– Normal variables
– Follows the C rules of naming the variables
– The ‘let’ statement
– Local and Global Variables
• NULL expressions
• Environment Variables
– Preceded with a $ sign
– Used to pass application wide parameters to the graphs
– Evaluated at the start time of the graph
• This_record expression
– Refers to the whole record
Extraction Expressions
• Element Expressions
Used to extract the data from the variables
dependents[I+3]
dependents[(I*12)+1)
• Field Expression
Used to deal with the fields in the records
record_type.field_name;
Operator Expressions
• Unary Expressions
+, - , ! , ~
• Binary Operators
+,-,*, / , % Arithmetic operators
==, !=, <, >, <=, >= Comparison Operators
||, && Logical Operators
|,^,& Bit Wise operators
<<,>> Shift operators
• Ternary Operators
Test expression ? Exp1: Exp 2 If statement
Type Casting Expressions
• Type casting
– Implicit Type casting
– Explicit Type Casting
- Restrictions
- Examples
Bd_Var1 = (decimal(2)) st_var;
- REINTERPRET_AS
Conditional Expressions
• If- Else Statements
– If (expression)
statements
• For statement
– For (I , I<N)
statements
• While Statements
- while (conditional_expression)
statements;
Transformation Functions
• Used to code the business logic in the system
• Structure of the transformation function

begin
local variable declaration
statement list
rule list
end
Rules-Transformation Rules
• Syntax:
Out_name[,Out_name] :priority:in_expression[,in_expression];
• Examples
– Master.ssn :: source.ssn;
– Out.address.street :: in.street;
– out,.id,out.name :: in.id,in.name;
– Out.* :: in.*;
• Priorities
– Examples
– Master.ssn :1: source1.ssn;
master.ssn :2: source2.ssn;
master.ssn :3: “999”;
Inbuilt functions
• CELING
• FLOOR
• MATH_ABS
• DECIMAL_ROUND
• REINTERPRET_AS
• STRING_CHAR
• STRING_REPLACE
• STRING_SUBSTRING

Ab Initio - V1.3

Uploaded by

Copyright:

Available Formats

Ab Initio - V1.3

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ab Initio - V1.3

Uploaded by

Copyright:

Available Formats

Ab Initio Training

Data Manipulation Language

• Interpreting Data – Types

• Grouping and ordering – Key Specifiers

• Computations of a single values – Expressions

• Transformations – Transform computations

• Collections of related types, variables or

• Types are used to interpret the data from the

• Used to store data which has internal

• Examples: A compressed file

• Syntax: void (integer constant)

• Syntax: [signed|unsigned][little endian |big endian]

• Endian – The way in which the values are stored in the

• Unsigned – Includes no signed bytes

– Ebcdic string(10) var_name;

– Ascii string(‘,’) var_name;

• Varying Length Strings

– DML is : string(decimal(2)) str;

[Character Set] date(format string[,split = split_string])

• Split – This is used to modify the century window.

• Syntax: [character set] datetime

• The split string is used to specify the century

• Syntax : element_type [expression]

• Expressions – can be integer constants

• Expressions – can be delimiters

• Default Character types for Records

– Rules that are to be followed

• All the records must end with a delimiter string

• Null Default values

• Function fields in record types

– The structure of the data is defined according to the

• Unions are used to share the same part of the

• Syntax: type type_name =type_specifier

• Used in the components that process records in

• Three types of key specifiers

Used in the SORT components

• Directional Specifiers are used to specify the order of

• Two types of Sequence Specifiers

– User defined sequence specifiers

• An expression is a simple computation using values

• An Expression can result in

Used to extract the data from the variables

Used to deal with the fields in the records

– Explicit Type Casting

Bd_Var1 = (decimal(2)) st_var;

• Structure of the transformation function

You might also like