Ab Initio - V1.3

Download as ppt, pdf, or txt
Download as ppt, pdf, or txt
You are on page 1of 37

Ab Initio Training

Data Manipulation Language


Objectives
• Introduction to Data Manipulation
Language
• Simple and Compound Data Types
• Key Specifiers
• Expressions
• Transformation Functions
• In-built Functions
Data Manipulation Language
What is a DML?
DML is the language that is used by Ab Initio to
describe, interpret and manipulate data.
What is the use of a DML

• Interpreting Data – Types

• Grouping and ordering – Key Specifiers

• Computations of a single values – Expressions

• Transformations – Transform computations

• Collections of related types, variables or


transform functions - Packages.
Types

• Types are used to interpret the data from the


staging area.
• Data types that are supported in Ab Initio

– Base Types
The base types are atomic types which can used to
store data.

- Compound Types
The compound types are made up of atomic types
or Base types.
Hierarchy of types
Void Types

• Used to store data which has internal


structure that is unnecessary to mention.

• Examples: A compressed file

• Syntax: void (integer constant)

• Example: void(10)
Integer Types
• Interprets the data as an integer

• Syntax: [signed|unsigned][little endian |big endian]


integer(integer- constant)

• Endian – The way in which the values are stored in the


computer.
• Signed – Includes a sign bit

• Unsigned – Includes no signed bytes

• Example: integer(10)
Decimal Types
• SYNTAX
[character set|packed] [decimal(size)|
decimal(size,zoned) |
decimal(size,position) |
decimal(size,position,zoned)]
• Examples:
decimal(8)
decimal(‘,’)
packed decimal(2)
decimal(3,0,zoned)
• Punctuation marks are not considered
String Types
• Syntax: [character set] string(length)
• Fixed String Examples

– String(20) var_name;

– Ebcdic string(10) var_name;

• Delimited Strings

– Ascii string(‘,’) var_name;

• Varying Length Strings


– Data is : 04abcd

– DML is : string(decimal(2)) str;


Date Types
• Date Syntax :

[Character Set] date(format string[,split = split_string])


[(delimiter_string)]

• Examples
date(“YYYYMMDD”); - 19060902
date(YYYY/MM/DD); - 1998/02/01
date(YYMMDD,split = “500101”);

• Split – This is used to modify the century window.


Date-time types

• Syntax: [character set] datetime


(format_string[,split=split-string][(delimiter string)]

• Examples:
ebcdic string(“YYYY-MMM-
DD:hh:mi:ss”)(“,”) timestamp;

• The split string is used to specify the century


window.
Vectors
• Vectors are equivalent to arrays

• Syntax : element_type [expression]

• Expressions – can be integer constants

• Expressions – can be delimiters

• Examples
– Unsigned integer(4) [7][3]
Record Types
• Record type is a compound type
• Simple record types
record
field_type field1;
field_type field2;
end

• Default Character types for Records


EBCDIC record
decimal(9) id;
string (10) first_name, last_name;
end
Record Types …..
• Record wise Delimiters
– Syntax : record … end (Delimiter string)

– Rules that are to be followed

• All the records must end with a delimiter string


• A delimiter string must not occur within a record

– Example

Record
string(“,”) first_name,last_name;
decimal(3) age;
End(“\n”)
Record Types

• Null Default values


record
integer a;
integer b = NULL(-1);
string(4) d = NULL(“ “);
end
Record Types …

• Function fields in record types

Record
string(6) item_name;
decimal(7,2) price;
decimal(3,0) quantity;
decimal(7,2) total() = price* quantity;
end
Field-validity
checking functions
• This is used to check the validity of the variables
present in the record
• Rules:
– Must begin with a is_valid;
– Must not have any arguments
– This must have a return type which is int
– Must be defined within the record .. End pair
• Example
Record
string(3) a;
int is_valid_a() = ! Is_blank(a);
End
Record types .. ..
• Conditional fields

– The structure of the data is defined according to the

– Example

Record
decimal(1) type_of_rec;
if (type_of_rec == 1)
string(10) name;
else
string(20) item_name;
end
Unions

• Unions are used to share the same part of the


memory
• Same as unions in C language
• All the field names must be of fixed types
– Example
Union
ascii string(3) str;
record
ascii string(1) a,b,c;
end
end
Used defined types
• Used to define a type with the other base and
compound types

• Syntax: type type_name =type_specifier

• Example
type address_type =
record
string(10) street1,street2;
string(12) city;
string(9) state;
end;
Key Specifiers
What are Key Specifiers
• Key specifiers are used to order, partion or group
records

• Used in the components that process records in


groups

• Three types of key specifiers


– Key Specifier for fields
– Direction modifiers
– Sequence modifiers
– Advanced Look up modifiers
Key Specifiers for Files
• Specifies the field name alone

• Example

{age;firstname}

Used in the SORT components


Directional Specifiers

• Directional Specifiers are used to specify the order of


the fields for processing

• Examples:
{ last_name; first_name;age descending}
Sequence Specifiers
• Specifies the priority

• Two types of Sequence Specifiers


– Pre-defined Sequence Specifiers
• Phonebook
• Index
• Machine

– User defined sequence specifiers


• Example
– {address alpha “,.” digit}
• Ignoring Characters
– {address alpha “,.” digit ignore whitespaces}
• Making Case insensitive
– {address alpha “,.” digit foldcase}
Expressions

• An expression is a simple computation using values


from constants , variables or other expressions to
form a new value

• An Expression can result in


– A new value
– An error
– NULL
Simple Expressions

• Constants
– Integer Constants
• 1,+12,-123,0x012,0o12,0b1101101
– Real Constants
• +12,-45.6,2E6
– Decimal Constants
• 12.3d
• +0.01D
• 11d
– String Constants
• “Characters”
• “Hi Hutch”
• “(‘)This is abInitio(‘)”
Simple Expressions ….
• Variable References
– Normal variables
– Follows the C rules of naming the variables
– The ‘let’ statement
– Local and Global Variables
• NULL expressions
• Environment Variables
– Preceded with a $ sign
– Used to pass application wide parameters to the graphs
– Evaluated at the start time of the graph
• This_record expression
– Refers to the whole record
Extraction Expressions
• Element Expressions

Used to extract the data from the variables

dependents[I+3]

dependents[(I*12)+1)

• Field Expression

Used to deal with the fields in the records

record_type.field_name;
Operator Expressions
• Unary Expressions
+, - , ! , ~

• Binary Operators
+,-,*, / , % Arithmetic operators
==, !=, <, >, <=, >= Comparison Operators
||, && Logical Operators
|,^,& Bit Wise operators
<<,>> Shift operators

• Ternary Operators
Test expression ? Exp1: Exp 2 If statement
Type Casting Expressions
• Type casting
– Implicit Type casting

– Explicit Type Casting

- Restrictions

- Examples

Bd_Var1 = (decimal(2)) st_var;

- REINTERPRET_AS
Conditional Expressions
• If- Else Statements

– If (expression)
statements

• For statement

– For (I , I<N)
statements

• While Statements

- while (conditional_expression)
statements;
Transformation Functions
• Used to code the business logic in the system

• Structure of the transformation function


begin
local variable declaration
statement list
rule list
end
Rules-Transformation Rules
• Syntax:
Out_name[,Out_name] :priority:in_expression[,in_expression];
• Examples
– Master.ssn :: source.ssn;
– Out.address.street :: in.street;
– out,.id,out.name :: in.id,in.name;
– Out.* :: in.*;
• Priorities
– Examples
– Master.ssn :1: source1.ssn;
master.ssn :2: source2.ssn;
master.ssn :3: “999”;
Inbuilt functions
• CELING

• FLOOR

• MATH_ABS

• DECIMAL_ROUND

• REINTERPRET_AS

• STRING_CHAR

• STRING_REPLACE

• STRING_SUBSTRING

You might also like