Ab Initio EME

EME Training Course Sivasakthipriya

 Course Content

EME (Enterprise Meta Environment)

Features of EME
Projects, Sandbox & Parameters
Reports, Versioning and Tagging
Checking out Projects, Sandbox, graphs and files
Editing the Sandbox, graphs and files
Checking in Projects, Sandbox, graphs and files
Dependency analysis
Air commands

May 18, 2010

Features of EME
What is EME
Why EME
Interfaces to the EME
EME Data Store Settings

July 6, 2010
What is EME?
Enterprise Meta Environment is high performance object oriented
storage system that inventories and manages various kinds of
information associated with Ab Initio applications which may range from
design information to operational data.

It acts as the hub for data and definitions.

In simple terms, it is a repository, which contains data about data

metadata.

It consists of two basic types of metadata

Technical metadata application related business rules, record
formats and execution statistics
Business metadata user defined documentation of what you
deem important to track in your company , such as logical
models, data dictionaries, roles and responsibilities.

May 18, 2010

Why EME?
Avoid setup scripts, use EME parameters more standardisation
Source Control
Dependency and impact analysis for the graphs in the repository

Interfaces to the EME

GDE
EME Web interface to Technical Metadata
EME Web interface to Enterprise Metadata

May 18, 2010

EME Data Store Settings
EME Data store is a specific instance of EME in the environment.
This is a repository where different versions of code and its related
data like the record formats, transformations etc are maintained.
At any point of time a user can connect to only one such EME
repository instance.
To access an EME Data store, go to Settings in the GDE Menu and
click on to Enable EME Connections which is shown below.

May 18, 2010

 After clicking the Enable EME Connections , The GDE is
connected to the EME data store. It is shown below.

May 18, 2010

Projects in EME
What is a Project
Project Structure
Structure of project in EME
Different Types of Project
Public project
Private project

July 6, 2010
 What is a Project ?

Physically, each project in the EME data store is a collection of

graphs, plans & related files stored in a well described structure.
Logically, a project is a fairly selfcontained and largely independent
content area , a suite of graphs and associated files that accomplishes
a business goal.

Project structure
The directory tree of a project contains subdirectories for graphs,
scripts, sqls , transforms etc.
/Projects default directory under which specific projects reside.

May 18, 2010

Structure of a project in EME

/Projects mp Sub directory for AI Graphs

run Sub directory for deployed shell scripts

Project1

xfr Sub directory for transform files

Project2
dml Sub directory for record format files

db Sub directory for database interface files

bin Sub directory for tools and utilities

sql Sub directory for sql queries

May 18, 2010

 Different Types of Project
Public Project
A Public project shares their data and metadata with other projects

A Project that is included by other Projects is termed as a Public

Project

Private Project
A Private project are not expected to be shared with other projects

. Projects including public Projects are known as Private Projects.

May 18, 2010

Sandbox
What is a Sandbox
Structure of Sandbox
Sandbox vs Projects in EME
Environment Structure
Creating a Project by creating a Sandbox

July 6, 2010
What is a Sandbox?
Sandboxes are the user work areas in order to work on files in EME
Project
To work on Projects, they must be checked out to a user work area
on the file system where development and modification of code can
happen.
This working area on the file system is known as Sandbox.
After the changes are complete the code is checked in from the user
sandbox area to the EME data store
This action creates a new version of the code in the EME Data store.

May 18, 2010

Structure of Sandbox
Sandbox has the same structure of Projects in the EME but it lies in
different file system. It is shown below.

/Mysandbox mp Sub directory for AI Graphs

run Sub directory for deployed shell scripts

Project1

xfr Sub directory for transform files

Project2
dml Sub directory for record format files

db Sub directory for database interface files

bin Sub directory for tools and utilities

sql Sub directory for sql queries

May 18, 2010

Sandbox vs. Projects in EME
Sandboxes are work areas used to develop, test or run code
associated with a given project. Only one version of the code can
be held within the sandbox at any time.
The EME Datastore contains all versions of the code that have
been checked into it.
A particular sandbox is associated with only one Project where as
a Project can be checked out to a number of sandboxes.

Check-in

Check-out

May 18, 2010

Environment Structure

Common Sandbox
(Optional)
/Common/Project1/mp
EME
Repository
User Sandboxes
/Projects/Project1/mp
/Mysandbox/Project1/mp

check in
Check
inCheckout
in

16

May 18, 2010

Creating a project by creating a Sandbox
Create an empty sandbox in the GDE. Select Project>Create
sandbox dialog.
Host : server name
Directory : The sandbox path
Location parameter : leave default or give some name
Click ok and the sandbox will be created in user area.
Check in the sandbox and put it as Project in EME.

May 18, 2010

Parameters
What is a Parameter
Graph Parameter
Sandbox Parameter or Project parameter

July 6, 2010
What is a Parameter?
A parameter is a name-value pair with some additional attributes that
determine when and how to interpret or resolve its value.

Parameters are used to provide logical names to physical location and

should always be used instead of hard coded paths in graphs.

Parameters are used to change the behaviour of graphs and projects

in a controlled and uniform way

There are two types of parameters.

Graph Parameters
Project Parameters or Sandbox Parameters

May 18, 2010

Graph parameters
Graph parameters are specific to the individual graphs and
are private to them. They affect execution of the graph for
which they have been defined.
Graph parameters can be defined by navigating to
Edit>Parameters in the GDE which opens the graph
parameters editor.

May 18, 2010

 Project Parameters or Sandbox Parameters

Project parameters are inherited by all the graphs in the Project

and are accessed from the GDE by the sandbox parameter editor in
Project>Edit Sandbox>Parameters.
This shows a dialog box prompting to enter the sandbox path.
Choose the correct host and the sandbox path and press OK to
open the sandbox parameter editor, which exactly like the graph
parameter editor shown as above.

May 18, 2010

 Reports
Version control & Tagging

July 6, 2010
Reports
Reports present summary EME data

View the report using EME Web Interface

The below list of reports can be taken from EME

Graph reports
Dataset reports
Job report
Directory report
Project report

May 18, 2010

Version Control and Tagging
Each object under EME source control, which may be a file, a
directory or a Project, exist as a series of versions, each of which is
a representation of what was checked in by some user.
It can optionally have a textual description attached to it called as a
tag and a description as a comment.
Each version is separately numbered and can be accessed by
either the version number or the tag attached to it.
Version numbers, which are integers and tags, are global to the
whole EME Datastore.
Tags are the basic units during migration of code across EME
Datastore instances.

May 18, 2010

Checking out Projects, Sandbox, graphs
and files
Check out
Check out Process in GDE

July 6, 2010
Check out
The Ab Initio GDE provides wizards to check out code from the
EME to sandbox.
Check out updates the sandbox with the particular version of code
that is being checked out from the EME. By default the latest
version of any object is checked out.
Even any version of code can be checked out.
Check out Process
Select Project>Checkout. Specify the Sandbox directory name and
EME path.
Specify the EME name, Project path in EME and Sandbox path in
user area. It is shown below.

May 18, 2010

 After specifying all details click on to check out button.
It will check out all the things from EME to sandbox
To check out a project in existing sandbox , right click the
sandbox name in the sandbox view and select check out
To check out only the current graph or plan , click the check
out current document icon( ) on the GDE menu bar
To check out any file in the project, right click the file in
sandbox view and select check out.

May 18, 2010

Editing the Sandbox, graphs and files
Need for Locking
Locking an Object
Lock Button

July 6, 2010
Need for locking
Lock is used to edit the graph or files.
The lock button on the GDE indicates the status of an object
with regard to locking.
Locking the graph prevents other users from changing it at
the same time you are working on it.
Otherwise , conflicts can occur between sandbox and data
store version of files.
Locking an Object
Clicking the lock button on the GDE button
Choose file> lock on the GDE menu bar
Right click a file on sandbox view and select lock

May 18, 2010

Locking Button
The lock buttons appearance identifies the locking status of the object
you are viewing
If an object has never been checked in , there is no lock associated with it.
The lock button on the tool bar is an open gray lock on gray back ground.
When a graph or file is unlocked , the lock button on its toolbar
is an open yellow lock against a gray back ground. A unlocked
object is read only and cannot be edited in the GDE.
Sometimes an object lock button indicates that another user
already owns the lock , which means you cannot edit the object .
In this case , the lock button is a closed lock on red back ground.
You can click the lock button and display the owner of the lock.

May 18, 2010

 There is no way in the GDE to break another users lock.
Once you set the lock , you are the owner. You can modify ,
save, and check in the locked object. The lock button is now a
closed recessed in green back ground
Locking when sandbox copy of the object is out of date. It will
show below dialog box.

Releasing a lock without checking in changes , it will displays a

dialog. Save changes or release lock without reverting

May 18, 2010

Checking in Projects, Sandbox, graphs and
files
Check in
Check in process in GDE

July 6, 2010
Internal and
Check in
Once you have edited your sandbox files, you have to check
them in to put the new versions under source control in the
EME datastore and make them available to other users.
Check in process in GDE
To check in an entire sandbox , right click the sandbox name
in the sandbox view and select check in , or select
Project>check in on the GDE menu bar.
To check in only the graph , you are currently editing , click
the check in document icon ( ) on the GDE menu bar.
To check in any sandbox file, right click the file in the sandbox
view and select check in.

May 18, 2010

Dependency Analysis

July 6, 2010
Dependency Analysis
Dependency Analysis is a process by which the EME
examines a project in its entirely and traces how data is
transformed and transferred, field by field , from component to
component within and between graphs.
Using Dependency analysis we can observe the following
things:
How operations that occur later in a
graph(downstream) are affected by components
earlier (upstream) in the graph
What data is operated on by each component, and how.
All details of every component in the project.
What happens to each field in a dataset throughout a
graph.
All the graphs that use a particular dataset.

May 18, 2010

There are four ways in which the dependency analysis can be controlled
:

1) It can be directly invoked in the GDE, by selecting Project >

Analysis. To start the Analysis wizard.
2) It can be directly invoked from the command line, by running the air
project analyze- dependencies command.
3) It can be invokes implicitly at check-in, depending on the settings you
select in the Check-in wizards Advanced Options under the Analysis
tab.
4) You can also create an analysis_level parameter and use it to specify
the amount of dependency analysis you wish to take place during
check-in. The setting of this parameter(if it exists) restricts all other
analysis level settings.

May 18, 2010

AIR Commands
Command line syntax of air command
Air Sandbox group
Air Project group
Air object group
Air Lock group
Air repository group
Air Branch group

July 6, 2010
Command line syntax of Air command:
The air utility is a command-line or shell-script interface for administering
and managing the EME.
All air commands have the following command-line syntax.
air[-root root] [-version version | tagname][-remote]
-root root specifies root of the data store.(Optional)
-version version (Optional) specifies the version number of the data
store to be accessed by the air command. The version number can
be specified as an integer or as a tag.
A tag is a symbolic name that you can associate with a specific
version when you check in a project.
-remote specifies that the air utility should start the EME server
remotely even if a direct connection is possible.

May 18, 2010

Air Sandbox group

The air sandbox commands perform operations on sandbox.

air sandbox create creates the sand box in specified path.

syntax- air sandbox create path

Path specifies a directory.

[ -template path ] creates the sandbox from a specified

template sandbox at path.
[-prefix prefix] Add prefix to the predefined sandbox
parameters.
[-replace s1 s2] change one thing to another thing

May 18, 2010

Air Sandbox group

air sandbox diff displays difference between two graphs or two files

syntax- air sandbox diff {path | path1 path 2}

[Path] specifies a directory in which file is located .
{ -version version path | -version version path1 version version
path2} specifies the version which has to be compared
{-text} forces text differencing , regardless types of files.

air sandbox find identifies root of sandbox

syntax- air sandbox find [path]

[Path ] specifies a directory in which to start looking .
{-up|-down}- upward and downward search , towards and
downwards of the directory specified.
{ -sandbox} specifies sandbox found should be listed
{-project} specifies EME project the sandbox is associated with
should be printed.

May 18, 2010

Air Sandbox group

air sandbox info displays info about the files associated with
sandbox
syntax- air sandbox info {path}
[Path] specifies a directory.

air sandbox lock performs lock operation on sandbox objects

syntax- air sandbox lock path
[Path ]the path to file checked in sandbox
[-set|-release|-break|-release].- set locks in a file, releases lock ,
breaks another user lock , resets the lock broken by another user

air sandbox parametercreates sandbox parameter

syntax- air sandbox parameter [pset] parameter [interpretation]
[value] [flag]
[Pset]specifies the location of parameter
[-parameter].- name of the parameter
[interpretation] interpretation used for value.

May 18, 2010

Air Sandbox group

air sandbox pset creates or modifies input value for graph.

syntax- air sandbox pset pset-path
[pset-path] path to input values.
[-create] creates an input value set.

air sandbox revert reverts sandbox object to versions in data store.

syntax- air sandbox revert basedir sandbox
[-basedir] path to sandbox
[-files path ].- sandbox relative paths of object to be reverted

air sandbox run runs a graph or file.

syntax- air sandbox run path
[Path ] path to a graph (.mp ) file, graph task or plan or .pset file

May 18, 2010

Air Project group

This manages the datastore projects.

air project create creates a new project and specifies various project
attributes.

syntax- air project create project name

Project-name name of the project
[ -location location ] name of the project parameter that
specifies the location of the project.
[-prefix prefix] prefix for default project parameters created
automatically.

May 18, 2010

Air Project group

air project export check out or copies projects from datastore into a
sandbox.

syntax- air project export project-path basedir sandbox-root-dir

Projectpath project path and name in datastore
[ -basedir ] specifies an existing or new sandbox

air project import check in or copies sandbox files into an existing

datastore project.

syntax- air project import project-path basedir sandbox-root-dir

Projectpath path to the project to which you want to check in
files
[ -basedir ] path to sandbox

May 18, 2010

Air Project group
air project mkdir creates a directory in specified project.

syntax- air project mkdir [-q] project-name [-no-export] dir-name

-q supress error if directory already exists
[-no-export] the directory is not exportable

air project add Manually adds one or more files to a datastore

projects internal list of files.

syntax- air project add [-q] project-name relative path

-q supress error if relative path already exists
Relative path relative path of file to be added

air project modify edits the specified project location parameter.

syntax- air project modify project-name [-location loc-parameter]

Project-name name of the project
[-location loc-parameter] Name of the project parameter

May 18, 2010

Air Project group

air project move Moves a project from one place to another within
datastore.
syntax- air project move old-name new-name
Old-name datastore path specifying the original location
New-name the path which project to be moved

air project parameter deals with project parameter.

syntax- air project parameter proj-path [-export] parameter

[interpretation] value
Proj-path path to project
-Export exported to environment
Parameter the parameter name you want to add or edit
Interpretation determines how value is interpreted
Value the new value of the parameter .

May 18, 2010

Air Project group

air project revert reverts one or more objects in a Project

syntax- air project revert proj-path version ver-path relative-path
Proj-path path to the project containing objects you want to
revert
-version ver-path version number
Relative-path relative path of the object to be reverted.

air project set-executable ensures that a file will be marked

executable when it is exported.

syntax- air project set-executable proj-path relative-rpath[-not]

Proj-path path to project containing the file
relative-rpath relative path of file within the project
-not specifies than an executable file be marked not
executable.

May 18, 2010

Air Project group

air project set-type overrides the mime type assigned to a single

datastore project
syntax- air project set-type proj-path -force relative-path mime-type
Proj-path path to the project in which object resides
-force by default you cannot set . You have to force.
Relative-path relative path to the object whose MIME type you
want to change.

air project show displays project attributes.

syntax- air project show proj-path

Proj-path path to project

May 18, 2010

Air Object group
It manipulates object and directories in the datastore and control
permission to access objects.

air object access To test whether all permissions exists for the
project.
syntax- air object access rpath [r|w|x|f]
Rpath path to a data store project
-r test whether you can read the specified object.
-w - test whether you can write the specified object
-x - test whether you can execute the specified object
-f test whether you can see the existence of the specified
project.

air object show displays project attributes.

syntax- air project show proj-path

Proj-path path to project

May 18, 2010

Air Object group

air object chown Changes the ownership of one or more datastore

objects. Only the EME administrator can perform this operation.

syntax - air object chown[-Rf]user[.group] rpath......

-R - Optional. Recurses into rpaths if they refer to a directory.

-f - Optional. Attempts to continue even if the operation fails
for one or more rpaths.
user - New owner expressed as a valid user name.
.group - Optional. Group designation, preceded by a period.
rpath - Path to a datastore object, or object ID, whose
ownership you want to change. You can specify multiple
paths.

May 18, 2010

Air Object group

air object cp Creates a copy of the object known as rpath1 in a

location within the datastore. The command copies rpath1, retaining links
to any other objects in references.

syntax - air object cp rpath1 [-R] rpath2

rpath1 Path to a datastore object, or object ID, to be copied.

-R Optional. If specified, command copies the specified

object
rpath1 and all objects referred to by rpath1. If not specified,
the specified object rpath1 is copied, but objects referred to
by rpath1 will be linked but not copied.

rpath2 New datastore path for the copied object.

May 18, 2010

Air Object group

air object create-parameter-set Creates a parameter set file

from the parameters associated with a datastore object.

syntax- air object create-parameter-set file graph_path

file - Name of the parameter set file. It should have an

extension of .pset
graph_path - Path to the graph in the datastore.
switch - The name of a switch parameter.
binding - A value specified for switch.

May 18, 2010

Air Object group

air object delete Marks one or more objects, and other objects in
the same domains, for permanent removal from the datastore.

syntax - air object delete R wholedomain [-force] rpath [

rpath2]

-R - If rpath is a directory, all of its directory child and

grandchildren and all objects in the descendants
domains will be marked for deletion.
-whole-domain - All objects in the same domain as the object
specified by rpath will be similarly removed.
-force - Do not prompt for confirmation of deletion.
rpath rpath2 - Datastore paths to the objects you want to
delete.

May 18, 2010

Air Object group

air object In - Creates an additional name , rpath2, for the object

rpath1.

syntax - air object In rpath1 rpath2

rpath1 Path to a datastore object, or an object ID.

rpath2 New name for the existing object.

air object mv - Renames the object rpath1 to rpath2.

syntax - air object mv rpath1 rpath2

rpath1 Path to the datastore object you want to rename.

rpath2 New name for the object rpath1.

May 18, 2010

Air Object group

air object load - Loads objects from a file in portable

interchange format. This file was created by running the air object save
command. The objects are loaded to the datastore location corresponding
to the ones in which they were saved.

syntax - air object load [-table-of-contents] file-name

file-name - Standard UNIX file-name of the file storing the

objects you want to load. If file-name is -, then
the data is read from standard input.

-table-of-contents - Specifies that nothing be loaded, but

instead print a listing of the files contents.

May 18, 2010

Air Object group

air object revert - Reverts one or more objects to an earlier

version. Objects newer than specified version are deleted from the
datastore.

syntax - air object revert [-revert-links] version rpath

-revert-links - Optional; if specified, then objects linked to rpath

will also be reverted. If not specified, no action is taken on linked
objects.

version - Version number (integer) or tag (string) of the

datastore that you wish to back out to.

rpath - Path to an object, or object ID.

May 18, 2010

Air Object group

air object mkdir - Creates a datastore directory. Parent directories

are created as necessary.
syntax - air object mkdir rpath

rpath Path of the directory you want to create.

air object unlink - Unlinks a directory pathname of an object.

syntax - air object unlink[-f] [-no-project-remove] rpath

-f - Specifies that no error be generated if rpath

does not exist.
-no-project-remove - Specifies that the object be removed
without removing it from source code control.
rpath - Datastore pathname of the directory.

May 18, 2010

Air Object group

air object rm - Removes an entry from a directory. If the entry is in a

datastore project, then the system performs an implicit air project remove.

syntax - air object rm[-f][-r][-no-project-remove] rpath..

-no-project-remove - Specifies the object be removed without

removing it from source code control.
-f - Specifies that no error be generated if
rpath does not exist.
-r - Specifies that subdirectories of rpath be
removed recursively.
rpath - Path to the directory entry you want to
remove.

May 18, 2010

Air Object group

air object save - Saves datastore objects to a file in portable

interchange format. You can subsequently load the file by running air
object load. You can use air object save to migrate objects from one
datastore to another.

syntax - syntax - air object save file-name rpath

[-include{common| local| rpath}]
[-external { common| local| erpath}]
[-format {1| 2| 3}]
[-export epath]
[-settings common| project]
[-save-jobs]
[-analyzed]
[-no-default]
[-tag tag]
[-comment comment]
[-Stag tag]
[Scomment comment]
[ May 18, 2010
Air Object group

air object versions - Displays the revision history of all tagged

versions or all versions of the specified object or the datastore. History
includes version number, date, user, tag, and comment for the specified
object.

syntax - air object versions[-verbose][rpath]

-verbose - Optional. Displays all versions for an object or the

entire datastore. (If nor specified, only tagged versions will be
displayed.)

rpath - Optional. Path to the object about which you want

version information. If not specified, displays
information for the datastore as a whole.

May 18, 2010

Air Lock group
It manages locks to prevent file collision.
air lock set sets or modifies a lock on datastore objects or projects.

syntax- air lock set [-force] [-breakable | -unbreakable] [-auto-

release | manual-release] [-modify attribute] [-object object | -project
project]
Force breaks another lock and sets a lock for you
[ - breakable | -unbreakable] specifies whether other user can
break the lock
[auto-release | manual-release] the lock is automatically has to
be released automatically or not after check in
[-modify attribute] modifies the breakability and release
attributes of an existing lock
-object object is the path which you want to set lock
-Project Project is the path which you want to set lock

May 18, 2010

Air Lock group

air lock release sets or modifies a lock on datastore objects or

projects.

syntax- air lock release [-object object] [-project project]

-object object is the path which you want to set unlock
-Project Project is the path which you want to set unlock
air lock break break users lock on a given datastore object.

syntax- air lock release [-object object] [-project project]

-object object is the path which you want to break
-Project Project is the path which you want to break
air lock show displays a listing of locks for a given user , object or
project.
syntax- air lock show [-user username] [-object object] [-project
project]
-object object is the path which you want to display
-Project Project is the path which you want to display

May 18, 2010

Air Repository group
The air repository commands operate on the storage system of the
datastore

air repository create - creates a datastore and installs certain default

system objects .
air repository chmod change permissions on datastore.
air repository chown change ownership of the datastore.
air repository rename renames all files that makes up an EME
data store.
air repository restart shuts down and restarts the EME process.
air repository start starts the EME process.
air repository shutdown Shuts down the EME process.
air repository backup it is used to backup of datastore

May 18, 2010

Air Branch group
The air repository commands operate on the storage system of the
datastore

air branch create - creates a new branch with the name new-branch
from an existing branch .
air branch delete deletes object on the branch.
air branch freeze freezes or unfreezes a branch .
air branch list displays information about objects on the branch.
air branch modify allows you change the title of the branch .

May 18, 2010

QUESTIONS ?

May 18, 2010

THANK YOU

May 18, 2010 66