Autosys Job Management Reference Guide
Autosys Job Management Reference Guide
com/doc/22606788/Autosys-Tutorial
http://writetrends.files.wordpress.com/2009/09/autosys-edk2uaj45cie.pdf
http://sc.tamu.edu/shortcourses/SC-unix/introToUnix.pdf
AutoSys is used for defining, scheduling and monitoring jobs. These jobs can be a UNIX
script, java program or any other program which can be invoked from shell. Before
starting we assume that user has already setup an AutoSys environment. This
environment consists of autosys server and autosys client.
Event Server
The event server is a AutoSys database which stores all system information and events
as well as all job, monitor, and report definitions. Sometimes this database is also called
as a data server, which actually describes a server instance. That is, it is either a UNIX or
Windows process, and it is associated data space (or raw disk storage), that can include
multiple databases or tablespaces.
Event Processor
This is main component of the autosys system. This processes all the events it reads from
dataserver. The event processor is the program, running either as a UNIX process or as a
Windows service that actually runs AutoSys. It schedules and starts jobs. When you start
the event processor it continually scans the database for events to be processed. When it
finds one, it checks whether the event satisfies the starting conditions for any job in the
database.
Remote Agent
On a UNIX machine, the remote agent is a temporary process started by the event
processor to perform a specific task on a remote (client) machine. On a Windows
machine, the remote agent is a Windows service running on a remote (client) machine
that is directed by the event processor to perform specific tasks.
The remote agent starts the command specified for a given job, sends running and
completion information about a task to the event server, then exits. If the remote agent is
unable to transfer the information, it waits and tries again until it can successfully
communicate with the database.
Explanation
1. The event processor scans the event server for the next event to process. If no event is
ready, the event processor scans again in five seconds.
2. The event processor reads from the event server that an event is ready. If the event is a
STARTJOB event, the job definition and attributes are retrieved from the Event Server,
including the command and the pointer (full path name on the client machine) to the
profile file to be used for the job. In addition, for jobs running on Windows machines, the
event processor retrieves from the database the user IDs and passwords required to run
the job on the client machine.
3. The event processor processes the event. If the event is a STARTJOB, the event
processor attempts to establish a connection with the remote agent on the client machine,
and passes the job attributes to the client machine.
The event processor sends a CHANGE_STATUS event marking in the event server that
the job is in STARTING state.
4. On a UNIX machine, the inetd invokes the remote agent. On a Windows machine, the
remote agent logs onto the machine as the user defined as the job’s owner, using the user
IDs and passwords passed to it from the event processor.
5. The remote agent sends an acknowledgment back to the event processor indicating that
it has received the job parameters. The socket connection is terminated. At this point, the
event processor resumes scanning the event server database, looking for events to
process.
6. The remote agent starts a process and executes the command in the job definition.
7. The remote agent issues a CHANGE_STATUS event marking in the event server that
the job is in RUNNING state.
8. The client job process runs to completion, then returns an exit code to the remote agent
and quits.
JIL stands for Job Information Language. Using this you can instruct autosys to save job
definitions. This information saved in autosys database. You can also create a jil file
which contains job definition. You can then pass this jil file to autosys.
The job owner specifies whose user ID the command will be run under on the client
machine. This attribute is automatically set to the user who invoked jil or the GUI to
define the job, and cannot be changed except by the edit superuser.
1. command: The command attribute can be the name of any command, executable,
UNIX shell script or batch file, and its arguments.
2. machine: This attribute specifies the client machine on which the command should be
run.
3. date_condition: The start date/time dependencies attribute is a toggle, which specifies
whether or not there are date, time, or both, conditions required for starting the job.
4. days_of_week: The days of the week attribute specifies the days on which the job
should be run.
?
1insert_job:echoJob
2machine:unixMachine
3owner:username
4command:echo “Hello this is command job”
To add this job in atosys db. Run following command from unix:
You can create/modify jobs either through the GUI interface (Schedular console) or
you can script it with the JIL command. Here is a sample JIL code.
/* ----------------- sample_ls ----------------- */
Suppose that you save the code into code.txt you would do the following;
jil < code.txt
autorep -j sample_ls
autorep -j sample_ls -q
There are numerous other options and you really should read an Autosys guide to
get an uderstanding of the relationships between jobs and boxes, runstatus, etc.
As far as troubleshooting batch problems, autosys writes an autoremote log file, and
that file is placed where you specify in your setup of autosys. For example, the file
might be in the /tmp directory and called auto_rem.389743.93299.1
The file should always start with auto_rem and the numbers that follow will
correspond to the job ID in the database.
Defining Jobs :
There are the two methods you can use to create job definitions:
¦ Using the AutoSys Graphical User Interface (GUI).
¦ Using the AutoSys Job Information Language (JIL) through a command-line interface.
Autosys Jobs:
The difference between "on hold" and "on ice" is that when an "on hold" job is taken off hold,
if its starting conditions are already satisfied, it will be scheduled to run, and it will run. On the
other hand, if an "on ice" job is taken "off ice," it will not start, even if its starting conditions
are already satisfied. This job will not run until its starting conditions reoccur.
The other major distinction is that jobs downstream from the job that is "on ice" will run as
though the job succeeded. Whereas, all dependent jobs do not run when a job is on "on
hold"—nothing downstream from this job will run.
Starting Parameters :
AutoSys determines whether to start or not to start a job based on the evaluation of the
starting conditions (or starting parameters) defined for the job. These conditions can be one
or more of the following:
¦ Date and time scheduling parameters are met (it is or has passed the specified date and
time).
¦ Starting Conditions specified in the job definition evaluate to true.
¦ For jobs in a box, the box must be in the RUNNING state.
¦ The current status of the job is not ON_HOLD or ON_ICE.
Every time an event changes any of the above conditions, AutoSys finds all the jobs that may
be affected by this change, and determines whether or not to start them.
sample jil code / Writing jil code:
Runs the Job Information Language (JIL) processor to add, update, and delete AutoSys jobs,
machines, monitors, and reports. Also used to insert one-time job override definitions.
-------------------------------------------------------------------------------------------------
/* ----------------- SAP_UAT_MU03_C ----------------- */
insert_job: SAP_UAT_MU03_C job_type: c
command: /local/SAP/processCheckUAT.sh
machine: MU03-UAT
owner: admin@MU03-UAT
permission: gx,wx,mx,me
days_of_week: all
alarm_if_fail: 1
max_exit_success: 1
-------------------------------------------------------------------------------------------------
The above example is a simple jil code of autosys which we have to write if we want any
particular job to be thru autosys.
bash-3.00$ jil<>
This section lists which AutoSys commands to use for specific tasks. All commands are for
both Windows and UNIX, unless otherwise specified.
This will display a list of jobs with complete details with box/jobname, last/latest run date &
time, status, exit code, etc.
Viewing JIL code for any Autosys job
To obtain the underlying JIL (Job Interaction Language) source code for any Autosys job, run
command:
autorep -J (job name here) -r (No of runs back) example : autorep -J (job name here) -r 1
would generate a report for the job run one runs back
?
1sendevent –E FORCE_STARTJOB -J <job_name>
2
3sendevent -E STARTJOB -J <job_name>
?
1sendevent -E OFF_ICE -J <job_name>
2
3sendevent -E ON_ICE -J <job_name>
4
5sendevent -E KILLJOB –J "Job Name Here"
sendevent -E KILLJOB -J
job_name
To terminate a Job:
The following statements apply both to jobs ON HOLD and those ON ICE:
Even though the command may be successful, the job status may not immediately change to
ON_HOLD (or ON_ICE). The timing will depend upon how long it takes the event processor to
process this event.
You cannot put a job ON_HOLD if it is in STARTING or RUNNING state. The sendevent
command will run without error, but the EP will not change the job status and you will not
receive a notification.
To successfully put a job ON_HOLD, you should precede and follow the job with autorep
commands, as in the following example:
autorep -J job_name
sendevent -E JOB_ON_HOLD -J job_name
autorep -J job_name -d
sendevent -E JOB_OFF_ICE -J
job_name
What is the difference between ON_HOLD and ON_ICE?
Both ON_HOLD and ON_ICE will prevent a job from running when it is scheduled. However,
there are differences. Your choice of whether to put the job ON_HOLD or ON_ICE depends on
when you expect the job to run after you take it OFF-HOLD / OFF-ICE and whether the job is in
a stream or not.
When you put a job ON_HOLD
When the job is taken off hold, it will run immediately (once) if any runs were missed during the
ON_HOLD period (assuming all other conditions are met). Otherwise, it will start at its normal
time.
Note, however, that if the job has dependent jobs, these dependent jobs will not start. Also, if the
job is in a box, the box will not finish. Therefore, if a job is in a stream and you put it
ON_HOLD, this will effectively stop the stream.
When you put a job ON_ICE
When the job is taken off ice, it will start at the next scheduled run, not before. Moreover, if the
job has dependent jobs, the dependent jobs will see that job's status as SUCCESS. Thus in a
stream, the downstream jobs will run normally (but see the note below for an Autosys gotcha).
Putting a job ON_ICE effectively removes that job from the stream execution without having to
delete the job definition.
Note that when a job in a stream is put ON_ICE dependent jobs are started immediately after the
job is put ON_ICE (other conditions allowing). Therefore, putting a job ON_ICE can trigger the
portion of the stream that follows this job.
What happens to my job if Autosys is down?
If either the Event Processor is down or the Event Server (database) is down for any reason (such
as maintenance or a crash), then Autosys jobs will not be started. The jobs that were started
before Autosys went down will continue to run (as long as the client machines are up). However
the job status in the database will not be updated until both the Event Processor and the Event
Server are back up . When Autosys is back up, all the jobs that were supposed to start during the
downtime period are started. If a job has missed more than one run, it will only be started once.
What is a Chase alert?
Chase is a system utility that monitors Autosys to verify that jobs don't get stuck in STARTING
or RUNNING state, and generates alerts if it finds stuck jobs. In general, a chase alarm means
that there is something wrong either with the job or with the client machine. Because these are
job-related not infrastructure problems, the user is responsible for responding to the alerts.
For more information about the chase utility, see
[[http://miscwebapp.ms.com/sam/autosysweb/docs/support/troubleshooting.html#chase]
[Troubleshooting]] or Command Lookup.
Job & Machine Definitions
How do I view job definitions?
autorep -J job_name -q
How do I update a job using Jil?
To modify a job: use Jil as in the following example:
update_job: job_name start_times:
"0:00"
If you are not logged into the @machine specified in the job definition, then you can use the
command chgownermach to change the @machine to another valid machine. You must use the
physical host name for the machine (e.g., obtained via uname -n ), not a machine alias.
chgownermach -J job_name -m
machine
To see a list of all of the real and virtual machines that are defined in an Autosys instance,
use:
autorep -M ALL
What if my job edit machine is down?
You log in to another machine and use the chgownermach command to change @machine on the
jobs you need to edit.
How fast can I change @machine in case of an emergency?
We estimate the rate at which jobs can be changed in a steady stream to be better than ten per
minute. This rate should not be substantially affected by multiple users running the
chgownermach command simultaneously.
How do I append the current date to a standard output or error file?
To append the current date to standard output or standard error:
std_out_file: >/tmp/$AUTO_JOB_NAME.$AUTOSERV.`date "+%y%m%d"`.out
std_err_file: >/tmp/$AUTO_JOB_NAME.$AUTOSERV.`date "+%y%m%d"`.err
File watcher jobs do not create output files, so any entry in std_out_file: or std_err_file: will be
ignored
How do I append the current date to a file watcher?
To append the current date to a file you are watching for:
watch_file: /my_full_path/filename.$(date +%y%m
%d).txt
Job Features
How do I redirect stderr and stdout?
By default, stderr and stdout files go to /dev/null. We recommend that you save these as log files
in order to have them for troubleshooting a job failure. To do this, you need to specify the
redirection using the job attributes for std_err_file, and std_out_file .
std_out_file: /var/tmp/$AUTO_JOB_NAME.out
std_err_file: /var/tmp/$AUTO_JOB_NAME.err
Interfaces
What GUIs are available for Autosys?
The Autosys GUIs are accessible via the following command, which brings up a small menu by
which to select a GUI: autosc The most useful
choices are:
Ops Console
a console to monitor jobs.
Job Definition
a single-job definition GUI.
Jobscape
a job stream monitor that shows both box hierarchy and job or box status.
The Calendars GUI is intentionally disabled. However, for job definitions, we recommend that
you just the JIL. It allows you to manipulate many jobs at a time and to save the JIL definitions
under a separate source control system.
Is there a PERL API for Autosys?
No. There is no PERL API (or any other APIs) available for Autosys.
MS Environment
How do I use "module load" for my command in an Autosys job?
One tempting place to load a module is in the job profile. However, for reading the profile,
Autosys uses sh (the Bourne shell), which does not understand functions and therefore the
module command. (In fact, module is a function.) by default, and cannot be changed. So, don't
use the module load in the profile.
However, if your command is a ksh script, you can load the module as part of it.
If your command is a perl script, use the perl module: Env::Modulecmd.
If the command is neither a ksh nor perl script, you can create a ksh wrapper for it.
real 0m20.00s
user 0m0.22s
sys 0m0.97s
If you want to activate the failed job than do first on-hold the job ,than off-hold the job
The null device is typically used for disposing of unwanted output streams of a process,
or as a convenient empty file for input streams. This is usually done by redirection.
/dev/null is a special file, not a directory, so one cannot move files into it with the Unix
mv command. The rm command is the proper way to delete files in Unix.
You need to understand the theory first and then its upto you how and where you
want to apply that theory. I'll try to explain above to you.
The greater-than (>) in commands like these redirect the program’s output
somewhere. In this case, something is being redirected into /dev/null, and
something is being redirected into &1.
There are three standard sources of input and output for a program. Standard input
usually comes from the keyboard if it’s an interactive program, or from another
program if it’s processing the other program’s output. The program usually prints to
standard output, and sometimes prints to standard error. These three file descriptors
(you can think of them as “data pipes”) are often called STDIN, STDOUT, and
STDERR.
Sometimes they’re not named, they’re numbered! The built-in numberings for them
are 0, 1, and 2, in that order. By default, if you don’t name or number one explicitly,
you’re talking about STDOUT.
That means file descriptor 0 or fd0 denotes STDIN or standard input and file
descriptor 1 or fd1 denotes STDOUT or standard output and file descriptor 2 or fd2
denotes STDERR or standard error.
You can see the command above is redirecting standard output into /dev/null, which
is a place you can dump anything you don’t want (often called the bit-bucket), then
redirecting standard error into standard output (you have to put an & in front of the
destination when you do this).
The short explanation, therefore, is “all output from this command should be shoved
into a black hole.” That’s one good way to make a program be really quiet!
servinfo -v mrda202
pts mem nayakr
wk '{printf ("%s%s", $2,"\t")} END {print "\n"}'
njilgrep -h <hostname>
/u/rave_system 8$ du -k --max-depth=1 .
du -h --max-depth=1 .
free –m (free space in server)