Silo - Tips Job Scheduling On Windows
Silo - Tips Job Scheduling On Windows
Silo - Tips Job Scheduling On Windows
White Paper
*
Independent research based on data published by the Transaction Processing Performance Council (http://www.tpc.org) assumes
a 16-processor server and SAN configuration.
The business demands for information processing continually increase in volume and
complexity. These needs have placed great burdens on operations staff. Organizations
have diverse application workloads to process, which may include packaged
applications, several different platforms, and significant integration of operations across
business functions.
The most recent generation of job scheduling has been driven by the broader
integration needs that have arisen from Internet-based business activities. To compete
in their own markets, organizations have been forced to rethink the way they do
business. Customers want self-service applications, and expect such applications to
provide real-time, integrated access to personalized information. For these scenarios,
any product and departmental silos must be hidden behind Web applications that
present the company, products, and services to the outside world.
To address these needs, IT organizations are building complex, real-time, automated
business processes using a patchwork of existing and packaged applications. A Web-
based product order may trigger several dependent applications to complete the
ordering process. Examples include order tracking, billing applications, product
assembly and packaging instructions, shipping notifications, and inventory
management. A workflow of jobs that use existing applications is often built with batch
integration technology that is provided by job scheduling software.
Another innovation in job scheduling products is event-driven schedules. This
represents a clear move by batch tool vendors into the online camp. Jobs may be
IBM Tivoli Workload Scheduler for z/OS, based on the original Tivoli Operations
Planning and Control (OPC) product, provides mainframe-based workload management
with capabilities to manage workload on Linux, OS/400, Unix, and Windows via
scheduling agents. When a job fails, IBM Tivoli Workload Scheduler for z/OS handles
automatic dataset cleanup to restart the job. The central scheduler provides automatic
recovery from system failure by using a hot standby architecture. Users can configure
alternate workstations to automatically reroute workload in case of primary workstation
unavailability. The Workload Scheduler balances workload by integration with the IBM
Workload Manager (WLM) component of z/OS.
IBM technology uses an open interface and can be used with other scheduling engines.
Fault-tolerant agents carry a local schedule plan to continue workload processing in the
event of network or scheduling server failure. The related product IBM Tivoli Workload
Scheduler for Applications integrates workload scheduling with Oracle, PeopleSoft, and
SAP business applications.
AppWorx
AppWorx Corporation, formed in 1990 provides job scheduling software and consulting
services and has 500 customers worldwide. AppWorx software manages workloads on
Linux, OpenVMS, OS/400, Unix, and Windows platforms. AppWorx also supports
Oracle, PeopleSoft, Retek, SAP NetWeaver, and SunGard Banner. Its AppMaster
product provides a central point of control, and a graphical analysis package provides
operations forecasts and reports.
ASCI ActiveBatch
Advanced Systems Concepts, Inc., formed in 1981 specializes in OpenVMS, Unix, and
Windows software. Its ActiveBatch job scheduler runs on Windows to control workloads
on Linux, OpenVMS, Unix, and Windows and integrates with the Microsoft Windows
security model and Kerberos. ActiveBatch includes a rich scripting language to control
workload automation for calendar-based and event-based workflows. ActiveBatch
supports Windows and the Symantec Veritas Cluster Server for failover, and integrates
Microsoft SQL Server™ jobs.
ASG-OpsCentral
Although BMC Control-M incorporates a central scheduling server and remote agents to
control satellite workloads, the server engines that run on Unix, Windows, or z/OS are
functionally the same. By using the same server on both the mainframe and distributed
IBM Tivoli Workload Scheduler (TWS), based on the original Tivoli Maestro product,
provides distributed-based workload management for Unix, Linux, and Windows
platforms. It has the capability to manage z/OS, OS/400, and other platform workloads
via agents. A TWS network contains at least one domain, the master domain, which is
the management hub. Additional domains can be used to divide a network into locally
managed groups. Automatic recovery is provided by domain managers using a hot
standby architecture. Fault-tolerant scheduling agents carry a local schedule plan to
continue workload processing in the event of network or scheduling server failure.
The TWS for Applications component integrates workload scheduling with Oracle,
PeopleSoft, and SAP business applications. IBM Tivoli Dynamic Workload Broker
(TDWB) provides virtualization and load balancing in distributed environments and
ORSYP S.A. based in Paris, France, was established in 1986 to provide IT automation
software and related consulting services. Dollar Universe is a cross-platform job
scheduler with 950 customers worldwide that supports Linux, MPE, OpenVMS, OS/400,
Unix, and Windows. Options are provided for integration with SAP, Oracle, PeopleSoft,
JD Edwards EnterpriseOne and Microsoft Business Solutions—Axapta® (now Microsoft
Dynamics™ AX). Central operations are managed via a GUI or Web client; integration
is provided for HP, BMC, IBM, and CA systems management tools.
Redwood Cronacle
Redwood Software, founded in 1993, offers Cronacle for job scheduling. Redwood lists
3100 customers, which includes clients of their document management tools. Cronacle
supports workloads on OpenVMS, OS/400, UNIX, Windows, and z/OS, and integrates
with SAP, PeopleSoft, Oracle, and others. The Redwood Explorer interface provides a
Windows-based central console and a Web-based client. Cronacle features failover
support and dynamic load balancing. Redwood also provides an integrated version of
Cronacle for SAP NetWeaver called SAP Central Job Scheduling.
SMA OpCon/xps
SMA (Software and Management Associates) was established in 1980 to provide job
scheduling software for Unisys platforms. The company created OpCon/xps in the mid-
1990’s to respond to the emerging client/server market on Windows and Unix platforms.
OpCon/xps now provides cross-platform automation solutions on a broad range of
mainframe, Linux, Unix, and Windows platforms.
OpCon/xps has about 130 customers, most of whom automate cross-platform
applications on many systems, including traditional IBM and Unisys mainframes and
distributed platforms. OpCon/xps runs on a single dedicated Windows server with a
failover capability and can control workload on mainframes and other servers through
agents. SMA has successfully transferred the majority of its original customer base to
the new product. Many have migrated; SMA provides a Unisys agent for those who
remain. A production server can be shut down for servicing without stopping jobs
scheduled for other servers. The z/OS agent includes a JES2 restart capability.
Tidal Software formed in 1979 provides products and services for cross-platform job
scheduling and application performance management. Tidal Enterprise Scheduler
automates workloads on and across Linux, MPE, OpenVMS, OS/400, Tandem, Unix,
Windows, and z/OS. Tidal also integrates with SAP, PeopleSoft, Oracle, Informatica, JD
Edwards, Tivoli Storage Manager, Veritas, Business Objects, and Lawson, and provides
centralized access from a single graphical console.
Tidal Enterprise Scheduler is used by over 400 organizations, typically to automate IT
processes across several diverse platforms. The scheduling server, which runs on Unix
or Windows, has a high-availability failover option. Alternatively, the scheduler can use
Unix or Windows clusters. Application restart must be handled by application coding
conventions. On z/OS, Tidal offer an agent that manages task initiators, or a gateway
option for use with JES2. Integration with Tidal’s performance management software
provides the basis for critical process monitoring.
UC4:global
UC4 Software was formed in 1985 to provide Seimens mainframe scheduling software.
In 1996, UC4 Software released their first cross-platform job scheduler, now called
UC4:global. UC4:global, is used by over 750 customers worldwide, supports Bull GCOS
8, Linux, MPE, OS/400, OpenVMS, Siemens BS2000, Tandem, Unix, Windows, z/OS
platforms. UC4:global supports SAP, Oracle, PeopleSoft, and Seibel applications.
UC4:global uses agents on remote platforms called executors. It has redundant
scheduling servers to handle failover conditions automatically. Rollback is provided for
z/OS JCL. File version management (similar to z/OS GDGs) on Windows simplifies
recovery in the event that prior copies of flat files need to be restored. It includes a rich
event-based architecture, calendar scheduling, and a central point of control and visual
designer for cross-platform operations. Schedules can be stored in Oracle, DB2, or SQL
Server.
Vinzant Software has been providing job scheduling software since 1987 and lists
almost 100 customers on their website, primarily from the U.S. Its Global ECS software
incorporates a Windows-based administration client and server software. Agents are
available to manage workloads on Linux, NetWare, Unix, and Windows platforms. A
software development kit provides integration APIs for C, C++, and Microsoft Visual
Basic®.
Technology
The feature descriptions for batch processing, workload management, and workload
automation were used as the basis for collecting and aggregating the relative strength
and maturity of each feature on the two platforms. The assessment is limited to
scenarios where mainframe-based schedulers are used primarily to schedule
mainframe workload, and Windows-based schedulers are used primarily to schedule
distributed workload.
From a technical standpoint, this analysis shows that there is little to choose between
mainframe-based and Windows-based job scheduling products. The following points
arise from observations of consistently strong features by host platform.
Strengths of mainframe-based job schedulers
Mainframe workload that is scheduled by mainframe-based job schedulers runs under
the control of JES2 or JES3, and is thus scripted using JCL. JES2 and JES3 are
therefore important components of the overall mainframe job scheduling environment.
• Service classes. Service classes were built into JES2 and JES3 from the outset, to provide a
mechanism for prioritizing batch processing based on resource requirements and job priority.
Although similar mechanisms have been built into distributed-based products, service classification
has been a lower priority.
• JCL restart and recovery. Because JCL and the JES engines support multi-step processes, they
incorporate a sophisticated checkpoint and restart capability that is not found in the equivalent
scripting languages that are used on distributed platforms for workload execution.
Market forces
It is clear that the mainframe and Windows platforms come to job scheduling from very
different starting points. Batch processing was central to early mainframe operations,
because these machines were designed to handle varied workloads on behalf of a large
number of users who had to wait for their output. Windows was introduced as the
operating system for the personal computer, where the emphasis has been on ease of
use and immediate access to personal information.
Cost
Users look for ways to optimize the use of shared IT resources to minimize operational
costs and to avoid costs associated with capacity upgrades. Job scheduling provides a
means balance workloads across existing servers and to shift non-essential workloads
away from peak on-line processing periods. Traditional batch programs have been
developed for the mainframe as a way to allow work to be accumulated for overnight
processing. This in turn has driven the creation of rich batch processing capabilities on
mainframe platforms. On distributed platforms, cost is still a major driver, but platform
costs are lower, thus reducing the pressure for 100 percent utilization at all times. The
dominant concern for many organizations today is the cost of integrating and
automating complex on-line systems and processes. Workload management and
automation address this concern by reducing the need for costly real-time integration
services and software and by simplifying operational procedures.
The Internet
The Internet is one of the key driving forces for software innovation The Internet is
“always on” so there is little need for overnight batch activity when Web, application,
and database servers must be responsive to users around the globe in all time zones.
Self-service Web applications create a need for rich real-time integration of back-office
applications to fully automate the service that is provided to customers.
Regulatory compliance
To be compliant with new regulations, particularly in the healthcare and financial
services sectors, IT organizations must ensure that key processes are fully automated,
monitored, and logged for future audits. Job schedulers are fulfilling a need for
coordinated cross-platform integration between servers, packaged applications, and
core applications to ensure that business activities can be traced through the various IT
components that support them.
Growing complexity
Systems administration to optimize resource utilization becomes more complex with the
increasing use of server farms, blades, clustering, virtualization, and storage area
networks, which in turn demands greater sophistication in workload management and
automation features. Job scheduling can now be extended to optimizing and improving
To summarize, there is a great deal of overlap in the market forces on the mainframe
and distributed platforms, but these drivers also help to explain how job scheduling has
evolved differently on the these platforms.
The arrival and adoption of job schedulers on Windows has brought with it a rich and
mature capability for more traditional batch processing and workload management.
These job schedulers are modelled on or have been ported from traditional mainframe
tools and thus exhibit the characteristics that are required to move large batch
workloads to the Windows platform.
It is clear that the need for workload automation—real-time integration—is driving
innovation in the job schedulers that run on distributed and mainframe platforms. There
is less innovation in batch processing except where extremely high workload volumes
require the use of multi-system computing architectures to handle high and highly
variable capacity demands.
If you are not preserving the JCL, you will use the native scripting environment provided
by the platform to define the details of the processes to be run. Some job schedulers
provide a platform-independent scripting language.
Table 3 shows a summary view of the level of JCL support provided by either Fujitsu
Computer Systems or Micro Focus for each Windows job scheduler.
Both Fujitsu and Micro Focus provide a simple batch command-line interface that
provides basic support for a job scheduler to start JCL processes and either wait for a
completion code or cancel the job.
Advanced support provides additional capabilities such as variable substitution, remote
execution, asynchronous operations, and the ability to start and stop service classes or
the entire JCL subsystem. In some of the product combinations shown as “Tested
(basic support)”, support may include a small subset of these advanced capabilities.
This is not shown, to keep the table simple.
JCL statement support
Fujitsu and Micro Focus both claim full syntax support for JCL statements, but execution
support is not provided for IBM’s JES2 or JES3 control statements. In general, this
stance provides an acceptable solution, because the JES2 and JES3 control statements
are either irrelevant in the new environment, or superseded by facilities that are
provided by the job scheduler.
†
See http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx
‡
You can find more details at http://www.microsoft.com/technet/windowsvista/mgmntops/taskschd.mspx
File management Define and schedule file management jobs, which includes sorting,
renaming, labelling, copying, comparing, platform conversion (code page,
byte order, and format, as for VSAM to C-ISAM), and batch editing.
Integration with security Security integration includes operator authentication, access controls, job
systems authentication, and access controls for resources that are required by jobs,
including integration with SAF, RACF, ACF2, and third-party Web security
products.
Operator alerts Operators and users are notified of problems or other conditions that
require intervention in such a way that jobs may be manually restarted or
cancelled.
Service classes Jobs are scheduled based on service classifications (to which hardware
and software resources are assigned) to control resource contention by
restricting the number of concurrent jobs in each service class.
Spooling devices Spooling devices minimize scheduling delays by moving the wait time for
slow devices that are used for job input and output (such as readers and
printers) to a separate processing queue.
Scheduler throughput The scheduler manages very high-volume workloads (ideally by scaling
linearly with the raw processing capability of the execution nodes) by
minimizing network and other scheduling latency.
Workload failover The scheduler manages workload server failures by redirecting workload
and resubmitting failed jobs to standby or pooled machine resources.
Cyclical scheduling Job definition includes calendar schedules to run jobs on a regular,
predetermined cycle such as daily, weekly, monthly, yearly, or (for
example) the second Tuesday in the month.
Deadline scheduling Job definition includes calendar schedules to specify a date and time for
job completion up to one year or more in the future.
Dynamic resource The scheduler determines the job mix on each processing node
balancing dynamically, to balance the use of critical hardware and software resources
including servers, CPUs, memory, IO subsystems, applications and
databases.
External task monitor The scheduler monitors external tasks (jobs that are run under other
subsystems, operator-started tasks, and packaged applications) and uses
completion events to trigger dependent jobs or schedules.
Multiple calendars and Job schedules are based on one of several customizable calendars
time zones (different public holidays, different time zones, local business deadlines).
On-demand scheduling Users and operators submit individual jobs, which will be scheduled
alongside those jobs already scheduled to run, and which can act as
predecessors to trigger dependent jobs.
Scheduling packaged Jobs that run under the control of packages such as SAP, Oracle, and
applications PeopleSoft are defined, scheduled, and monitored from the central
scheduling console and integrated into cross-application workflows.
Single point of control An operator monitors and manages workloads that run on multiple nodes of
a heterogeneous processing network from a central console.
Workload analysis Numerous analysis tools are available, such as queue displays,
performance displays, schedule displays, schedule trace-back,
configurable reports, historical reports (for auditing), and summaries.
Dynamic schedules A feature that can automatically alter schedules, resources, and workload
mix based on dynamic conditions (such as changing resource configuration
to improve customer response times without intervention).
Programmable System programmers can use a documented API or write exit routines to
scheduler API provide more advanced features to create and modify schedule and
resource definitions, and to control schedule execution engines.
Virtualization Define multiple logical scheduling systems or partitions that share a uniform
hardware platform, to enable separation of monitoring, access controls,
auditing, and billing for multiple clients or departments.
Workload forecasting Forecast future workloads based on schedules that have already been
defined, including reports on resource utilization, load balancing, time
windows, and business function SLAs (such as payroll) under both normal
and failure conditions.
Workload planning These tools assess the impact of new workloads or workload changes on
capacity and performance. (Such tools are often based on simulation.)