PhUSE EU Connect 2018

Paper SM02

The “Unveiled” secret of “Windows Batch Scripting”

Nicolas Rouillé, Cytel, Geneva, Switzerland

Laura Phelan, Cytel, Paris, France

ABSTRACT
A batch file is a file that contains a sequence, or batch, of commands. Batch files are useful for storing sets of
commands that are always executed together because you can simply enter the name of the batch file, or double
click on it, instead of entering each command individually. For example entering a copy or rename command.
For example, the Windows Batch Script language can be used with the interpreter cmd.exe, which is the default
interpreter on Windows® platforms: NT, XP, Vista, 7 and later.
Taking the example of a repetitive task, such as the run of several SAS programs at once, we would like to introduce
the basic of Windows Batch Scripting language. The example will show the following:
- Select a set of programs by dragging the selected programs into the icon representing the batch program
- Get automatically the name of the files “passed” to the script and generate a loop
- Within the loop:
 Batch submit each selected SAS program
 Prompt the user with the execution status
 Promptly inform the user if the program generated some errors or warning by analyzing the SAS
log
- Prompt the user when the batch is terminated with some metrics e.g. elapsed time
Other examples include creating automatic backups of your study task folders, running P21® in batch, creating a
SAS® program standard layout (e.g. with standard header with automatically filled).

INTRODUCTION
There once was a programmer, Simone Margaritelli, a researcher at Zimperium, who hacked his coffee machine to
brew coffee using the command line:

Another “lazy” programmer wrote some scripts to get his jobs done: Nihad Abbasov, contributing as “Narkoz” on
GitHub ; he created some scripts with funny names, including one which sent automatic emails with message like
“not feeling well/working from home” if no “interactive sessions on the company server at 8:45am” with his login,
even selecting random reasons from a pre-defined list.

While we may not have the audacity of the latter programmer, we hope to show how the use of batch commands can
make everyday tasks one keystroke away.

WHAT IS WINDOWS BATCH SCRIPTING?

From Wikipedia, the free encyclopedia:
“A batch file is a kind of script file in DOS, OS/2 and Microsoft Windows. It consists of a series of commands to be
executed by the command-line interpreter, stored in a plain text file. A batch file may contain any command the
interpreter accepts interactively and use constructs that enable conditional branching and looping within the batch
file, such as IF, FOR, and GOTO labels. The term "batch" is from batch processing, meaning "non-interactive
execution", though a batch file may not process a batch of multiple data.”

In other words, a batch file is a text file that “batches” (bundles or packages) into a single file, a set of commands
that would otherwise have to be presented to the system interactively from a keyboard one at a time, so that multiple
actions can be automated on your computer via a simple mouse-click.

1
 PhUSE EU Connect 2018

BUILDING A SCRIPT WITH PRACTICAL EXAMPLES

BATCH RUNNING A SET OF PROGRAMS

We are all familiar with the “Batch Submit with SAS 9.x…” option, which can be handy in its own right but all the
action is a little bit behind the scenes with regards to whether the code ran ok.

Users have to manually (groan) open the log, and manually (double groan) search for errors, warnings or other
suspicious notes like uninitialized or not referenced. Plus, and this may be more subjective, the log is stored in the
same folder as the program which may not follow your company’s standard practice – requiring yet another “manual”
action to move the log?
Running in batch SAS programs could also avoid dependency from one’s user profile, which would have been
loaded during an interactive SAS session – typically templates inappropriately stored in a sasuser library ; to execute
in batch the SAS programs becomes then particularly relevant for a production run.

This is one scenario where Batch scripting can come to the rescue by running one SAS program or a selection of
SAS programs, storing the logs in the correct folder, searching the logs for keywords and giving the user a summary
of what it found in the logs.

But before we get to that, you may want to back up what is in your folder – in case things go unexpectedly wrong!
Why not use batch to manage this as well? Let’s start by a quick introduction to some of the basic script commands.

CRASH COURSE ON SOME BATCH SCRIPTING COMMANDS

Script Action
Echo display as text anything written after echo
@echo off removes all the paths where the actions are being performed – makes
execution/messages easier to read
Pause batch “pauses” after execution so user can see what happens (unlike the
SAS batch run, for example..)
echo. may be used to display a blank line, between user messages for more
elegant presentation, for example
:top Example of a loop
(script actions…)
goto top
set creates a variable, e.g. set name=Laura, and to resolve what the name is
surround it with % %, e.g. echo %name% would display Laura
/a to get mathematical interpretation
== or EQU is equals, e.g. if %you%==1 echo %v1% - if the variable you equals 1 then
the variable v1 will display
NEQ is not equals
rem use to ignore everything on the line following the rem, including quotes,
redirection symbols, and other commands
%~d1%~p1 active folder

MAKING A BACKUP FOLDER SCRIPT

So, let’s get back to backing up our folder (see Script example 1). Based on the above, we can define a subfolder
including the date in a specific format as the name, create that subfolder via mkdir, and copy our files to there via
xcopy *.*. Now, we are fussy, and we like our subfolders to be chronologically sorted so we prefer the
YYYY_MM_DD naming convention compared to the %date which resolves as “day MM/DD/YYYY”. Not unlike the
SAS substr function with concatenation, we can select and reorder the date parts using the minus sign to read from
the end of the date variable to get for example the year: %date:~-4,4%=2018.

2
 PhUSE EU Connect 2018

Script example 1: Backup folder contents

ECHO Back-Up folder name = _BackUp_%date:~-4,4%_%date:~-10,2%_%date:~7,2%
mkdir _BackUp_%date:~-4,4%_%date:~-10,2%_%date:~7,2%
ECHO.
ECHO Copying files of the active folder into the back-up folder
xcopy *.* _BackUp_%date:~-4,4%_%date:~-10,2%_%date:~7,2% /y

And in so doing, by running our script, we have backed up our files and are now covered in case of accidental file
overwrite or removal.

CLEANING SUBFOLDERS SCRIPT

Like we said, we are fussy. We also like to keep things clean, and before we test running any programs with this new
script, we’d like remove all leftover outputs and logs, etc., without manually going from folder to folder, selecting files
and deleting them - and risk removing the wrong files in the process. So we made a batch script to do it for us.

@ECHO off
ECHO.
ECHO.To clean content of the sub-folders Lst/Log/Output/Data
ECHO.
ECHO =============== Execution Start ===============
ECHO.
SET /P AREYOUSURE=Are you sure (y/[n])?
IF /I "%AREYOUSURE%" NEQ "y" GOTO END
ECHO - Removing lst files stored within the 'Lst' folder
del "%~dp0\..\Lst\*.lst" /f /q
ECHO - Removing log files stored within the 'Log' folder
del "%~dp0\..\Log\*.log" /f /q
ECHO - Removing rtf files stored within the 'Output' folder
del "%~dp0\..\Output\*.rtf" /f /q
ECHO - Removing sas7bdat files stored within the 'data' folder
del "%~dp0\..\Data\*.sas7bdat" /f /q
ECHO.
:END
ECHO =============== Execution completed ===========
ECHO.
PAUSE

First we SET or create the variable AREYOUSURE, because we’ve had feedback in the past that people were not so
sure they wanted to delete all their work, understandably. The /P is a sort of switch which allows the developer to set

3
 PhUSE EU Connect 2018

the value of a variable to a line of input entered by the user, i.e. the user needs to answer the question “Are you sure
(y/n)?” with a y or n before any files are removed. The /I may be used to ignore case sensitivity here. If they answer
“n”, the script will go to the end (GOTO END) and not perform any action. Otherwise, it moves to the folders of interest
and deletes the file types we have specified in those folders, e.g. logs in the Log folder, lsts in the Lsts and so on.
The “%~dp0” variable when referenced within a Windows batch file will expand to the drive letter and path of that
batch file, so when the script is run from a specific folder, in our case we run from a Pgm sub-folder, del
"%~dp0\..\Lst\*.lst" /f /q moves up and then down a level to the Lst sub-folder and removes all lst files it finds
there, ignoring any read-only setting (/f) and without prompting (/q for “quiet mode”).

DRAG AND DROP PROGRAM(S) RUN BATCH SCRIPT

Stepping the pace up a bit, to run a program or set of programs in batch, saving the log and lst in their appropriate
folders, we could use the following:
@ECHO off
ECHO ============================== batch BEGIN ===========================================
ECHO.
ECHO Batch execution started at %Time%
ECHO Detection of the active folder = %~d1%~p1

if exist "%~d1%~p1\log" (set LogFolder=%~d1%~p1\log) else if exist "%~d1%~p1..\log" (set

LogFolder=%~d1%~p1..\log) else (set LogFolder=%~d1%~p1)
if exist "%~d1%~p1\lst" (set LstFolder=%~d1%~p1\lst) else if exist "%~d1%~p1..\lst" (set
LstFolder=%~d1%~p1..\lst) else (set LstFolder=%~d1%~p1)

set /A NbProgsRan=0

FOR %%A IN (%*) DO (

set /A NbProgsRan+=1

ECHO '%%~nA%%~xA' Execution is ongoing

"D:\SAS94\SASHome\SASFoundation\9.4\sas.exe" -sysin "%~d1%~p1%%~nA%%~xA" -log
"%LogFolder%\%%~nA.log" -print "%LstFolder%\%%~nA.lst" -icon -nosplash -rsasuser
ECHO '%%~nA%%~xA' Execution completed at %Time%
ECHO.
ECHO.

ECHO.
)
ECHO.
ECHO ================================= batch END ================================================
PAUSE

First the log and lst folders need to be set: if they are subfolders in the folder from where the program is run,
%~d1%~p1\log or in another subfolder at the same level as the program %~d1%~p1..\log , or neither, in which case
they are stored with the program.

Every program dropped into the batch script, FOR %%A IN (%*) DO ( is then run
"D:\SAS94\SASHome\SASFoundation\9.4\sas.exe" -sysin "%~d1%~p1%%~nA%%~xA".
with their corresponding log/lst saved in the LogFolder/Lstfolder
-log "%LogFolder%\%%~nA.log" -print "%LstFolder%\%%~nA.lst"

4
 PhUSE EU Connect 2018

We’d also however like the script to now tell us what happened in the logs. We can do this by adding a log scan for
messages of interest,
set /A TotalNbOfLogERROR=0
set /A TotalNbOfLogWARNING=0
….
ECHO --- Log scanning - messages are displayed by decreasing order of criticality:
findstr /A:4F /n "ERROR fatal" "%LogFolder%\%%~nA.log"
findstr /A:2F /n "WARNING uninitialized unequal" "%LogFolder%\%%~nA.log"
findstr /A:1F /n /c:"MERGE statement has more than" "%LogFolder%\%%~nA.log"
ECHO.
)
and summarize the number found using,
rem --- to compute the cumulative number of ERROR or WARNING message(s) in the log files
if exist "%LogFolder%\%%~nA.log" (
for /f "tokens=*" %%j in ('findstr "ERROR" "%LogFolder%\%%~nA.log"') do @SET /A
TotalNbOfLogERROR += 1
for /f "tokens=*" %%j in ('findstr "WARNING" "%LogFolder%\%%~nA.log"') do @SET /A
TotalNbOfLogWARNING += 1
)

By adjusting color we can highlight for users any important messages found, we liked red for errors or warnings
found, and green if there were none, following the color codes:

For example:
if "!TotalNbOfLogERROR!" gtr "0" (
rem Call :Color 4F "FAILED:"
rem ECHO. Total number of ERROR messages in the log files: !TotalNbOfLogERROR!
powershell.exe "(Write-Host -NoNewLine -BackgroundColor """DarkRed""" -ForegroundColor
"""White""" Total number of ERROR messages in the log files: !TotalNbOfLogERROR!. )"
echo.
)

5
 PhUSE EU Connect 2018

The full script can be found in the appendix.

OTHER EXAMPLES
PROGRAM CREATION WITH STANDARD HEADER
Because we like to follow Good Programming Practice, “A standard header should be used for every program”, we
again use batch script to do the work (see the appendix for the full script).

Using again the switch

/p SASfile=Please enter the name of the program to be created (file name without .sas
extension):,

users are prompted to enter a program name. To avoid unintentional overwriting of an existing program, one could
add a check if the file already exists, or will receive a warning if the program already exists:

if exist %SASfile%.sas goto :FileAlreadyExists.

…
:FileAlreadyExists
ECHO - The program %SASfile%.sas already exists within folder xyz.
ECHO - Please delete this file in case you wish to re-create it.

Details to include in the program header and body can be sent to the sas program (defined in SASfile) via all
commands starting with @echo and ending with >> "%SASfile%.sas”. The directory in which the program is created
can be added using %CD% if desired.

Macro calls must be masked with a leading %, e.g.

@echo %%getdirf(flvl=4);>> "%SASfile%.sas"

Similarly, comments can be included in the body of the program using a leading % symbol, i.e.
@echo %%*--- at output level: save lst file if generated;>> "%SASfile%.sas"

6
 PhUSE EU Connect 2018

To ensure the FileAlreadyExists message is not displayed even when a new program is being created, a goto
may be deployed to skip it goto :DONE… (:FileAlreadyExists….) :Done.

VBS SCRIPT TO TRACK CHANGES OF RTF/DOC FILES ON-THE-FLY

As a programmer, sometimes you need to make a small adjustment in your ADaM, and rerun the associated
outputs. The person on the receiving end, be it a statistician or medical writer or other, may wish to know the exact
differences in the outputs they’ve just reviewed (for example difference between draft and final versions), or even
gone as far as including into a study report. Something like this may be appreciated more than the not so user-
friendly proc compare?

This example is managed by a VBS code which is having 2 arguments, i.e. the 2 files selected that the user wants to
compare. The script is opening in hidden mode the first file with MS Word, and is making use of the track changes
facility to open the second file and proceed to the comparison between the 2 files.
The differences are highlighted and saved in a new doc file as displayed above; the MS Word session opened
temporarily is then closed so the user would only see on the screen one window at the completion of the execution,
as following:

The comparison could be done on any file that MS Word is able to open, that is including txt files: so this script can
also be used advantageously to compare the 2 versions of one SAS program.

The full script can be found in the appendix.

CALLING SCRIPTS
One of the selling-point of scripts like these is that they can be called or launched in a variety of ways, at least one of
which should satisfy the most fastidious of users. These principles could apply to any script.

7
 PhUSE EU Connect 2018

 Scripts can be called from a command line and passing arguments:

 They can also be called from other software,

o for example from SAS, by calling the script and passing arguments, which can then be looped in a
macro over a selection of files (docs, .rtfs..), though you don’t have to open software like SAS to
use them.
o Other useful software can live in perfect symbiosis with scripts: they are text editors like UltraEdit®
or TextPad®. For example, the execution of one SAS program, the check of its log, the look to its
associated lst file, can be done by calling script(s) declared in the ‘Preferences’ menu as shown
hereinafter in this TextPad screenshot:

 Taking the run program script mentioned earlier, this is called simply by dragging-and-dropping the
arguments (=a selection of files!) over the .bat script, directly from the window explorer interface:

 Another option is via the windows explorer ‘send to’ menu. Take the compare script we covered, it is copied
to the users local profile e.g. [drive]:\Users\[username]\AppData\Roaming\Microsoft\Windows\SendTo. Back
at the server, the 2 outputs to compare are selected, right-clicked and the user scrolls to the script action, in
this case “Compare the 2 word files selected”.

8
 PhUSE EU Connect 2018

ADVANTAGES OF USING BATCH SCRIPTS

Among the reasons for getting familiar with and using scripts, is that they are:
 known for their portability and low friction, being
o versatile, with low dependence upon new release of platforms
o ‘universal’, recognized by any windows system
 easy to interact and integrate within other software
 very well documented - lots of resources are available
 commonly learned at school - so they are easy to investigate, and suitable for a broader audience
 particularly interesting to manipulate files: copy, find , rename
 ideal to automate routine tasks and save time

IS MS-DOS NOW OBSOLETE?

MS-DOS enthusiasts are fairly worrying at every new release of Microsoft platforms that the cmd.exe interpreter
would not come by default - as it is for now since … more than 3 decades! The direction taken by Microsoft is indeed
to encourage the use of PowerShell, of VB script, of JavaScript: they are object-oriented, better structured.
For now the coexistence of all is allowing for instance to use PowerShell commands to gather attributes of some files
(owner, dates of creation, etc.) within a .bat file containing also DOS commands; you can refer to the appendix listing
our ‘drag and drop program(s) run batch script’ in order to see an illustration of this use.

OTHER WINDOWS SCRIPTING LANGUAGES

Of course there are other scripting languages out there that need to be carefully considered depending on your
environment and strategy; Wikipedia is quoting the following:

 Windows Script Host (.vbs, .js and .wsf) - released by Microsoft in 1998, and consisting of cscript.exe and
wscript.exe, runs scripts written in VBScript or JScript. It can run them in windowed mode (with the

9
 PhUSE EU Connect 2018

wscript.exe host) or in console-based mode (with the cscript.exe host). They have been a part of Windows
since Windows 98.

 PowerShell (.ps1) — released in 2006 by Microsoft and can operate with Windows XP (SP2/SP3) and later
versions. PowerShell can operate both interactively (from a command-line interface) and also via saved
scripts, and has a strong resemblance to Unix® shells.
 Unix-style shell scripting languages can be used if a Unix compatibility tool, such as Cygwin, is installed.
 Cross-platform scripting tools including Perl®, Python®, Ruby®, Rexx® Node.js and PHP® are available for
Windows.
 Since 2016, active release and support of Windows Subsystem for Linux (WSL) upon which several Linux
distros now run; WSL aims for native Linux compatibility.

CONCLUSION
Following the thought process involved as described in this paper, you can use scripts as stepping stones to build a
bigger process. Whatever could be the scripting language of your choice, start with considering a script as a unit of
task:
o Split the process, identify routine tasks
 e.g. Back-up / Clean-up / Program Run / Log check / Summary
o Select most appropriate programming language, ideally, trying to do abstraction:
 from other languages
 from specialized software (code resources will be more limited)
o Build each process like a wall: modules can be added brick by brick to build a complete ‘solution’,
o Take care of the user experience
 in requesting minimal action from his side (e.g. by counting the number of mouse clicks
requested to perform one operation) ,
 and in prompting his input appropriately: for instance confirmation before a critical action
is performed.
o Interface your program with other software (text editors, call from SAS with the x command, etc.), so
their integration is optimal
o Be minimal in the writing of a program. for maintenance and ease of updates

 … And then, the scripts will follow you in time!

An organized library of scripts can result in an entire solution, e.g. automating all the sequence of a production run:

10
 PhUSE EU Connect 2018

REFERENCES
Narkov scripts: https://fossbytes.com/this-lazy-programmer-wrote-some-scripts-to-secretly-automate-a-lot-of-his-job/
Windows batch scripting: https://en.wikipedia.org/wiki/Batch_file
Guide to Windows Batch Scripting: http://steve-jansen.github.io/guides/windows-batch-scripting/
DosTips - The DOS Batch Guide: https://www.dostips.com/
Running Windows or MS-DOS Commands from within SAS:
http://support.sas.com/documentation/cdl/en/hostwin/63285/HTML/default/viewer.htm#exittemp.htm
SAS Batch processing under Windows:
https://communities.sas.com/t5/SAS-Communities-Library/Batch-processing-under-Windows/ta-p/475977

ACKNOWLEDGMENTS
We would like to also thank Remi Gerin for his “laziness” and “fussiness” leading to several script developments that
has made the lives of our programming teams easier.

CONTACT INFORMATION
Your comments and questions are valued and encouraged. Contact the author at:

Nicolas Rouillé
Cytel Inc.
Route de Pré-Bois 20
1215 Geneva – Switzerland
Email: [email protected]

Laura Phelan
Cytel France Sarl
63, Avenue des Champs Elysées
75008 Paris, France
Email: [email protected]

Web: www.cytel.com

Brand and product names are trademarks of their respective companies

11
 PhUSE EU Connect 2018

APPENDIX
FULL SCRIPT: DRAG AND DROP PROGRAM(S) RUN BATCH SCRIPT

12
 PhUSE EU Connect 2018

FULL SCRIPT: PROGRAM CREATION WITH STANDARD HEADER

FULL SCRIPT: VBS TO COMPARE ON-THE-FLY

13