Python

Success Stories

8 true tales
of flexibility,
speed, and
improved
productivity

Industrial Light & Magic

LivingLogic AG
Rackspace
Siena Technology
United Space Alliance
Wing IDE
University of St Andrews
Journyx

“ We achieve immediate functioning code so much faster in

Python than in any other language that it’s staggering.”
—Robin Friedrich United Space Alliance
Essential Tools for Success with Python

Order Now
See back page for order form. Or call 800-998-9938

python.oreilly.com
Introduction
by Alex Martelli & Guido van Rossum

Python is widely admired for its simplicity Engineers since the Roman Vitruvius have
and elegance. However, these undeniable recognized that a good design exhibits
qualities must not overshadow Python’s solidity (firmitas), delight (venustas), and
usefulness: with Python, you *can get usefulness (utilitas). Italian architect, Leon
the job done*! Battista Alberti showed during the
Python is highly scalable–suitable for large Renaissance that these characteristics are
projects as well as for small ones, excellent achieved by harmony (concinnitas) the
for beginners yet superb for experts. It is art of ordering disparate parts into an
stable and mature, with large and powerful organized whole. Python’s design makes
standard libraries, and a wealth of third- these theories come to life, no less than
party packages for such widely varied tasks Alberti’s Tempio Malatestiano.
as numerical computation, Internet and web Managers love Python because its simplicity
programming, database use, graphical user minimizes programmers’ learning efforts,
interface development, XML handling, but the same simplicity underpins Python’s
image processing, three-dimensional solidity, avoiding any bugs related to
modeling, and distributed processing with obscure, misunderstood and ad-hoc features,
CORBA or SOAP. Python plays well with in both the language’s implementations and
others, easily integrating with either C/C++ programs coded in Python. Designers are
or Java you can extend it, you can embed drawn to Python’s elegance, which allows
it in your existing applications, and you concise and readable expression of design
can use it to integrate multiple applications. ideas, but the same elegance and readability
This booklet illustrates Python’s usefulness ease maintenance, modification and reuse
by presenting a small sample of Python for programs coded in Python. The ease of
success stories, real-life examples where interfacing Python to existing C/C++ or Java
Python played a central role in the devel- libraries and applications, plus Python’s
opment and delivery of working, useful large standard library and the wealth of
software systems of substantial size. The existing third-party Python extensions,
systems are in a wide variety of application combine to make Python the ideal
areas, and they go up the scale, all the language for challenging integration tasks,
way to enterprise-wide integrated systems. no less than for “green-field” development
Thus, you’ll see that Python is not just projects. All together, these characteristics
a scripting language (not limited to coding make Python the language of choice for
small and simple scripts, even though it’s high-productivity software development—
quite suitable for such frequent, small one of the most rapid development
tasks), nor is its usefulness confined to any environments on the planet.
one application area. In the hands of experi- Once this booklet has whetted your
enced software developers, Python offers appetite, you can explore Python further,
high productivity for projects of all sizes, starting at www.python.org. For many
in all application areas. excellent books about Python, and for
Python’s usefulness comes exactly from its additional Python success stories,
simplicity and elegance, harmonized into a see python.oreilly.com.
seamless whole by its highly pragmatic design.
1
 Industrial Light & Magic Runs on Python
Tim Fortenberry

About the Author Background

Tim Fortenberry joined Industrial Industrial Light & Magic (ILM) was started in 1975 by
Light & Magic in 1999 as an filmmaker George Lucas in order to create the special
intern. Later that same year effects for the original Star Wars film. Since then, ILM
he began to work full time in has grown into a visual effects powerhouse that has
the Resources department. contributed not just to the entire Star Wars series, but
He worked as a scripts/tools also to films as diverse as Forrest Gump, Jurassic Park,
programmer. Shortly after, Who Framed Roger Rabbit, Raiders of the Lost Ark, and
Fortenberry joined the Research
Terminator 2. ILM has won numerous Academy
and Development department.
Awards® for Best Visual Effects, not to mention a string
of Clio awards for its work on television advertisements.
He is one of the founding
members of the Pipeline and While much of ILM’s early work was done with
TD Tools groups that helped miniature models and motion controlled cameras,
bridge the gap between artists ILM has long been on the bleeding edge of computer-
and technology. generated visual effects. Its computer graphics division
dates back to 1979, and its first CG production was
As an engineer, Fortenberry is
the 1982 Genesis sequence from Star Trek II:
responsible for developing and The Wrath of Khan.
maintaining the myriad of appli-
In the early days, ILM was involved with the creation
cations used for rendering and
of custom computer graphics hardware and software
pipeline control flow of images
for scanning, modeling, rendering, and compositing
at ILM. Prior to joining ILM,
(the process of joining rendered and scanned images
Fortenberry worked as a Linux®
together). Some of these systems made significant
systems administrator for VA
advances in areas such as morphing and simulating
Linux Systems. muscles and hair.
Originally from Southern Naturally, as time went by many of the early innova-
California, Fortenberry received tions at ILM made it into the commercial realm, but
his Bachelor of Arts degree from the company’s position on the cutting edge of visual
the University of California at effects technology continues to rely on an ever-changing
Berkeley in Anthropology with combination of custom in-house technologies and
an emphasis in Archaeology. commercial products.
www.ilm.com Today, ILM runs a batch processing environment
capable of modeling, rendering and compositing tens
of thousands of motion picture frames per day.
Thousands of machines running Linux, IRIX®,
Compaq® Tru64®, Mac OS® X, Solaris®, and Windows®
join together to provide a production pipeline with

2
approximately eight hundred users daily, turns out, Python enabled much more
many of whom write or modify code for ILM than just process control.
that controls every step of the production Unlike Unix shell scripting, Python can
process. In this context, hundreds of be embedded whole as a scripting language
commercial and in-house software compo- within a larger software system. In this case,
nents are combined to create and process Python code can invoke specific functions
each frame of computer-generated or of that system, even if those functions are
enhanced film. Making all this work, and written in C or C++. And C and C++ code
keeping it working, requires a certain degree can easily make calls back into Python
of technical wizardry, as well as a toolset code as well.
that is up to the task of integrating diverse
and frequently changing systems. Using this capability, ILM integrated Python
into custom applications written in C or
Enter Python C++, such as ILM’s in-house lighting tool,
Back in 1996, in the 101 Dalmation days, which is used to place light sources into
ILM was exclusively an SGI IRIX shop, and a 3D scene and to facilitate the writing,
the production pipeline was controlled by generation, and previewing of shaders
Unix® shell scripting. At that time, ILM was and materials used on CG elements. It is
producing 15-30 shots per show, typically the lighting tool that is ultimately respon-
only a small part of each feature length film sible for writing the 3D scene out to a
to which they were contributing. format that a renderer can interpret
Looking ahead towards more CG-intensive and render.
films, ILM staff began to search for ways At the same time, more and more compo-
to control an increasingly complex and nents, such as those responsible for ILM’s
compute-intensive production process. many custom file formats and data structures,
It was around this time that Python version were re-wrapped as Python extension modules.
1.4 came out, and Python was coming into As Python was used more widely, extending
its own as a powerful yet simple language and customizing in-house software became
that could be used to replace Unix shell a lot easier. By writing in Python, users
scripting. Python was evaluated, compared could recombine wrapped software compo-
to other technologies available at the time nents and extend or enhance standard CG
(such as Tcl and Perl), and chosen as an applications needed for each new image
easier to learn and use language with which production run. This let ILM staff do
to incrementally replace older scripts. exactly what production needed at any
At ILM, speed of development is key, and given time, whether that meant allowing for
Python was a faster way to code (and re- a specific look for an entire show, or just a
code) the programs that controlled the single CG character or element.
production pipeline. As it turned out, even some of ILM’s non-
technical users were able to learn enough
Python Streamlines Production
Python to develop simple plug-ins and to
But Python was not designed just as a create and modify production control
replacement for shell scripting and, as it scripts alongside the technical users.

3
Industrial Light & Magic Runs on Python, continued

Python Unifies the Toolset Once Python was in use, it made the
Encouraged by its successes in batch production control system portable. This
process control and in scripting applica- gave ILM additional freedom in making
tions and software components, ILM started hardware technology choices, including a
to use Python in other applications as well. large-scale introduction of commodity PC
hardware and Linux, a move that has saved
Python is now used for tracking and the company substantial amounts of money
auditing functionality within the production in recent years.
pipeline, where an Oracle database keeps
track of the hundreds of thousands of Source Code Access Important
images that are created and processed for After having used Python intensively for six
each film. DCOracle2, one of the Oracle years, ILM has yet to run into significant
integration libraries available for Python, bugs or portability issues with the language.
has performed well in this task and is now As a result, since Python 1.5 ILM has been
in use on Linux, IRIX, Tru64, and Solaris. able to rely on stock distributions in
Python is also used to develop the CG unmodified form.
artist’s interface to ILM’s asset management However, availability of source code for the
system. Designed to be modular, this tool language acts as an important insurance
has been enhanced by a large group of policy should problems arise in the future,
developers and non-developers alike to or should custom extensions or improve-
extend well beyond its original mandate. ments become necessary. Without this,
The application is now used not only to ILM could never have bought into
manage CG assets and elements, but also in Python so heavily for its mission-critical
daily shot review, as a network-based white- production process.
board, as an instant messenger, and even One case where access to source has already
allows an occasional game of chess. been beneficial was in ILM’s continued use
As Python was applied in more ways, it of Python 1.4, which is generally considered
slowly crowded out a plethora of competing obsolete. Because the production facility is
technologies for shell scripting and batch under continuous use, upgrading systems to
control, embedded scripting, component new Python versions would result in signif-
software development, database application icant disruption of the production process.
development, and so forth. Python’s versa- Instead, ILM installs new systems with
tility ultimately simplified the developers’ newer versions of Python but maintains
toolset and reduced the number of tech- older systems only so they can run the same
nologies that needed to be deployed to scripts as the newer systems. Supporting
ILM’s thousands of production computers. this mix has required access to the Python
This new, simpler toolset translated directly sources in order to back-port some changes
into an easier to manage and more reliable found in newer Python versions, and to
development and production process. reimplement portions of newer support
Hardware Costs Reduced libraries under older versions of Python.
Although chosen initially for its ease of use ILM is currently running a mix of Python
and integration capabilities, Python’s porta- 1.4, 1.5, and 2.1.
bility to many other operating systems
eventually became one of its key strengths.
4
Python Tested by Time
The visual effects industry is intensely
competitive. To stay on top of the heap,
ILM continuously reviews its production
methods and evaluates new technologies
as they become available.
Since its adoption in 1996, the use of
Python has also been reviewed numerous
times. Each time, ILM failed to find a more
compelling solution. Python’s unique mix
of simplicity and power continues to be the
best available choice for controlling ILM’s
complex and changing computing environment.

5
 XIST: An XML Transformation
Engine Written in Python
Dr. Walter Dörwald and Dr. Alois Kastner-Maresch

About the Authors Summary

Before receiving his Ph.D. in XIST is a XML transformation engine written

2000, Dr. Walter Dörwald completely in Python at LivingLogic AG, a software
researched forest growth development company specializing in web technology.
simulations and artificial life
XIST was designed to facilitate the task of creating and
maintaining large web sites.
at BITÖK (the Bayreuth Institute
of Forest Ecosystem Research), Background
and developed a large C++ Soon after we began creating web pages in 1994, it
framework for simulation and became clear that typing HTML files by hand is tedious
visualization. In 2000, he co- and cumbersome, and we began to search for tools to
founded LivingLogic AG, and simplify the repetitive task of HTML generation.
has been responsible for the Early on, we discovered and started to use an HTML
company's fundamental tools preprocessor named hsc. This tool supported gener-
and technologies ever since. ation of pages from templates by defining new markup
After getting his Ph.D. in tags and controlling how these tags would be trans-
Mathematics in 1990, Dr. Alois formed into HTML, somewhat like XML/XSL does now.
Kastner-Maresch lead the Unfortunately hsc had certain limitations: It didn't
Forest Ecosystem Simulation support local variables, and there were no control
research team at BITÖK. structures except conditionals. Even arithmetic was
In 2000, he co-founded not possible. Our first web sites developed with this
LivingLogic AG, where system consisted of a mix of hsc macros and Perl
he is CEO. scripts that generated hsc source files.
www.livinglogic.de/Python/xist/ In 1998, hsc's author halted further development,
and we became quite motivated to find an alternative.
At first we decided to continue development of hsc
ourselves, and planned to make it compatible with
XML, which was beginning to become popular at the
time. But extending hsc, which is written in C, proved
quite difficult. For example, adding Unicode support
required rewriting the entire I/O system. It became
clear that we needed to find another toolset for our
web development.
XIST is Born
Around this time we discovered Python and decided
that it might be a good way to completely rewrite hsc
from scratch. Python includes XML parsing capabilities
that we felt could be used as the basis for our work:

6
Instead of writing macros in hsc, we could is now used in all of our web projects at
write XML that could be processed through LivingLogic AG.
a simple mapping from XML element types Content Management with Python and JSP
to Python classes.
On top of XIST, LivingLogic has developed
In this approach, XIST generates an a content management system called
extended Document Object Model (DOM) XIST4C (4C means Content, Community,
on parsing each XML file. Classes defined Collaboration and Commerce). This system
for each element in the file are instantiated combines the advantages of XIST's
as the DOM is generated, and methods on abstracted page layouts with pre-compi-
the classes are used to perform the necessary lation of page templates to Java Server
XML transformations during page gener- Pages that are ultimately used to deliver
ation. This allows us to realize our web the content to the web.
templates with the full power of an
object-oriented scripting language. By using XIST tag libraries instead of JSP
tag libraries we are able to build optimized
During implementation, we found that all Java Server Pages that run considerably
of the key features of hsc could be supported faster than their JSP tag library counter-
quite easily in Python: parts, without any changes to the JSP files.
• Automatically calculate image sizes? This performance gain is a result of the fact
The Python Imaging Library does this that the XIST preprocessor takes care of
with ease. many compute-intensive operations, that
• Parse XML files? There are several XML might require dynamic type introspection,
parsers available in Python. string processing etc. and would be
executed by the Java tag library code
• Load and store XML to and from
during page load.
databases? The Python DB-API is
standardized and modules exist for Fast development combined with low
MySQL, Postgres, Interbase, Oracle, hardware requirements make XIST4C
ODBC, Sybase, and others. especially suitable for small and medium-
sized enterprises. This has allowed us to
• Fetch XML from the web? Python’s
achieve a unique competitive advantage,
urlparse and urllib standard libraries were
and to realize projects at a much lower cost.
made for that. Handle Unicode? Python
2.0 fully supports Unicode out of the box. Prototyping Java Systems with Python

Implementing the first prototype version In 2000, LivingLogic was engaged to

took a few weeks of spare time pro- develop a product for modeling business
gramming and turned out to be very process work flow and to automate business
successful. Python provided a much shorter processes. Inspired by our earlier successes
path from concept to implementation than with XIST, we decided to develop a Python
any of the other programming languages prototype. This decision was made even
we have used. And so XIST was born. though our contract required us to produce
a Java-based prototype for delivery to a large
XIST’s development continued in Python
group of developers that would turn
and today XIST is the basis of a successful
it into a marketable product.
company which employs 15 people. XIST
7
XIST: An XML Transformation Engine Written in Python, continued

The approach of using Python early in • Extensive and easy to use introspection
prototyping made it possible for us to study facilities make Python easy to learn inter-
concepts using working code almost from actively by discovering its capabilities,
the start of this project. Although we had including documentation, from the
to rewrite our Python prototype in Java, the command prompt.
overall time spent on prototyping was lower • Python is readily extensible in C or C++,
than we have seen in other projects that so it is easy to incorporate non-Python
used only Java. modules into an application.
Conclusion Python has played an important role in
Python is easy to learn and use, produces the success of LivingLogic AG, and will
maintainable code, and packs enough continue to be the basis for most of our
power to make it a suitable choice for many software development efforts.
application domains and project sizes.
Some of the features that we like best
about Python include:
• Python’s extensive standard library and a
considerable range of available third party
packages support development in many
application domains.
• An unsurprising syntax and the
widespread and consistent use of a
few basic concepts, like namespaces,
help to make Python code readable
and maintainable.

8
 Python is Rackspace’s CORE Technology
Nick Borko

About the Author Introduction

Nick Borko is the Director To be the industry leader in managed hosting, you
of Internal Application have to be fast and flexible. By using Python to
Development and the project implement our enterprise data systems, Rackspace
manager for Rackspace’s can quickly and effectively change its internal systems
enterprise database application, to keep up with shifts in the industry and in our own
CORE. Rackspace Managed business processes. We do this through a central
Hosting is the leader in deliv- customer information system called “CORE,” which
ering managed hosting services
is used both for Customer Relationship Management
to small and medium enter-
(CRM) and Enterprise Resource Planning (ERP).
Python and CORE are key factors that enable
prises. All customer platforms
Rackspace to provide our Fanatical Support(tm)
include state-of-the-art data
and faster customer service.
centers, customized servers,
burstable connectivity, 99.999% Background
uptime SLA, a dedicated Rackspace’s central customer database started as a
account manager, instant simple ERP system to provision and track managed
emergency response and servers. It began humbly, as a small collection of PHP
access to live expert pages that did the job nicely for the few hundred
technicians 24x7 for support servers that was the beginning of Rackspace’s
of all hardware and core customer base.
software. Founded in 1998 As Rackspace grew, that small PHP system became
and headquartered in San the center of business at Rackspace. Every time an
Antonio, TX, Rackspace opportunity to automate a process presented itself,
manages servers for it was rolled into that system.
customers in more than After a couple of years, the result was a big, un-
eighty countries. maintainable mess of thousands of PHP pages and
www.rackspace.com modules that had been written and maintained
primarily by one person. The limits of PHP (then
version 3) had been stretched thin, the system was
too much for one person to maintain, and it was
difficult to bring in new people to help with it.
Our first attempt to update the system came when PHP
version 4 was released. This release promised better
object-oriented capabilities, and the time was right for
Rackspace to dedicate more people to the project.
The system was totally redesigned from the ground
up, including new database schemas and application
design strategies. At this time we re-dubbed the
project “CORE,” an acronym for Core Objects Reused

9
Python is Rackspace’s CORE Technology, continued

Everywhere, in order to reflect the overall The Power of Introspection

design goal for CORE: modularity and One of the first tasks in writing the new
reusability across all systems in the company. framework was to build its database interface.
With that goal in mind, our team went to
work using the object-oriented features of PHP. Python’s introspection model had been
significantly enhanced with the release of
While we were able to re-fit the application Python 2.2. We decided to use it to build a
and add increased functionality, the project generic database interface class, based on a
ultimately failed due in large part to the DBI 2.0 compliant database connector. In
problems encountered while using the this approach, rather than writing queries
object framework provided by PHP. or table-specific wrappers by hand, a meta-
Memory leaks, inconsistent interfaces, incon- class abstracts all database queries into a
sistent internal data models, randomly freed single clean API.
objects, multiple object copies despite We create descendents of this meta-class
explicit use of references, internal PHP errors, to make an API for each table. Each table’s
and untraceable code failures made the class contains a few class constants that
task all but impossible to accomplish in PHP. describe the columns in the database. This
Even after we achieved a relatively stable way we can add new tables to the overall
code base, we were nowhere near our API quickly and simply without having to
goal of Core Objects Reused Everywhere worry about implementation details for
because we had to depart from pure object- any specific table.
oriented methods just to work around the The API also uses meta-data to automati-
problems inherent in PHP. It became clear cally validate and convert values passed to
that PHP was unsuitable for our large scale, the database. This is done by a “normalizer”
mission critical projects. A new solution function that converts the Python data types
had to be found. being passed through the API into valid
Python in CORE SQL values. The function also verifies types
We had always considered Python to be an and formats that are not necessarily checked
excellent candidate for implementing our by the database or by Python, such as
enterprise system, but it was initially passed phone numbers and ZIP codes.
over in favor of building upon the existing Reusing Objects Everywhere
(vast) code base we already had in PHP. At Once the database API was complete, we
that time, we felt that PHP could be used created a second layer of classes on top of
successfully in CORE by introducing a it. This higher level API implements the
better structured system design. business logic for specific applications, such
Unfortunately, that wasn’t enough to as contact management or trouble ticket
overcome our other problems with PHP, handling. It also prevents users from
so we re-evaluated our situation. The first performing operations that are inconsistent
alpha version of Python 2.2 had recently with Rackspace’s business practices, or
been released, and we decided to begin assigning data that would result in other
work on a new CORE framework using types of high-level corruption of the data
the new features that were available in in the database.
that version. With the creation of this second layer, we

10
achieved our original goal of Core Objects parser pre-compiles and caches the Python
Reused Everywhere. Programmers servlet code that it produces, the CGI
throughout the company began to use this solution is sufficient for our needs. It also
API to implement interfaces to application allows us to resolve issues such as database
functionality. This required little interaction connection pooling and restricting the
with our API development team, and it could execution environment outside of Python.
be done without fear of misusing the API. Unit Testing
While we designed the API primarily for Thanks to the unit testing module that
CORE, the central enterprise application, it comes with Python, our projects are
is reused in a number of other systems at reaching production with far fewer bugs
Rackspace. For example, one group built a than we had ever thought possible when
SOAP server on top of the API, in order to we were using PHP. During maintenance
access it from their PHP applications. Other with PHP, there was always a question of
applications use the API directly, and it has been whether a change in one place would break
extremely gratifying to see our work reused and something else in another part of
integrated so easily with other systems. the application.
Integrating Python with Apache
We now write unit tests for each and
With the API in place, our next task in every API as the API is being designed.
developing CORE was to find a useful This means that we can verify the changes
templating module to integrate our Python in one module as well as its effects on all
code with HTML pages running on the the others simply by running the unit tests
Apache web server. for the entire API.
After looking at a number of available Since introducing Python and unit testing,
Python-based templating modules, we opted the nature of the bugs that we see in
to create a simple parser of our own. Our deployed applications has shifted to include
approach was to convert server-side template primarily those in the user interface, such as
pages into Python servlets whose output is layout problems or faulty event handling.
sent by the HTTP server to the user’s browser. These days, very few bugs come from the
Although this was a fairly simple exercise, API itself, and even those are generally the
we did run into some problems stemming result of poor revision management or DBA
from our design of the CORE database coordination during application
meta-class. We found that altering classes deployment. Python can’t solve _all_
and modules at runtime, as is done by the problems during development, but it
meta-class, violates guidelines imposed by certainly has reduced the number of critical
Python’s optional restricted execution system defects for us.
environment. Since we felt that restricted
Documentation
execution was a necessary component in
supporting a persistent web module, we opted Lack of documentation has been a major
to deploy CORE using CGIs rather than problem with our previous development
mod_python or similar persistent solutions. efforts. We tried several tools and policies
to document our PHP efforts, but in the end
Since fast hardware and multiple servers are
these failed. Code changed too quickly, and
readily available, and since our template
the code-level documentation tools available

11
Python is Rackspace’s CORE Technology, continued

for PHP at the time were too finicky to Conclusion

justify the amount of effort required to get Python has dramatically improved devel-
the documentation to parse correctly. opment processes for the CORE project,
Additionally, despite careful planning and and it has led to the faster development
coding strategies, the mixture of PHP and times and more rapid releases that allow
HTML made deciphering and under- us to keep up with Rackspace’s ever-
standing the code more difficult. changing business processes.
Fortunately, Python was designed with Python enabled us to create a sophisticated
documentation in mind, with the use of dynamic data model that is flexible and easy
“doc strings” for modules, classes and to use for abstracting database operations.
methods. Since documentation is actually With it, we realized our goal of Core
a part of the language itself, and pydoc is a Objects Reused Everywhere.
standard module in the Python distribution,
it was easy to extract API documentation to Python’s integrated unit testing and documen-
HTML and other formats. tation tools greatly enhance our ability to
deploy and maintain a more stable, error-
Over time, we have found that the syntactic free product.
structure of Python makes for extremely
readable code, and that in itself helps in The result is a successful enterprise appli-
the overall task of documenting and cation that is instrumental in the delivery
maintaining code. of Rackspace Managed Hosting’s promise
of Fanatical Support, Unmatched Speed,
and Unlimited Flexibility in the managed
hosting industry.

12
 Putting Web Services to Work with Python
Dr. Tim Couper, Marc-Andre Lemburg,
and Siena Technology Ltd.

About the Authors Introduction

Dr. Tim Couper (tim@siena- Consultants naturally try to provide their customers
tech.com) is the chairman of with the best solutions for a problem. Sometimes this
Siena Technology. He holds a means exploring new areas together with the customer
mathematics D.Phil. and has or directing the project into a solution space that better
twenty years’ experience fits the problem than the usual “buzzword-compliant”
running software companies. approaches. We’ve seen these fail too often, misleading
He now spends most of his the project into solving problems relating to the
time consulting as architect
selected technology, rather than meeting the original
and technical lead for large
project plan.
development projects, and has Python Goes Fortune 500
been extensively involved in In a recent project for one of our customers, we faced
planning and coordinating the a problem that is quite common in Fortune 500
2002 and 2003 Python UK companies: Multiple clients running on unmanageable
conferences. client machines are tied closely to complicated
Marc-Andre Lemburg database relationships, making management and
(mal@siena-tech.com) is the
further development very difficult.
Chief Technical Officer (CTO) Thanks to the marketing efforts of several large server
of Siena Technology. He holds vendors, our solution space was quickly identified.
a degree in mathematics from What the company needed was Web Services, or put
the University of Duesseldorf. simply, a way for a client application to talk to the
Marc-Andre has been working server side in a reasonably standardized manner. Our
with Python since 1993, is a customer was already using two of the dominant
Python Core Developer, board technologies in this area: Microsoft .NET and BEA’s
member of the Python
WebLogic® J2EE server.
Software Foundation (PSF), When we came into the project, a team was already
author of the well-known trying to solve some of the company’s problems using
eGenix.com mx extensions the J2EE platform. However, we found that they were
(mxODBC and mxDateTime), spending more time trying to work around design
and was one of the executive problems in the technology than actually writing code
organizers of EuroPython 2002. for the services. Since the company had already been
introduced to Python in a smaller XML project, we
www.siena-tech.com
were able to convince the project lead to try a new
technique based on Python.
The idea was to leverage the efficiency of Python
programming together with its good database connec-
tivity to compete against the J2EE team. What we

13
Putting Web Services to Work with Python, continued

needed was a stable and robust server the new technique. Just as service writers
implementation and a flexible way to write shouldn’t need to think about low-level
and publish Web Services. Fortunately, the database connectivity, we felt that client
server had already been written in the form application programmers shouldn’t be
of the easily extensible eGenix.com bothered with the details of setting up
Application Server, so our task was simply connections and talking to the server side.
one of adapting this server to make writing We wanted to design a very simple appli-
services as easy as possible. cation programmer’s interface (API) which
Keep IT Simple, But No Simpler
would hide all the complications inherent
to networked client/server interaction.
Following the Python paradigm of “obvious
is better than obscure, explicit is better than Client side agents for Java® and Windows’
implicit,” we chose the most straight- COM interface made this possible by enabling
forward possible way of dealing with Web access to the Web Services from all major
Services: Each service was mapped to a client application environments such as
Python class, which provided the public Visual Basic (VB), Visual Basic for
methods to call from the client application. Applications (VBA as used in Word, Excel,
There were two reasons that we felt that and Access), Delphi, C++, Java, C# and
service implementations should look no others. These not only hide the protocol
different than any standard class implemen- level from the application programmer,
tation in Python: (1) a programmer should but also provide the key to enabling
not need to learn a new way of coding just security and fail-over solutions.
to be able to write services, and (2) existing Web Services standards are still in the
integrated development environments planning stage, and it is not at all clear
(IDEs) should be used to make coding which of the proposals will be accepted
services even easier. by standards organizations. By creating a
Since Python is an object-oriented true middle tier, we were able to hide the
programming (OOP) language in all particular methods and protocols we chose
respects, the implementation we chose inside of the Siena client and server, and
placed basic shared functionality into a were free to use existing security standards
Service base class that hides networked and authentication modes to build a secure
server interaction from the programmer. communication channel.
As a result, the service developer only Above and Beyond Web Services
needs to think about the business logic With these basic building blocks, we were
in the service methods and can rely on the moving towards realizing the Siena Web
server to automatically provide database Services Architecture, which included a
connection pooling, protocol handling, Python-based server side, a COM client
transaction control, and all the complicated side agent (also written in Python), and
interactions that are needed to make a a Java client side agent.
server side implementation robust.
One distinct advantage of writing both the
Client-side Bliss server side and COM client side of our Web
Another design goal for the system was to Services architecture in Python was that we
simplify client-side programming as much could automatically replicate services from
as possible, in order to make it easy to adopt the server to the client side. This was
14
possible whenever a service did not development possible. And IT management
depend on database connectivity or other was excited to see the overall high
elements specific to the server-side performance of our solution.
environment. Python’s data packaging The Siena Web Services Architecture has
facilities and portable byte code format become a crucial mission-critical component
made this operation quite easy to for this customer, as it moves from a two-
implement. The result was a significant tier to three-tier architecture and adds
boost in application performance, reduced fail-over and security to their Web Services.
network bandwidth requirements, reduced
network latency, and increased server The Siena Web Services Architecture will
performance, all without sacrificing the soon become part of our product line. If
efficiency of centralized server-side you are interested in the solution, please
management that makes Web Services visit our website at www.siena-tech.com/
so attractive to IT management. or contact us directly.
PS: After several months of effort, the
The Python Mantra
J2EE team never did get their Web
Still missing in our plan were the skills Services working.
needed to code Python servers and clients.
Most of the programmers in our team knew
only a mix of Java, Visual Basic, and C++.
While the J2EE group was working on
solving J2EE problems, we invested a day
in teaching Python to the rest of the team.
Python wasted no time making its way into
the hearts and minds of these programmers.
It was a thrill to hear fellow programmers
chiming in with our own Python mantra:
“This is what I’ve always been looking for.”
Results
Happy programmers are good programmers,
and good programmers work efficiently.
That’s what project management learned
at this point in our effort. The group’s Web
Services programmers quickly caught on to
the new Python-based system and devel-
opment progressed at amazing speed.
Services could now be implemented in a
few minutes rather than the days needed
using the typical J2EE approach. Now most
services were completed and deployed in
less than a day, and the ease and speed with
which they can be modified and tested has
made an incremental approach to service

15
 Python Streamlines
Space Shuttle Mission Design
Daniel G. Shafer

About the Author Introduction

Dan Shafer is a freelance Software engineers have long told their bosses and
author and sometime Python clients that they can have software “fast, cheap, or
coder who hangs out on right,” as long as they pick any two of those factors.
California's central coast. He Getting all three? Forget about it!
is a member of the PythonCard But United Space Alliance (USA), NASA’s main shuttle
Open Source development support contractor, had a mandate to provide software
team creating a GUI-building that meets all three criteria. Their experience with
framework for Python applica- Python told them NASA’s demands were within reach.
tions. He makes his living as Less than a year later, USA is nearing deployment of
a writer and a product devel- a Workflow Automation System (WAS) that meets or
opment consultant. A founder exceeds all of NASA’s specifications.
and former editorial director “Python allows us to tackle the complexity of programs
of Builder.com, Shafer has like the WAS without getting bogged down in the
been part of the web devel- language,” says Robin Friedrich, USA’s Senior Project
opment community almost Engineer. Friedrich conceived of the WAS project in
from its inception. response to a significant gap in the way shuttle
This article was previously mission planning was handling data management.
publised on Builder.com “Historically,” Friedrich says, “this data has been
communicated using paper and, more recently, data
file exchange. But both of these approaches are
error-prone. Catching and fixing errors as well as
responding to frequent change requests can bog
such a system down.” Complicating the issue was
the challenge of finding money to improve the flight
design process in an era of declining budgets for
space activities.
“Just in time” Provides a Solution—and More Problems
USA decided they needed a way to “minimize data
changes and the resulting rework.” The shortest route
to that goal would be to shift the design work to the
end of the process so that flight characteristics would
have a good chance of already being finalized. In other
words, as Friedrich says, “We decided we needed to
do this data management work ‘just in time’.”
A just-in-time solution, however, generally puts more
stress on both people and systems to get things right

16
the first time because postponing these The result? “We achieve immediate
activities to the end of the process means functioning code so much faster in
a loss of scheduling elasticity. Python than in any other language that
“The obvious answer,” according to Friedrich, it’s staggering,” says Friedrich. “Java and
“was to create a central database repository C++, for example, have much more baggage
to help guarantee consistency and to provide you have to understand just to get a
historical tracking of data changes.” An functioning piece of software.
Oracle database was designed to store the “Python also shines when it comes to code
information, but a graphical front end to maintenance,” according to Friedrich.
manage the process of workflow automation “Without a lot of documentation, it is hard
was clearly an essential component of an to grasp what is going on in Java and C++
effective solution. “We knew from experience— programs, and even with a lot of documen-
we do a good bit of Java coding in our tation, Perl is just hard to read and maintain.”
group–that using C++ or Java would have Before adopting Python, Friedrich’s team
added to the problem, not the solution,” was doing a good bit of Perl scripting and
Friedrich maintains. C++ coding. “Python’s ease of maintenance
Python a Mainstay Since 1994
is a huge deal for any company that has any
significant amount of staff turnover at all,”
Enter Python. “I literally stumbled across says Friedrich.
Python as I was searching the pre-Web
Gopher FTP space for some help with a The team had already developed a moder-
C++ project we were doing,” says Friedrich. ately large number of C++ libraries. Because
Being an inveterate systems engineer, of Python’s easy interface to the outside
Friedrich “just had to investigate it.” He world, USA was able to retain these
was stunned by what he discovered. libraries. “We wrote a grammar-based tool
that automatically interfaced all of our C++
“Twenty minutes after my first encounter libraries,” says Friedrich.
with Python, I had downloaded it, compiled
it, and installed it on my SPARCstation. It Another aspect of Python that Friedrich
actually worked out of the box!” found eminently significant is its shallow
learning curve. “We are always under the
As if that weren’t enough, further investi- gun on software projects, like everyone
gation revealed that Python has a number else,” he says. “But for any programmer,
of strengths, not the least of which is the picking up Python is a one-week deal
fact that “things just work the first time. because things just behave as you expect
No other language exhibits that trait like them to, so there’s less chasing your tail and
Python,” says Friedrich. far more productivity.” He contrasts that
He attributes this characteristic to three with C++ and Java, which he says takes a
primary language features: good programmer weeks to grasp and
• Dynamic typing months to become proficient.
• Pseudocode-like syntax Friedrich says that even the non-
• The Python interpreter programming engineers at USA learned to
do Python coding quickly. “We wanted to

17
Python Streamlines Space Shuttle Mission Design, continued

draft the coding energy of the engineering At the core of the design is the Task object,
staff, but we didn’t want them to have to which stores all information relevant to a
learn C++. Python made the perfect 4GL single task in the workflow network. The
programming layer for the existing C++ end user can view the network in a PERT-
classes.” style chart layout, where color coding
One Coder and 17,000 Lines of Code Later
reveals at a glance which tasks are finished,
which are in process, and which have not
The WAS project, which has required some- yet been started.
what less than a man-year of effort, has been
coded by a single senior software engineer, Two other graphical interface windows
Charlie Fly, who has cranked out some 17,000 allow the user to manage the dependencies
source lines of code (SLOC). Python plays among data items in the network and to
the central role, managing data interactions view and edit individual task details.
and the task network. All of the code for the UIs was also done
In the system, user tasks communicate with in Python, using the popular Tkinter library
a Python data server, which in turn connects along with an open source package of
to an Oracle server via DCOracle. Using supporting modules. Tkinter is included
Oracle’s built-in trigger mechanism to send in all standard Python installations.
a message to WAS as data records are “USA is pleasantly surprised by how much
updated, the WAS calculates which tasks quality software we can deliver,” Friedrich
are now data-ready and notifies the says. “And each time we demonstrate
appropriate user. success with Python, we add a few more
believers to my growing list!”

18
 Wing IDE Takes Flight with Python
Stephan R.A. Deibel and John P. Ehresman

About the Authors Introduction

Stephan R.A. Deibel has been Wing IDE™ is a commercial integrated development
designing and developing environment for the Python programming language.
software for almost twenty Wing provides developers with a full-featured source
years. He has worked exten- editor, debugger, code browser, and many other tools
sively in medical informatics specifically designed for use with Python. Wing works
and as a software consultant. with all forms of Python, whether running as a stand-
Before finding Python, his alone app, under a web server, or in a custom
projects included one of the
embedded scripting environment. Several GUI layers
(wxPython, PyQt, PyGTK, and Tkinter) are supported,
first Macintosh-based multi-
as are Zope and mod_python for web development,
media authoring systems and
and pygame for game development.
an early CORBA implemen-
tation. Stephan is now CEO Wing was inspired in 1999 by several experiences
and co-founder of Archaeopteryx we, its developers, had using Python alongside other
Software, Inc., makers of technologies. At that time, we were working as
Wing IDE.
consultants charged with evaluating a number of
alternatives for tiered web development. Some of
John P. Ehresman has been these were based on Java and some on Visual Basic,
programming for more than MTS, and ASP. Concurrently, we happened to be using
ten years in medical informatics Python to prototype some of the functional require-
and as a software consultant. ments for the web-deployed business applications
He has been using Python we were developing.
since version 1.2 and is now
It wasn’t long before we found ourselves comparing
co-founder of Archaeopteryx
our Python prototypes favorably to the actual systems
Software, Inc. we were developing. Python was a much more
www.wingide.com productive way to work, and it seemed to result
in at least as good an end product.
Unfortunately, our clients never seriously considered
Python simply because it wasn’t a mainstream (namely
Java or Microsoft) technology. But it was clear to us
that Python could have been a significant cost saver
and competitive advantage for them, and we saw a
business opportunity in helping other organizations
benefit by using Python.
Development Approach
Work on Wing IDE started almost right away, in mid-
1999, initially on a part-time basis. We realized that

19
Wing IDE Takes Flight with Python, continued

writing an entire IDE wasn’t going to be cation, scalability, rock-solid stability, and
easy and wanted to be sure that Python its strong support for mixed-language
was really as good as it appeared to us at development.
the time. The logical way to approach this Productivity
was to develop the IDE itself in Python.
This would give us proof of concept and Over the course of this project, we have
let us become early users as we started to been able to write on average over 175
develop and debug Wing IDE with itself. lines of debugged, documented, tested
code per developer per day. Over a period
To speed development and keep costs of 660 FTE days, we produced a total of
down, we chose to base Wing on as many approximately 121K lines, of which 77K
open source modules as we could find. were written in Python. Even without
The GUI was written with GTK, which considering that a line of Python is typically
is accessed from Python via PyGTK. The equivalent to ten or more lines of C, we
source editor is based on Scintilla, an were extremely pleased with this result.
open source code editor component.
And printing is implemented via The entire product, including third-party
py2pdf from ReportLab. open source modules, actually contains
on the order of 1.2 million lines of code,
Initial development was on Linux but of which 274K lines are Python.
we planned to support at least Windows
and eventually other Unix-like operating So why was using Python so productive,
systems. For this reason, we avoided even when only 63% of the code we wrote
platform-specific implementations and was in Python? There are several answers
chose cross-platform technologies. to this question:

Additional development tools used in 1) Simple syntax—Although use of

the project included gcc, Gnu make, indentation to indicate program structure
latex, pdflatex, latex2html, emacs/xemacs sometimes turns off first-time Python users,
(before Wing was functional), Visual the reduced typing burden that comes with
C++ 6, and cygwin. avoiding {}’s and similar syntactic sugar
does matter over the course of writing
Results and rewriting hundreds of thousands
Our work on Wing IDE has been quite a of lines of code.
success. We were able to develop faster 2) Dynamic high-level data typing—Lack
than we originally expected, and to deliver of strong data typing, another commonly
Wing IDE on Linux, Windows 98 through cited “weakness” in Python, is in practice
XP, Mac OS X with XDarwin, Solaris, and a significant advantage for most kinds of
FreeBSD without major platform-specific software development. The bottom line is
development work. Today, our product is that you don’t need strong data typing in
receiving good reviews and is selling well. the context of a language like Python.
All of this has been possible without any Common coding errors are caught anyway
outside funding and with a development by the type system: You still can’t add a
team of just two people. number to a string, reference past the end
The biggest benefits of using Python have of an array, or call a non-existent class
been in overall productivity, cross-platform method. Dynamic high-level data typing
deployment, speed of the resulting appli- cuts out great volumes of support code
20
and makes it possible to write flexible and 6) Faster development and deeper
introspective code (more on this below). prototyping—Python increases speed of
3) Powerful, easy-to-use data structures— development to the point where proto-
Python’s built-in list and dictionary data typing can be integrated into and
structures can be used in combination to interleaved with the primary development
build just about any fast runtime data process. When it takes only half a day
structure in a snap. This further reduces the to try out a new approach to a problem,
amount of support code you need to write. rather than the week it might take in C
or C++, programmers are more often
4) Extensive standard library—Python comes empowered to rework existing imperfect
with a vast standard library supporting code, and to try out new ideas. This results
everything from string and regular expression in the more rapid incorporation of experience
processing to XML parsing and generation, into an application’s design, and leads to
Web Services tools, and internet protocol higher code quality.
support. Many common programming
tasks have already been built into the Cross-platform Deployment
standard library, making it possible to do Wing IDE runs on a variety of Posix®
more with less code. Third-party modules operating systems and Windows. Through-
are also available for database access, CORBA, out our development process, we’ve been
COM, statistics, math, image processing, very happy with the way that Python
and much more. performed across platforms. The same
5) Introspection—Python’s flexible data Python source or compiled Python byte
typing system extends also to code docu- code files can be shipped to clients
mentation, classes, methods, and even the regardless of target platform, making
way in which methods are called. Python support quite easy.
makes introspection extremely accessible Speed, Scalability, and Stability
to the programmer and, remarkably,
When we started to write in Python, our
introspective code remains readable and
previous experience in compiled languages
maintainable. This can be very useful in
led us to believe that we would be spending
redirecting I/O to classes of your own
a fair amount of time either optimizing
design, writing a tracer that determines
code or converting it into C or C++ once
code coverage, packing up data to store
we had prototyped it. As it turned out,
on disk or send over the network, devel-
most of the time Python produced a
oping glue code, writing table-driven
snappy end product that didn’t require
algorithms, extracting documentation
any extra work.
from code at runtime, applying design-
by-contract development methods, and This happened partially because most
in building various types of meta-classes. Python code is really just a thin interpreted
For almost every programming task, layer over functionality that is written in C
Python makes it not only possible but or C++. In our case, this included not just
quite easy to build meta-code where one Python’s fast built-in data structures and
might otherwise end up building gobs standard libraries, but also the bulk of the
of manually crafted code. GTK GUI development layer and the
Scintilla source editor.

21
Wing IDE Takes Flight with Python, continued

In the course of development and in these results confirm without any doubt
responding to thousands of support tickets that working in Python is far more
over a three year period, we have never run productive than working in C or C++.
into any significant problems with Python In hindsight, we believe that we could
itself, either in scalability or stability. Wing have converted smaller units of code into
IDE can handle software projects with C, by writing more general data-driven
thousands of Python files, and in many processing engines and by more carefully
cases can run for weeks without problems. selecting code to convert instead of
To our knowledge, we have yet to see Wing converting whole modules at a time. Our
IDE crash because of a flaw in the Python primary goal for Python in the future is to
interpreter or its standard libraries. be able to use it more often, even in
Mixed-language Development performance-critical sections of code.
Python is almost always fast enough but This effort should benefit from projects
we did run into a few cases where the like pyrex, which allows the use of Python-
interpreter introduced too much over- like code in the development of compiled
head. The Wing IDE debugger and extension modules, and psyco, which is a
the source code analysis engine both just-in-time compiler for Python.
contain modules that engage in extremely Quirks
CPU-intensive processing. These modules There are just two quirks that affected our
needed to be written in C in order to development with Python. The relative
squeeze out as much speed as possible. impact they had on our project was tiny
Fortunately, Python is designed to make it compared to Python’s benefits, but for
quite easy to call back and forth between balance we feel they are worth mentioning:
Python and C or C++.
1) Like Java and other languages, Python
In most cases, we wrote and debugged occasionally deprecates old features, or
code first in Python, and then converted fixes minor bugs in a way that can poten-
by hand into C. This approach worked tially break existing code. This is done over
well for us. Working initially in Python the course of a number of releases, so that
was much more efficient and the programmers will first see deprecation
conversion process relatively painless. warnings, and only later be impacted by
Analysis of our records shows that 360 the change. We ran into this only once
days were spent on 77K lines of Python when Python 2.0 began to disallow
code and 300 days (almost as much) on multiple arguments to the sequence append
44K of C, C++, or other code. From our method. This problem required changing
experience with code conversions, we exactly three easily found calls in our code
believe it is roughly correct for most types base of over 77K lines of Python.
of performance-critical code to equate one
line of Python with ten lines of C or C++ 2) Different versions of Python can produce
code. This means that about 5-10% of our incompatible compiled byte code and
application functionality is in C or C++ requires that C/C++ extension modules
and the rest is in Python. Even considering are compiled against a specific version of
that the C/C++ code is somewhat more Python. For example, while Python 2.2.2
complex than most of the Python code,

22
works happily with Python 2.2.1 or 2.2.0 Conclusion
byte code and extension modules, it will Without Python, we could not have sustained
print warnings and may run into problems the Wing IDE development effort long
running against those compiled against enough to produce what is now a successful
Python 2.1.x or earlier. There are solid software product. Python has been more
technical reasons behind this design productive, robust, and portable than any
choice, but it does require some additional other technology we have tried. Through
work when packaging applications for our experiences providing technical support
distribution to users running different for the IDE, we know that we are not alone
versions of Python. In the Wing IDE in these assessments. Feedback from our
debugger, we solve the problem simply customers often includes strong endorse-
by storing separate directories for each ments for the productivity of Python, Wing
interpreter version and importing modules IDE, and related technologies such as Zope.
accordingly at runtime. For the IDE itself,
we solve it by shipping with a specific
Python interpreter included; a task that’s
easily accomplished with support found
in the Python standard library’s
distutils package.
No other language we have used has been
this devoid of quirks, even those we have
used much less intensively and across fewer
language revisions.

23
 Python Enterprise-Wide at the
University of St Andrews in Scotland
Hamish Lawson

About the Author Introduction

Hamish Lawson is a software The IT Services department at the University
developer in the IT Services of St Andrews, Scotland, develops and maintains
department at the University software systems used in a variety of capacities
of St Andrews, Scotland. He throughout the university.
specializes in web-based I had several years of experience working with Perl
information systems. when I took my first serious look at Python back in
www.st-andrews.ac.uk 1999. Our team’s projects were becoming bigger and
more complex, and it was obvious that we needed to
bring to them more structure and clarity. I had been
looking at Java for some time, but its potential benefits
seemed to come at the cost of a steep learning curve,
and an overall increase in development time. In contrast,
Python appeared to offer the prospect of having both
clarity and productivity at the same time. And if we
ever needed to make use of Java’s class libraries there
was always Jython, an implementation of Python for
the JVM. The increasing number of Python books
being published testified to the language’s growing
popularity, and the number of available libraries was
beginning to rival Perl’s. This convinced me to give
Python a try.
Python Finds a Home
Soon thereafter, I introduced Python to my fellow
developers in the IT Services department at the
University of St Andrews, Scotland. It is now the
mainstay of our software development efforts.
Python has been used successfully by IT Services for
a number of projects. These have included systems
administration scripts, where it is used alongside Perl
and shell scripts, and also sizeable enterprise systems
deployed across the university.
By using it on a number of projects, we have come to
understand that Python’s dynamic nature, support for
high-level data structures, and easy object-orientation
all lower the barrier to writing well-structured reusable
code in less time. The language’s clear and simple
syntax helps to reveal the sense (or otherwise!) of our
code. This makes it easier to understand and reason
24
about code during development and–more in the spring and summer, through to
critically–during later maintenance. The fact final validation during matriculation at
that Python is so easy to learn has been quite the start of the new academic year. The
useful as well. system implemented work-flow and
Job Vacancies Facility
notification mechanisms.

One of our earliest Python projects was a Because of the number of concurrent users
facility for university job vacancies. This that the system was expected to support,
was implemented using Zope, an innovative particularly during matriculation, we felt
Python-based web application server that that a traditional CGI approach could lead
provides a range of web components plus to performance problems. Instead, we
powerful facilities for templating and employed mod_scgi, an Apache module
integration. I started the project with a that implements the client end of the SCGI
colleague who had more experience in web protocol for long-running web processes
design than programming. After some time, (similar to FastCGI). The server end of
we found that Python’s powerful simplicity the SCGI protocol was implemented by
enabled my colleague to improve his Quixote, a Python-based web framework.
programming skills rapidly, to the point Quixote also provided a URL mapping
where he was able to continue development mechanism that simplified the job of
of the system by himself. Zope itself also publishing objects on the web. Cheetah
helped in this regard, by reducing the was again used for handling presentation,
amount of programming that needed to and Oracle was used for storage.
be done in the first place. We were assigned this project somewhat late
in the process, and it was clear that it would
Student Record System
come into active use while parts of it were
One of our larger projects was a system still being developed. To support this, we
for managing student records. It employs a planned to deliver the system in stages, but
model-view-controller architecture in order we were aware from the outset that many of
to promote reuse. Presentation is handled the system requirements still had to be
by Cheetah, an open-source templating discovered and understood. We knew this
technology for Python. The interface would lead us to rewrite parts of the system
between Python and the underlying Oracle® as we went along and worked to obviate
database is handled by the high-quality this with design decisions aimed at decou-
DCOracle2 driver made freely available pling the system’s components. Python’s
by Zope Corporation. dynamic nature and supreme flexibility
Matching Students with Classes made it easier to write generic interfaces,
Python was also used for a web-based which later facilitated the rewriting and
system that managed the process of refactoring tasks we had to undertake.
matching students with the classes they Conclusion
wished to take in the coming year. The Since switching to Python, we are writing
system was used by most of our students better structured and more readable code
and a number of our staff for various stages in less time ... and it’s fun! I can’t think of
of the process–from selecting preferences a better testament to a programming tool.
25
 Python Powers Journyx Timesheet
Curt Finch and John Maddalozzo

About the Authors Introduction

Curt Finch, Journyx founder and Journyx Timesheet (tm) is a commercial application
CEO, started the company in that provides time, expense, and project tracking. In
1996 after a successful career 1996, Curt Finch, Journyx CEO and founder, was
in the consulting industry working in the staffing industry when he saw an
participating in and managing opportunity to use the Web to accurately collect and
engagements with Fortune 100 store employee timesheet information.
companies such as Tivoli®, IBM®, The first version of Timesheet focused on collecting
and Prudential Securities. accurate cost information, with an eye towards
John Maddalozzo, Journyx V.P.
applying that data in the formulation of new project
cost projections. Since then, Timesheet has expanded
of Engineering, joined Journyx
considerably to facilitate tracking of time, mileage, and
in 1999 after a twelve year
expenses, not just for project management but also for
career in Unix kernel devel-
billing and payroll purposes. Optional modules are
opment at IBM’s AIX
available for paper-less expense reporting, advanced
Engineering group.
user role management, automated billing and payroll,
www.journyx.com/ and to facilitate system access for disconnected
traveling users.
Today, Timesheet is platform-independent, flexible
enough to be reconfigured by customers to fit unique
organizational needs, and scales to tens of thousands
of users for the large enterprise.
Python from the Start
Curt Finch chose Python initially on the recommen-
dation of a friend, Steve Madere, who had founded
Dejanews.com (now a part of Google®). Describing
the rationale for his choice, Curt said “I looked at Java
and C and came to the conclusion that one line of
Python is ten lines of Java or one hundred lines of C.
Developers write code at basically a constant rate so
I chose Python which was (and is) the highest level
language I’ve ever seen that is also flexible enough to
be generally useful.”
Architecture
From the beginning, Timesheet was designed and
implemented as a web application. It uses a three-
tiered web application architecture with separate layers
for web presentation, business logic, and data storage.
As time has progressed, the application’s functionality
26 has advanced considerably, and Curt’s decision to use
Python for an implementation language has In developing Journyx, the two greatest
proven to be a good choice. benefits of Python were the speed with which
Python is currently used for all application features could be written and deployed, and
logic in the Timesheet application. This its true write-once-run-anywhere cross-
includes all code between the initial Apache platform capabilities. Journyx developers
dispatch, where mod_python is employed to have found that the simplicity and clarity of
expedite interpreter instantiation, through the Python combine with its object-oriented
application logic, and down to the point of properties to make it a very powerful and
call out to the database transport layer. productive language. Python’s rich standard
Timesheet uses not only the Python standard library, which includes modules for things like
library but also several independently string manipulation and HTML generation,
developed open source Python subsystems, further supports programmers in meeting
such as PyXML and ActZero’s SOAP support. aggressive development schedules.
PyXML is used to implement certain business Because of these properties of the language,
rules and to develop jxAPI, which is a SOAP- Python has enabled Journyx to add features
based API, into the application logic. Work is more quickly than our competitors. We’ve
in progress to extend this API to define Web been able to implement SOAP/XML and
Services Description Language templates for WSDL support and extended other aspects
the jxAPI functions. The application currently of the application’s functionality well ahead
builds and ships with Python 2.1.1. of competitive products. Some of the key
Timesheet also incorporates several non- enablers of this efficiency in maintenance
Python technologies. The Unix and Linux® and improvement is the inherent clarity and
distributions are packaged with the Apache readability of the Python language, a vibrant
HTTP server and PostgreSQL database. The and responsive Python development community,
Timesheet distribution for Windows® ships and the high degree of backwards compati-
with an optional Microsoft® Desktop Engine bility and stability we have seen as the
(MSDE) database and integrates with language design evolves over time.
Microsoft IIS. Timesheet can be configured Python’s cross-platform standard library and
to use a variety of third-party databases. platform-independent byte code file format
Results
allow the deployment of Python modules to
any platform, regardless of which platform
The Timesheet project has succeeded spec- the module was prepared on. This helped
tacularly, generating millions in revenue and not only in avoiding per-platform devel-
allowing Journyx to grow every year, even opment overhead but also facilitates customer
under the current economic conditions. support for the Timesheet software product.
Journyx, like many of our customers, uses
Timesheet internally as a mission critical Conclusion
part of the company infrastructure. It is Python has been an important competitive
used extensively for project tracking, advantage for us, and even as our Python
billing, and payroll. To date, approximately code base grows in complexity and maturity,
11 person-years have gone into the Journyx the natural advantages of Python enable us to
Timesheet product, resulting in over provide a high quality mission-critical appli-
150,000 lines of Python code. cation at a competitively low cost.
27
 Python Resources

Python Software Foundation

http://www.python.org/psf

Pythonology
http://pythonology.org

O'Reilly & Associates, Inc.

http://python.oreilly.com

O’Reilly Network Python Dev Center

http://www.onlamp.com/python

Python Advocacy HOWTO

http://www.amk.ca/python/howto/advocacy/advocacy.html

Python Starship
http://starship.python.net

Vaults of Parnassus: Python Resources

http://py.vaults.ca/parnassus

Daily Python-URL
http://www.pythonware.com/daily/index.htm

Pyzine
http://www.pyzine.com

Python Journal
http://pythonjournal.cognizor.com

Upcoming Python Conferences and Events

PyCon DC 2003 - March 26-28, Washington D.C.
http://www.python.org/pycon

Python UK Conference 2003 - April 2-3, Oxford, UK

http://www.python-uk.org

EuroPython 2003 - June 25-27, Charleroi, Belgium

http://europython.org

Python11 at OSCON 2003 - July 7-11, Portland, Oregon

http://conferences.oreillynet.com/os2003
 Embracing and Extending Proprietary Software
00
p to $4
Save u eg ister
ou R
when y 19
by May

Don’t miss the Python 11 Conference at the O’Reilly Open Source

Convention. Learn the latest on Python and Zope with comprehensive
case studies, tutorials, and sessions covering the latest modules.

http://conferences.oreilly.com/oscon

Wrap yourself around

our new nutshell.
In classic O’Reilly “In a Nutshell” style, Python in a Nutshell
offers Python programmers one place to look when they
need help remembering or deciphering the syntax of this
open source language and
its many modules. This
comprehensive reference
guide makes it easy to
look up all the most
frequently needed infor-
mation–not just about the
Python language itself, but also
Python in a Nutshell the most frequently used parts of
By Alex Martelli
the standard library and the most
ISBN 0-596-00188-6
important third-party extensions.
March 2003
$34.95

http://www.oreilly.com

Item # 30067