Schwake & Voit, 2004 (Improved Methods For Mathematically Controlled Comparisons of Biochemical Systems)

Theoretical Biology and Medical
Modelling BioMed Central
Research Open Access

Improved methods for the mathematically controlled comparison
of biochemical systems
John H Schwacke and Eberhard O Voit*
Address: Department of Biometry, Bioinformatics, and Epidemiology Medical University of South Carolina 135 Cannon Street, Suite 303
Charleston, SC 29425, U.S.A
Email: John H Schwacke - [email protected]; Eberhard O Voit* - [email protected]
* Corresponding author
Published: 04 June 2004 Received: 18 May 2004

Accepted: 04 June 2004
Theoretical Biology and Medical Modelling 2004, 1:1 doi:10.1186/1742-4682-1-1
This article is available from: http://www.tbiomed.com/content/1/1/1
© 2004 Schwacke and Voit; licensee BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted
in all media for any purpose, provided this notice is preserved along with the article's original URL.
Abstract
The method of mathematically controlled comparison provides a structured approach for the
comparison of alternative biochemical pathways with respect to selected functional effectiveness
measures. Under this approach, alternative implementations of a biochemical pathway are modeled
mathematically, forced to be equivalent through the application of selected constraints, and
compared with respect to selected functional effectiveness measures. While the method has been
applied successfully in a variety of studies, we offer recommendations for improvements to the
method that (1) relax requirements for definition of constraints sufficient to remove all degrees of
freedom in forming the equivalent alternative, (2) facilitate generalization of the results thus
avoiding the need to condition those findings on the selected constraints, and (3) provide additional
insights into the effect of selected constraints on the functional effectiveness measures. We present
improvements to the method and related statistical models, apply the method to a previously
conducted comparison of network regulation in the immune system, and compare our results to
those previously reported.
Background the benefits of one design over another and to understand

Metabolic and signal transduction pathways in biological the selection criteria driving an evolutionary design choice
systems are typically complex networks that necessitate we need methods by which objective comparisons of
the application of mathematical modeling and computer alternative designs can be performed.
simulation in efforts to understand their behavior. Math-
ematical models, developed through these efforts, have To perform these comparisons we first require a mathe-
value both as tools for predicting system behavior and as matical framework with which we describe the designs of
descriptions of the system that facilitate the study of the interest and compare those designs with respect to func-
embodied design principles [1,2]. A design principle, as tional effectiveness measures. The framework chosen here
defined by Savageau, is a rule that characterizes a feature is based on the form of canonical nonlinear modeling
of a class of systems and thus facilitates understanding the referred to as synergistic or S-systems. S-systems, devel-
entire class. As these rules are identified and characterized oped as part of Biochemical Systems Theory (BST), are sys-
a catalog of patterns will be developed for use in the iden- tems of nonlinear ordinary differential equations with a
tification of additional instances of these patterns within well-defined structure [4-6]. The time rate of change of
biological systems [3]. To gain a greater understanding of each quantity in the system is described by a differential
Page 1 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
equation of the form given in Equation 1 where X i indi-

cates the first derivative of quantity Xi with respect to time
and Vi+ and Vi− are positive-valued functions represent-
ing the influx and efflux respectively.
A
X i = Vi+ − Vi− X4 X1
n+ m n+ m (1)
g h
= α i ∏ X j i , j − β i ∏ X j i , j for i ∈ 1...n
j =1 j =1 X5 X2
-
These quantities may represent, for example, substrate,
enzyme, metabolite, cofactor, or mRNA concentrations
and are referred to generically as pools. The system con- X6 X3
sists of n equations of this form, one for each of the n
dependent variables in the system. The remaining m vari- B
ables, Xn + 1 … Xn + m, represent independent quantities.
The right-hand side of each equation consists of two X4 X1
terms, one describing the influx or production of the pool
of interest ( Vi+ ) and one describing its degradation or X5 X2
efflux ( Vi− ). Both terms are in power-law form. Any other
pool (independent or dependent) in the system that influ- X6 X3
ences production or degradation appears as a factor in the
appropriate power-law term of the effected pool's differ-
ential equation. The exponential coefficient of the factor,
referred to as its kinetic order, determines the direction Figure 1 and alternative systems
Reference
and degree to which the change is influenced. Positive Reference and alternative systems. Biochemical maps
kinetic orders indicate that the influence increases or acti- for the reference system (with suppression) and the alterna-
vates the flux and negative kinetic orders indicate that the tive (without suppression) are given in A and B respectively.
influence decreases or inhibits the flux. Kinetic orders Adapted from Irvine and Savageau [21].
associated with the influx term are typically given the label
gi,j where the indices i and j denote the influence of varia-
ble Xj on the influx to Xi. The label hi,j is typically given to
kinetic orders associated with the efflux term. The multi-
plicative factors αi and βi are positive quantities referred to the system under study to the parameters of the system. If,
as rate constants. They scale the influx or efflux rate and for example, the influx to a variable of interest, Xi, is regu-
thus control the time scale of the reaction. The validity of lated by some other variable Xj then the parameter gi,j will
this power-law representation has been analyzed exten- be non-zero. If the regulation inhibits the influx, the
sively and demonstrated in a variety of biological system parameter takes on negative values and if the regulation
modeling applications [7-9]. activates the influx, the parameter takes on positive val-
ues. This property of S-systems is particularly useful when
The S-system representation offers two key advantages in performing a controlled comparison of two structures that
the performance of controlled comparisons. First, S-sys- differ in their regulatory interactions. The alternative
tems have a form that allows for the algebraic determina- structure, without a particular regulatory interaction, can
tion of the system's steady state by solution of a system of be determined from the reference by forcing the value of
linear equations under logarithmic transformation of the the appropriate kinetic order to 0 (Figure 2).
variables (see Appendix). From this steady-state solution,
it is possible to determine the local stability of the steady S-systems provide a convenient method for the character-
state, the sensitivity of the steady state with respect to ization of systemic performance local to the steady state.
parameter changes, and the sensitivity of the steady state System gains, parameter sensitivities, and the margin of
with respect to variation in the independent variables. The local stability are easily determined and often form the
S-system representation is also advantageous in that it basis of functional performance measures used in control-
provides a direct mapping from the regulatory structure of led comparisons. Logarithmic gains represent the change
Page 2 of 18
The method of mathematically controlled comparison

provides a structured approach for the comparison of
A design alternatives under controlled conditions much like
a controlled laboratory experiment [10]. The approach, as
X4 X1 X2 X3 currently applied, is implemented in the following steps.
- + (1) Mathematical models for the reference design and one
or more alternatives are developed using the S-system
X5 modeling framework described above. The alternatives
X& 1 = αX 4g 1,4 X 3g1 ,3 − β1 X 1h1 ,1 are allowed to differ from the reference at only a single
process that becomes the focus of the analysis. (2) The
X& 2 = β1 X 1h1 ,1 − β 2 X 2h 2 ,2 alternative design is forced to be internally equivalent to the
X& 3 = β2 X 2 2 ,2 − β 3 X 3 3 ,3 X 5 3 ,5 reference by constraining the parameters of the alternative
h h h
to be equal to those of the reference for processes other

than the process of interest. (3) Using the mathematical
framework, selected systemic properties or functions of
B those properties are identified and used to form con-
straints which fix the, as yet, unconstrained parameters in
X4 X1 X2 X3 the alternative design. Typically, steady-state values and
+
selected logarithmic gains are forced to be equal in the
X5 reference and alternative. Parameters for the process of
interest in the alternative are then determined as a func-
X& 1 = α ′X X − β1 X
g1′ , 4 0 h1,1
4 3 1
tion of the parameters in the reference so as to satisfy these
X& 2 = β X − β 2 X 2h2 , 2
1 1
h1 ,1 constraints. The application of these constraints forces the
reference and alternative to be externally equivalent with
X& 3 = β X2
h2 , 2
− β 3 X 3h3 ,3 X 5h 3, 5
2 respect to the selected properties. The term "external
equivalence" refers to the fact that the alternative and ref-
erence are equivalent to an external observer with respect
Figure 2mapping: pathways to S-systems
Example to the constrained systemic properties. Constraints are
Example mapping: pathways to S-systems. The S-sys- imposed until all of the free parameters in the alternative
tem framework provides for a straightforward mapping of are determined. (4) Finally, measures of functional effec-
biochemical pathway maps into systems of equations. The tiveness relevant to the biological context of these designs
pathway and equations for cases A and B differ only in the are determined and used to compare the reference and its
feedback inhibition of the first step in the process. This inhi- internally and externally equivalent alternative through
bition is represented by a single parameter, g1,3. algebraic methods.
In many cases the comparison of these functional effec-

tiveness measures cannot be determined independent of
the parameter values. To improve the applicability of the
method in these cases, Alves and Savageau extended the
method of controlled comparisons through the incorpo-
in the log value of the steady state of a dependent variable ration of statistical techniques [11,12]. Under this exten-
or flux as a result of a change in the log value of an inde- sion, parameter values are sampled from distributions
pendent variable (see Appendix). A log gain of Li,j = L(Xi, representing prior knowledge about the likely ranges for
Xj) can be interpreted as an indication that a 1% change in those parameters. An instance of the reference design is
independent variable j will result in an approximate Li,j% constructed from the sampled parameters and an instance
change in the steady-state value of dependent variable i. of the alternative is then constructed from the reference by
Logarithmic gains provide a measure of the effect or applying the constraint relationships. Functional effec-
"gain" of an independent variable on the steady state of tiveness measures are then computed for the each sam-
the system. A related measure, referred to as system sensi- pled reference and its equivalent alternative (MR,i and
tivity, measures the robustness or the degree to which MA,i). The ratio of the performance measure of the refer-
changes in the system parameters (kinetic orders and rate ence relative to that of the alternative is computed for all
constants) affect the steady state of the system (see Appen- of the samples and plotted as MR,i/MA,i versus a property P
dix). A sensitivity of S = S(Xk, gi,j) indicates that a 1% of the reference design. A moving median plot is then pre-
change in parameter gi,j will result in an approximate S% pared by plotting the median of MR,i/MA,i versus the
change in the steady-state value of dependent variable Xk. median of P in a sliding window to reveal both the
Page 3 of 18
median of the relative measure and its variation across the

range of P. If M is defined such that smaller values indicate
greater functional effectiveness, ratios of MR,i/MA,i < 1
indicate that the reference is preferred to the alternative A
according to the given measure. Examination of the den-
sity of ratios and moving median plots allows determina- X4 X1 X2 X3
tion of preference for the reference over the alternative (or - +
visa versa) and how that preference varies with the
X5
selected property. These extensions have been applied to
the analysis of preferences for irreversible steps in biosyn-
B
thetic pathways [13] and to the comparison of regulator
gene expression in a repressible genetic circuit [14]. X4 X1 X2 X3
- +
Rationale for Improvements

X5
While the Method of Mathematically Controlled Compar-
isons has been successfully applied in many cases [13-20],
we offer for consideration enhancements to the method
that extend the application of sampling and statistical Figure 3 pathway alternatives
Biosynthetic
Biosynthetic pathway alternatives. Biosynthetic path-
comparison given by Alves and Savageau [11,12]. These
ways similar to that illustrated were compared using the
enhancements are offered primarily to (1) allow for the method of mathematically controlled comparison by Alves
incremental incorporation of constraints in the model, and Savageau [13]. These biosynthetic pathways differ only in
(2) provide evidence for the generalization of compari- the reversibility of the first step.
sons, and (3) provide additional insight into the effects of
the selected constraints on our interpretation of the
results. The enhancements also address two concerns with
the method as presently applied. First, the current
approach requires that we identify a number of con- erty is equal in both systems adds no information to the
straints sufficient to numerically fix all free parameters. An comparison. Unlike the epidemiological study, a
objective of our approach is to relax this requirement for controlled comparison requires us to identify constraints
cases where the identification of a sufficient number of sufficient to eliminate all of the free parameters in the
constraints is not practical or not desired. Second, the alternative. If we cannot identify a sufficient number of
enhanced approach incorporates a step that excludes the constraints with meaningful interpretations, we may be
use of unrealistic alternatives resulting from the applica- forced to select constraints for mathematical convenience.
tion of constraints. Since our observations are conditioned on the constraints
imposed in the analysis, the choice of mathematically
The existing method currently requires the identification convenient constraints may lead to complications in
of enough constraints to remove all degrees of freedom interpreting the results.
associated with parameters of the alternative model not
fixed by internal equivalence. The construction of an The application of constraints in forming instances of the
alternative pair for a given reference in a controlled com- alternative design has the potential of producing systems
parison is similar to the process of matching in an epide- that are unreasonable with respect to their parameter val-
miological study in that both attempt to prevent ues and thus alternative systems constructed through the
confounding by restricting comparisons to pairs that have application of these constraints must be evaluated for rea-
been matched on the confounding variable. The key dif- sonableness. Clearly, these parameter values are related to
ference is that in an epidemiological study cases and con- the kinetic parameters of the underlying biological
trols or treatment groups are drawn from the sample process and thus are expected to fall within ranges repre-
population and then matched whereas in a controlled sentative of the physical limits of the modeled process. In
comparison the reference is drawn and the alternative is some cases, the application of constraints can yield alter-
constructed from the reference to enforce the match. In natives with parameter values far from those expected in a
both cases we become unable to make statements with realizable system. Unlike the epidemiological study, the
regard to differences in the systemic properties (con- alternative is constructed so as to satisfy the given con-
founding variable) that we have matched on. Since both straints without concern for the reasonableness of the
the reference and alternative system were matched at a alternative. Under these conditions, we might mistakenly
constraint of our choosing the observation that the compare a reference that matches our prior belief about
matched property or any function of the matched prop- realistic parameter ranges to an unrealistic alternative. In
Page 4 of 18
the existing approach, there is no explicit evaluation of the alizability or robustness of statements made when point
likelihood or reasonableness of an alternative formed estimates for these parameter values are used in a control-
from a given reference. Constraints on resulting kinetic led comparison. Consider, for example, the immune
orders have been imposed in some previous applications response model described in [8,21]. The referenced study
of controlled comparisons [14,17] but the step has not compares the functional effectiveness of systems with and
been applied in methods using statistical extensions. Con- without suppressor lymphocyte regulation of effector
sider, for example, the analysis of irreversible step posi- lymphocyte production (Figure 1). Antigen and effector
tions in unbranched biosynthetic pathways presented in step responses to a four-fold increase in systemic antigen
[13]. The structure of the reference and alternative are were included as functional effectiveness measures in this
illustrated in Figure 3. As part of the numerical compari- study. The authors developed time courses for both the
sons, parameter values for kinetic orders and rate con- reference (with suppression) and alternative system
stants were drawn from uniform and log-uniform (without suppression) for a specific set of kinetic orders
distributions respectively. Kinetic orders were drawn from and rate constants determined to be reasonable based on
Unif(0,5) for positive or Unif(-5,0) for negative kinetic prior knowledge of the system being studied. They com-
orders and log (base 10) rate constants were drawn from pared time courses and concluded that the system with
Unif(-5,5). Constraint relationships were applied and ref- suppression was superior to one without suppression
erence models with irreversible steps at each position were with respect to the peak antigen and effector levels in
constructed. We repeated the described sampling process response to the step challenge. We repeated their calcula-
and constructed 4-step alternatives with an irreversible tions and reproduce the time courses in Figure 4A. As they
reaction at the first step. The following parameter values observed, the peak levels are lower in the reference system.
were drawn for one of the reference systems in our Next we examined the step response for models drawn
sampling: from a narrow neighborhood about the selected parame-
ters and found that the conclusion does not hold in gen-
eral. Figure 4B illustrates the step response for one such
, = −1.3865 g2,2 = −3.8822
g11 g3,3 = −3.5399 g4,4 = −3.0146 case. We see that for this case the system without suppres-
g1,0 = 0.4422 g2,1 = 1.1487 g3,2 = 2.4753 g4,3 = 0.1503
(2) sion is superior with respect to peak effector level. The
g5,4 = 3.2397 g1,4 = −1.8842 g5,5 = 0.4619 α1 = 1.0103 × 10−2
α 2 = 473.31 α 3 = 6.8966 × 10−2 α 4 = 8.2053 × 103 α5 = 1
analysis described by Irvine and Savageau, which however
preceded the extensions of Alves and Savageau by 15
Applying the constraints from [13] yields the following years, requires statistical methods to fully explore the reg-
alternative: ulatory preferences of the immune system. We provide
this example as reinforcement to the recommendations of
, =0
g11 g2,2 = −3.8822 g3,3 = −3.5399 g4,4 = −3.0146 Alves and Savageau and for reference as we repeat the
g1,0 = 0.4422 g2,1 = 1.1487 g3,2 = 2.4753 g4,3 = 0.1503
(3) comparison of regulatory preferences in the immune sys-
g5,4 = 3.2397 g1,4 = −290.7305 g5,5 = 0.4619 α1 = 4.8260 × 10174
α 2 = 473.31 α 3 = 6.8966 × 10 −2
α 4 = 8.2053 × 10 3
α5 = 1
tem model in the sections that follow.
As required by the defined constraints, the steady-state Methods

values, log gains with respect to supply, and sensitivity Below we describe the proposed enhancement to the
with respect to α1 are equivalent in the reference and alter- method of mathematically controlled comparisons. We
native. However, the application of these constraints set the following requirements in the development of this
resulted in a kinetic order (g1,4 = -290.7) and a rate con- method. (1) In the limit, as the alternative is forced to be
stant (α1 = 4.8 × 10174) that are well beyond the range of fully equivalent, the conclusions of the improved method
reasonable values. Since our prior belief is that kinetic must match those of the currently defined method for
orders should have magnitudes less than 5, this finding cases in which the current method provides unambiguous
gives rise to concern that the sampled reference is being conclusions and the alternatives are reasonable with
compared to an unrealistic alternative in the cases studied. respect to our prior knowledge of the parameter ranges.
We therefore recommend that references resulting in (2) The improved method should allow for various levels
unrealistic alternatives be eliminated from consideration of equivalence ranging from alternatives independent of
in statistical comparisons and that the rate of occurrence the reference to alternatives that are both internally and
of unrealistic alternatives be evaluated as part of the externally equivalent to the reference. (3) The improved
method. method must avoid comparisons of unreasonable alterna-
tives. (4) Finally, the improved method must provide a
In most cases, a parameterized model, defined by its statistically meaningful measure comparable across vari-
parameter values and implied structure, is but a sample ous levels of equivalence and must allow for a test of
from a population of models that might all represent the homogeneity of conclusions across those levels. The sta-
given design. In these cases one must question the gener- tistical model and the procedure for implementation of
Page 5 of 18
A
Step Response (Nominal) Step Response (Nominal)
200 7
Effector Concentration
Antigen Concentration
150
5
100 4
3
50
2
0 1
0 2 4 6 8 10 0 2 4 6 8 10
Time Time
B
Step Response (Group 8) Step Response (Group 8)
50 4.5
4
40
Effector Concentration
Antigen Concentration
3.5
30 3
20 2.5
2
10
1.5
0 1
0 2 4 6 8 10 0 2 4 6 8 10
Time Time
Figure
Step responses
4 to source antigen increase
Step responses to source antigen increase. Step responses to a four-fold increase in source antigen are presented for
both the nominal values (panel A) (from Irvine and Savageau) and for a case in which the values were drawn from a narrow dis-
tribution about those nominal values (panel B). Systemic antigen responses are shown on the left and effector on the right.
Solid lines indicate the response for the reference system (with suppression) and dashed lines are used for the alternative. Step
responses for the nominal values indicate a preference for the system with suppression. The step responses for the sampled
case indicate a preference for the alternative when considering dynamic peaks for effector concentration.
the method are described below. An example of its appli- respect to that outcome. In epidemiological studies, these
cation is given in the Results section. comparisons are supported by the methods of categorical
data analysis where observations are separated into
Statistical Methods for Comparison of Alternatives groups based on common traits (reference and alternative
As described above, a controlled comparison under the in this study). Categorical data analysis has a strong theo-
extensions of Alves and Savageau is similar to a prospec- retical basis, has been applied extensively, provides mean-
tive study in epidemiology. In both cases we sample from ingful measures of preference in the form of odds or odds
a population, construct comparison groups, observe the ratios, and allows for the assessment of statistical signifi-
frequency of outcomes for a given measure of effective- cance in those measures. For these reasons we have cho-
ness, and estimate a relative magnitude of effect that indi- sen to employ the methods of categorical data analysis in
cates the preference for one group over the other with performing controlled comparisons [22].
Page 6 of 18
We begin by defining the categories of observations however, directly measure the true merit of the design and
important to our analysis. In this analysis we wish to com- must infer it from the measurement of M for samples
pare a reference design to an alternative design at K levels from a population of instances that represent the design.
of equivalence. Each level of equivalence defines a set of We compute M for many instances of the reference design
constraints on the alternative that make it equivalent to and its associated alternatives and compare those results
the reference with respect to one or more properties. There to determine preference for one design over the other.
are, therefore, K + 1 comparison groups in this analysis Estimation of these preferences requires us to define an
where the first group includes all instances of the reference outcome that indicates the direction of preference. We can
design and the k + 1st group contains all instances of alter- either independently compare the effectiveness measures
native designs at equivalence level k. Instances of alterna- for each instance to a common threshold and represent
tive designs at level k are equivalent to their paired the resulting frequency of occurrences as an odds ratio or
references with respect to the same set of constraints. we can perform pairwise comparisons of each reference
Although not a requirement of the method, we generally and its paired alternative and measure the frequency of
order the application of constraints to form increasing lev- occurrence as an odds. Each method has its advantages.
els of equivalence. At the lowest level, the model parame- Consider a comparison in which the reference is always
ters of the alternative design instance and those of the better than the alternative but only by an infinitesimally
paired reference are independent. The reference and the small amount. In the first approach we would probably
alternative share only the values of the independent vari- detect no difference between the two designs because
ables and thus are subjected to the same external when compared to a common threshold both groups
environment. The next level constrains the alternative would demonstrate about the same odds (an odds ratio of
instance to be internally equivalent to its paired reference 1) of exceeding the threshold. In the second approach we
in addition to sharing common values for the independ- would find the odds of preferring the reference design to
ent variables. Increased levels of equivalence successively be infinite as it is always better than the alternative even
apply constraints eventually resulting in full external though only infinitesimally so. As with most applications
equivalence, the highest level of equivalence. The number of statistics, the key to the appropriate choice is in the
of constraints applied determines the number of levels of question to be answered. For applications of controlled
equivalence and thus the number of comparison groups. comparisons we recommend inclusion of both methods
of comparison as they provide both a measure of the mag-
Applying constraints in the construction of alternatives nitude of the difference and allow us to detect strict but
causes the alternative to be statistically dependent on the small differences that may have biological significance.
paired reference because its parameters are determined
from those of the reference and they share a common set Method 1
of values for the independent variables. When comparing Let W be a threshold such that systems for which M >W
the alternative and reference designs we must, in our sta- are taken to be part of a functionally desirable class. Mem-
tistical model, account for systematically high or low bership in this desirable class is therefore represented by a
functional effectiveness resulting from this dependence. dichotomous variable given by the outcome of such a test.
As such we define a second dimension of grouping to We formally define this as follows.
account for this effect. An instance of the reference design
and all alternative instances derived from that reference
are considered to be part of a matched group. If we sample
1 if M Sk , j > W
Yk , j = 
( ) (4)
J instances of the reference design and construct K alterna-  0 otherwise
tives from each reference instance we generate a popula-
tion of J·(K + 1) samples in J matched groups. The All alternatives in the same group j are derived from the
resulting set of instances can then be viewed as being part same reference instance, S1,j, and therefore the Yk,j within
of a J by K + 1 table where the K + 1 columns associated a matched group are correlated. We wish to compare the
with the comparison groups and the J rows with the odds of an instance of the reference design being a mem-
matched groups. We label a sample with the indices of ber of the desirable class to the odds of an instance of the
this table, thus Sk + 1,j is an instance of the alternative alternative design, at equivalence level k. The following
design at the kth equivalence level derived from the jth ref- log-linear model is used.
erence instance and S1,j is that paired reference instance.
J
Let M(Sk,j) be a measure that can be determined from the

( )
logit pk , j = θ1 x1, j + θ 2 x2, j + θ 3 x3, j + ... + θ K xK , j + ∑ γ qω q , j
q =1 (5)
reference and alternative instances' parameter values and (
pk , j = Pr Yk , j = 1 )
that orders their functional effectiveness. This measure is
taken to represent the true merit of the design. We cannot, where
Page 7 of 18
• Yk,j is the outcome for Sk,j (1 = member of the desirable tribution is assumed for F(·), the linear model is equiva-
class, 0 = not a member of the desirable class) with respect lent to the Bradley-Terry Model for paired comparisons
to M and threshold W, (see description in [23]). The Bradley-Terry model is most
often associated with analysis of orderings of objects in
• Xk,j are indicators taking value 1 if the instance is an alter- paired comparisons such as paired competitions in sports
native at equivalence level k formed from the jth reference or in subjective pairwise comparisons like wine tasting. In
instance. our application we compare, pairwise, the reference
design to several alternative designs under various levels
• ωq,j are a collection of J indicator variables where ωq,j of equivalence. Each new reference and its associated
takes value 1 if q = j and 0 otherwise. alternative instances yields a new set of observations from
matched comparisons of computed measures of effective-
The parameters (θk) are estimated by conditioning out the ness. Currently we consider only one reference and one
nuisance variables (γq) using conditional logistic regres- alternative design under various levels of equivalence. We
sion. The exp(θk) then give the odds ratios for desirable can, however, extend the model to include multiple
class membership comparing alternative structure at designs which could be compared simultaneously. Such a
equivalence level k to the reference structure after model would be useful in Alves and Savageau's study of
controlling for group effects. The methods of categorical preferred irreversible step positions in biosynthetic path-
data analysis and logistic regression are described in many ways [13]. Each possible irreversible step location could
texts on statistics, for example [22]. be included as another alternative in the statistical model.
For our purposes, we continue with the model comparing
This method allows us to address structural preference two designs which we describe as follows:
with respect to M by independently comparing both the
population of reference systems and the population of
alternative systems to a common threshold to determine
(
π1,k = F M ( S1,. ) − M ( Sk ,. ) ) (7)
odds of membership in the desirable class after control- logit ( π1,k ) = βR xR − β A x A + γ1e1 + " + γ K e K
ling for group effects. The odds of membership for the ref- where
erence are compared to the odds for the alternative in the
odds ratios estimated in the regression. Ratios found to be xR is an indicator variable taking value 1 if the reference is
significantly different from 1 indicate a preference with used in the comparison (always 1).
respect to measure M. For this method to be applied we
must choose threshold W. For consistency of comparison xA is an indicator variable taking value 1 if the alternative
with Method 2 we choose W to be the median of the is used in the comparison (always 1).
observed values of M for instances of the reference.
Although this selection for W is somewhat arbitrary, it has ek are indicator variables taking value 1 if the comparison
the desirable effect of making the odds of class member- is being made at equivalence level k.
ship for the reference system equal to 1.
The indicators, ek, representing the equivalence levels of
Method 2 the comparisons are treated as covariates in the model.
The method above provides us with a comparison of the The indicators xR and xA take fixed values for our example
alternative design and reference design based on a com- as we are comparing only two designs. A more general
mon threshold test. In Method 2 we perform a pairwise form of the model can be constructed to compare several
comparison of each alternative design instance and its design alternatives. For the reference instance and each
paired reference and compute the odds that the reference paired alternative instance we compute the effectiveness
is better than its paired alternative with respect to the measure M(·). We perform pairwise comparisons
measure of comparison. For this assessment we consider between the reference and each associated alternative to
the general linear model for paired comparison [23]. yield K outcomes per group and the data is then fit by
Under this model the probability that design Di is pre- logistic regression (without intercept). Under the given
ferred over design Dj is then given by parameterization, the design matrix does not have full
rank and so we employ the constraint βR - βA = 0. In this
πi,j = F(M(Di) - M(Dj)) (6) way, the regression parameter γk gives the log odds of pref-
erence for the reference versus the alternative at the kth
Where F(·) represents a symmetric cumulative distribu- level of equivalence. Performing pairwise comparisons
tion function centered at 0, M measures the true merit of only within matched groups eliminates within group
the design, and πi,j is the probability that Di is preferred dependencies. This method allows us to detect a prefer-
over Dj with respect to measure M. When the logistic dis- ence for the reference (or alternative) independent of the
Page 8 of 18
magnitude of the difference as measured by M as it specific logarithmic gains [13-15,20,21], combinations of

depends only on the frequency with which the effective- logarithmic gains [21], or specific sensitivities [13] in the
ness of a reference exceeds that of a paired alternative. definition of constraints. For each constraint, we identify
a relationship that fixes remaining free parameters in
Procedure for Controlled Comparisons terms of the parameters of the paired reference instance.
This section provides a step-by-step procedure for control- Constraint relationships are determined using symbolic
led comparisons under the proposed enhancements. Pri- steady-state solutions developed with a computer algebra
mary differences between the enhanced method and prior system such as the Matlab Symbolic Toolbox. For this
applications of controlled comparisons occur in steps 6 study we have employed BSTLab, a Matlab toolbox capa-
through 10. ble of developing symbolic solutions for S-system steady
states, sensitivities, and logarithmic gains [24].
Step 1 – Model Development
Using the chosen mathematical framework we develop a Step 5 – Sampling of the Reference Design's Population
mathematical model for the designs being compared, We construct an instance of a reference design by sam-
identifying each of the dependent and independent varia- pling model parameter values from the distributions
bles and the differential equations describing the behavior defined in Step 3. The model structure and the sampled
of the dependent variables. The mathematical parameters fully define one instance of the reference sys-
representation is derived from the biochemical map of the tem. For this study we sampled 1,000 reference design
system under study using the procedures described in [9]. instances for the main results and an additional 5,000
We identify the parameters associated with the process or instances to confirm some of our findings.
step of interest and identify the parameters fixed by the
definition of the alternative (e.g., fixing a kinetic order at Step 6 – Construction of Alternatives
0 for an influence we wish to eliminate in the alternative). For each sampled reference design we construct one or
more alternatives by applying the constraints identified in
Step 2 – Identification of Functional Effectiveness Measures Step 4. We first construct an independent alternative by
Based on our knowledge of the system's function we iden- sampling parameters from the distributions defined in
tify functional effectiveness measures. This step is depend- Step 3 followed by the application of constraints on the
ent on the system under study. Previous studies have parameters that are fixed by the alternative design's struc-
employed measures of margin of stability [13,14,17], sen- ture. We then construct additional alternatives by the
sitivity [14,17,21], aggregated sensitivity [13], logarithmic application of constraints starting with internal equiva-
gains [13-15,21], response time [13,14,20,21], and step lence and ending with full (internal and external) equiva-
response overshoot [21]. These measures are computed lence. The parameters computed through the application
through either steady-state or dynamic analysis using the of constraints in the alternative are then checked against
mathematical framework. the range of reasonable parameter values. Sampled refer-
ences and associated alternatives are discarded when any
Step 3 – Determination of Sampling Space of their parameters exceed the range of reasonable values.
We identify distributions representing our prior knowl- Steps 5 and 6 are repeated until the desired sample size is
edge for each of the parameters. These sampling distribu- achieved.
tions represent the population of models being studied.
The sampling space is chosen based on estimated variabil- Step 7 – Evaluation of Functional Effectiveness
ity in the model parameters (based on regression results) Functional effectiveness measures, identified in Step 2, are
or on uncertainty in our prior opinion about the parame- computed for instances of the reference and associated
ters. In cases where the parameter value distributions are alternatives. Alternatives and references are compared to
not known, a uniform distribution is employed. All con- the common threshold (for Method 1) and each alterna-
clusions of the analysis are conditioned on the chosen tive is compared to its associated reference (for Method 2)
sampling space. with respect to each measure. For Method 1 a binary out-
come is recorded for each instance and effectiveness meas-
Step 4 – Identification of Constraints ure and the outcomes for Method 2 are recorded as
We identify constraints that reduce the differences in the categorical values indicating that the reference is better
reference and alternative design instances. These con- than, equal to, or worse than the alternative with respect
straints are defined in terms of steady-state systemic prop- to the given performance measure.
erties that can be computed from the mathematical model
(steady-state values of dependent variables, logarithmic Step 8 – Analysis of Outcomes
gains, sensitivities, etc.). Previous studies have employed We analyze the outcomes for each case using conditional
steady-state values of dependent variables [13-15,20,21], logistic regression (for Method 1) or logistic regression
Page 9 of 18
(for Method 2). The estimated parameters for the regres-

sion model can then be interpreted as odds ratios (com-
X 1 = α1 X1 1,1 X41,4 − β1 X11,1 X21,2
g g h h
parison to a common threshold) or odds (paired
X 2 = α 2 X1 2,1 X32,3 X5 2,5 − β 2 X22,2
comparisons) for preference of the reference system over g g g h
(8)
the alternative given a specified level of equivalence. The
X 3 = α 3 X23,2 X6 3,6 − β 3 X33,3
g g h
analysis also provides confidence intervals on these
parameters allowing us to measure the significance of our
statements with respect to the given sampling of the refer- In this model all of the gi,j and hi,j are greater than 0 except
ence design population. We perform this analysis using a for g2,3 which takes values less than 0 in the reference
statistical computing system such as R [25]. design and is fixed equal to 0 in the alternative.
Step 9 – Identification of Significant Differences To facilitate comparison with the results of Irvine and Sav-
Odds or odds ratios found to be statistically significant ageau, we select the same seven functional performance
indicate differences between the reference and alternative measures. The first two performance measures are the
populations. Odds or odds ratios that are not significantly basal levels of systemic antigen and effector lymphocytes,
different from the null value of 1 are taken as an determined by the steady-state values of X1 and X2. We
indication that in this sampling there is no evidence of a also include the antigenic gain and the effector gain deter-
difference between the reference and alternative design mined by L1,4 and L2,4. Dynamic analysis yields two more
with respect to the given performance measure at the measures given by the magnitude of the overshoot of sys-
given level of equivalence. The ability to detect small dif- temic antigen and effector lymphocytes in response to a
ferences in preference depends on the size of the sample four-fold step increase in source antigen. These values are
used in the analysis. In these studies we have taken determined by integrating the system of equations for
between 1,000 and 5,000 randomly constructed groups each case, initially at steady state, in response to the four-
(one reference and one or more alternatives). We summa- fold increase in source antigen. The difference between
rize these data in the form of analysis tables giving the the peak value of the time course and the new steady state
odds and odds ratios for these comparisons along with as a fraction of the new steady-state value are taken as the
indications of significance and indications of those meas- functional performance measure. Finally we include the
ures fixed by equivalence. sensitivity of the logarithmic gain L1,4 with respect to
parameter h2,2 as a measure of the system sensitivity with
Step 10 – Generalization of Differences respect to parameter variation (S(L1,4, h2,2)). In all cases,
We next examine the homogeneity of conclusions across lower values indicate a more desirable design. The ration-
the levels of equivalence with respect to the direction of ale for the selection of these measures is given in [21].
the effect and with respect to magnitude. Where statisti-
cally meaningful differences are required, contrasts on the Values for each of the parameters are sampled in a neigh-
regression parameters are computed. borhood about the parameter values given in [8] from the
following distributions.
Results
We illustrate the proposed enhancements by repeating the
analysis of network regulation in the immune system per-
(
log10 ( α1 ) ~ N log10 ( 2.02 ) , σ2 )
formed by Irvine and Savageau [21] and summarized in log10 ( β1 ) ~ N ( log10 ( 2 ) , σ )
2
[8]. In particular, we focus on their comparison of systems

that include suppression of effector lymphocyte produc-
(
log10 ( α 2 ) , log10 ( β2 ) , log10 ( α 3 ) ~ N log10 ( 1 ) , σ2 )
β3 = 1
tion and those that do not. The schematic representations ( 9)
of the reference design (with suppression) and the alterna- (
g1,4 , g2,5 , g3,6 , h1,1 , h2,2 , h3,3 ~ N 1, σ2 )
tive (without suppression) are given in Figure 1. The only
difference in the designs occurs in the step associated with
(
, ~ N 0.9, σ
g11 2
)
the production of effector lymphocytes where, in the g2,3 ~ N ( −0.5, σ2 )
g2,1 , g3,2 , h1,2 ~ N ( 0.5, σ2 )

reference design, the production is inhibited by the con-
centration of suppressor lymphocytes. Using the proce-
dures in [9] the system of equations is written as follows. The rate constant β3 is fixed to set the time scale. When
sampling instances of the alternative design, the value of
g2,3 is set to 0. The distributions for kinetic orders are trun-
cated to prevent positive kinetic orders less than 0.1 and
negative kinetic orders greater than -0.1. The values of the
Page 10 of 18
independent variables X4, X5, and X6 are sampled from the four levels of equivalence. Fourteen cases were identified
following distributions. having values of g2,5 less than 0.1 (the lower bound on our
accepted range for g2,5). Two of those 14 cases had values
log10(X4), log10(X5), log10(X6) ~ N(0, σ2) (10) of g2,5 less than 0 demonstrating that application of the
constraints can lead to a change in the alternative design
The analysis is conducted at σ = 0.1 for baseline results (an activation becomes and inhibition) unless checks for
and again at σ = 0.2 to address sensitivity with respect to reasonableness of constrained parameters are employed.
the sampling space. Groups were also checked for the existence of stable
steady states. A total of 160 groups were eliminated
We define four levels of equivalence for this analysis. The because of unreasonable parameters or because one or
lowest level requires only that the alternative instance more cases in the group did not exhibit a stable steady
adhere to the sampling distributions for parameters state at either the baseline conditions (X4, X5, and X6) or
described above. Instances drawn as such are representa- at the steady state for a four-fold increase in source anti-
tives of the alternative design but do not exhibit internal gen. Summary statistics were computed for each of the
or external equivalence with the paired reference system. constrained parameters (Table 1). Eliminating the groups
The only values shared between the alternative and the identified above resulted in 4200 cases in 840 groups that
paired reference are those of the independent variables. were further analyzed.
We next define an internally equivalent alternative as one
in which the values for α1, α3, β1, β2, β3, g1,1, g1,4, g3,2, g3,6, Functional performance measures were computed for
h1,1, h1,2, h2,2, and h3,3 in the alternative are equal to the each case and analyzed according to both Method 1 and
associated values in the paired reference. We define a par- Method 2 using the statistical package R [25]. Density
tially equivalent alternative as one that is both internally plots for each of the effectiveness measures and associated
equivalent and that also has steady-state values of medians for the reference design are given in Figure 5. For
dependent variables equal to those of the paired reference. Method 1, functional effectiveness measures for all cases
This constraint is used to fix the value of α2 in the alterna- were compared to the median over all reference cases. The
tive instance. Symbolic solutions for the steady-state val- outcome was defined such that odds ratios greater than 1
ues of the dependent variables are computed for both the indicate a preference for the reference design. For Method
reference and alternative, expressions for X2 are set equal 2, functional effectiveness measures for each case were
and solved for α2. To construct a partially equivalent alter- compared to the functional effectiveness measures of the
native we form an internally equivalent alternative and paired reference. Cases in which the performance meas-
additionally compute the value of α2 to satisfy the con- ures were equal were eliminated prior to the regression as
straint. The fourth level of equivalence, referred to as full they contribute no information to an estimate of prefer-
equivalence, incorporates constraints sufficient to remove ence. Outcomes were again defined such that odds greater
all remaining degrees of freedom in the formation of the than 1 indicate a preference for the reference design. The
alternative. The fully equivalent form is constructed by conditional logistic regressions for Method 1 were com-
forcing two additional constraints which fix the values of puted using the clogit function and the logistic regressions
g2,1 and g2,5. The value for g2,1 is determined by requiring for Method 2 were computed using glm with binomial
that L1,4 be equal in both the reference and alternative sys- family and logit link. Regression parameters were
tems and the value for g2,5 is determined by requiring that exponentiated to yield odds ratios (Method 1) and odds
L1,5 + L1,6 be equal in both systems. Symbolic solutions for (Method 2). Results with σ = 0.1 for Method 1 are given
the log gains are computed, set equal and solved for the in Table 2 and for Method 2 in Table 3. Comparisons that
desired parameter. Fully equivalent systems are then were fixed by constraint are indicated in the table. Regres-
formed by forcing internal equivalence and by fixing α2, sion parameters were considered to indicate a preference
g2,1, and g2,5 to satisfy the given constraints. The selected if they were determined to be statistically different from
constraints match those employed by Irvine and Savageau the null value at the α = 0.05 level. The process was
[21]. repeated for σ = 0.2 to examine sensitivity with respect to
the sampling distributions. Those results are given in
Groups of cases were constructed by sampling the param- Tables 4 and 5.
eter distributions defined above. For each group we drew
one set of values for the parameters of the reference, one We find that the first two measures (basal levels of antigen
set of values for the independent variables, and for each and effector lymphocytes) show no statistically significant
level of equivalence we drew a set of values for those difference between the reference and alternative design
parameters not fixed by the associated constraints. Five even when the reference and alternative instances are
cases were constructed for each of 1,000 groups consisting independent. This finding holds for both methods of
of one reference and four alternatives, one for each of the analysis. Examining the pairwise scatter plots for these
Page 11 of 18
Table 1: Summary statistics for model parameters. Summary statistics for all of the parameters fixed by equivalence were computed
across the three alternatives for which equivalence constraints were applied after elimination of cases which violated the requirement
for steady state or lower bounds for g2,1 or g2,5.
Parameter
Property α2 g2,1 g2,5
Minimum 0.065 0.137 0.100

1st Quartile 0.830 0.395 0.501
Median 1.001 0.468 0.928
Mean 1.074 0.470 0.804
3rd Quartile 1.203 0.544 1.032
Maximum 22.965 0.843 1.398
Standard Deviation 0.682 0.107 0.300
Dynamic Effector Dynamic Antigen Sensitivity Effector Gain Antigenic Gain Effector Systemic Antigen
Density Density Density Density Density Density Density
0 2 4 6 0.0 1.0 2.0 0.0 1.0 2.0 0.0 0.4 0.8 0.00 0.20 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Reference
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
6
0 2 4 6 0.0 0.4 0.8 0.0 1.0 0.0 0.4 0.8 0.0 0.2 0.4 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
Independent
0 1 2 3 4 5
Alternative
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
0 2 4 6 0.0 0.3 0.6 0.0 1.0 0.0 0.4 0.8 0.0 0.2 0.4 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Equivalent
Internally
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
0 2 4 6 0.0 0.4 0.0 1.0 0.0 0.4 0.8 0.0 0.2 0.4 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Equivalent
Partially
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
0.8
4
2.0
2.0
2.0
0 4 8 0.0 0.4 0.8 0.0 1.0 0.0 0.4 0.8 0.00 0.20 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Equivalent
S(L14, h22)
Fully
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
Figure 5
Performance measure distributions
Performance measure distributions. Smoothed histograms were computed using R (density function) and are presented
for each of the 7 functional performance measures for each of the equivalence levels. The dashed vertical line indicates the
median of the distribution for the reference system. The alternative is preferred when more of its probability mass is distrib-
uted to the left of the dashed line.
Page 12 of 18
Table 2: Odds ratios for comparison using Method 1. Odds ratios for preference of system with suppression using conditional logistic
regression (Method 1). The data consisted of 840 groups of 5 cases each (reference and four alternatives) sampled as described. Values
shown give the odds that the reference system is preferred over the alternative. Values marked with asterisks are significant at the α =
0.05 level. Kinetic orders and rate constants were sampled with σ = 0.1. Cells marked with an odds ratio of 1 or = indicate cases that
were fixed by equivalence.
Functional Independent Internally Partially Equivalent Fully Equivalent Irvine and

Performance Alternative Equivalent Savageau
Objective
Minimize Basal Level 0.76 0.94 1 1 =

of Systemic Antigen
of Effector
Lymphocytes
Minimize Antigenic 0.34* 0.39* 0.36* 1 =
Gain
Minimize Effector 1.4* 1.5* 1.8* 1 =
Gain
Minimize Dynamic 32* 28* 34* 11* +
Levels of Antigen
Minimize Dynamic 2.3* 1.9* 2.4* 0.97 +
Levels of Effector
Minimize Sensitivity to 11* 12* 13* 5.5* +
Parameter Variation
Table 3: Odds ratios for comparison using Method 2. Odds for preference of system with suppression using paired comparisons and
logistic regression (Method 2). The data consisted of 840 groups of 5 cases each (reference and four alternatives) sampled as described.
Values shown give the odds that the reference system is preferred over its paired alternative. Values marked with asterisks are
significant at the α = 0.05 level. Kinetic orders and rate constants were sampled with σ = 0.1. Cells marked with an odds ratio of 1 or =
indicate cases that were fixed by equivalence. † Odds of ∞ shown for measures were reference was found to be better than the
alternative in every case sampled.

Objective

of Systemic Antigen
of Effector
Lymphocytes
Minimize Antigenic 0.63* 0.31* 0.25* 1 =
Gain
Gain
Minimize Dynamic 3.9* 69* 55* 55* +
Levels of Antigen
Minimize Dynamic 1.6* 1.9* 2.2* 1.2* +
Levels of Effector
Minimize Sensitivity to 2.2* 22* 21* ∞† +
Parameter Variation
cases (Figure 6) we see little correlation at this level of tistically significant difference for paired comparisons.
equivalence. The introduction of internal equivalence cre- The introduction of partial and full equivalence requires
ates a much stronger correlation between the values but these measures to be the same and thus contributes no
the correlation follows the null line (same basal level in information to the comparison.
both reference and alternative) and thus still shows no sta-
Page 13 of 18
Table 4: Odds ratios using Method 1 with increased sampling variance. Results from sensitivity analysis using increased variance in
sampled population (σ = 0.2 versus σ = 0.1) giving odds ratios for preference of system with suppression using conditional logistic
shown give the odds that the reference system is preferred over the alternative. Values marked with asterisks are significant at the α =
0.05 level. Kinetic orders and rate constants were sampled with σ = 0.2. Cells marked with an odds ratio of 1 or = indicate cases that
were fixed by equivalence.

Objective

of Systemic Antigen
of Effector
Lymphocytes
Minimize Antigenic 1.2 0.61* 0.48* 1 =
Gain
Gain
Levels of Antigen
Levels of Effector
Minimize Sensitivity to 5.4* 5.3* 5.2* 2.5* +
Parameter Variation
Table 5: Odds ratios using Method 2 with increased sampling variance. Results from sensitivity analysis using increased variance in
sampled population (σ = 0.2 versus σ = 0.1) giving odds for preference of system with suppression using paired comparisons and logistic
shown give the odds that the reference system is preferred over its paired alternative. Values marked with asterisks are significant at
the α = 0.05 level. Kinetic orders and rate constants were sampled with σ = 0.2. Cells marked with an odds ratio of 1 or = indicate cases
that were fixed by equivalence. † Odds of ∞ shown for measures were reference was found to be better than the alternative in every
case sampled.

Objective
Minimize Basal Level 1.2 0.82* 1 1 =

of Systemic Antigen
of Effector
Lymphocytes
Minimize Antigenic 0.89 0.59* 0.58* 1 =
Gain
Minimize Effector 1.4 1.5* 1.6* 1 =
Gain
Minimize Dynamic 1.8* 5.8* 6.8* 30* +
Levels of Antigen
Minimize Dynamic 0.98 0.76* 0.69* 0.55* +
Levels of Effector
Minimize Sensitivity to 2.1* 5.5* 6.0* ∞† +
Parameter Variation
Functional effectiveness measures of antigenic gain and than the reference for the independent, internal, and
effector gain show differing preference for the alternative partial equivalence cases. The reference system, however,
and reference designs. The alternative design shows statis- demonstrates statistically significant preference for
tically significant odds of having a lower antigenic gain reduced effector gain. These findings are again consistent
Page 14 of 18
Independent Internally Partially Fully

Alternative Equivalent Equivalent Equivalent
6
5
5
Systemic Antigen
4
X1
X1
X1
X1
3
3
2
2
1
1
0
0
0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6
X1 X1 X1 X1
2.0
2.0
2.0
2.0
1.5
1.5
1.5
1.5
Effector
X2
X2
X2
X2
1.0
1.0
1.0
1.0
0.5
0.5
0.5
0.5
0.0
0.0
0.0
0.0
0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0
X2 X2 X2 X2
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
Sensitivity
S (L14 , h22 )
S (L14 , h22 )
S (L14 , h22 )
S (L14 , h22 )
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8
S(L14 , h22 ) S(L14 , h22 ) S (L14 , h22 ) S (L14 , h22 )
Figure 6comparison of selected performance measures

Pairwise
Pairwise comparison of selected performance measures. Scatter plots of 840 pairs were produced in R and show basal
systemic antigen level (top), basal effector level (middle), and sensitivity (bottom) in the reference (x-axis) versus the same
value in the associated alternative (y-axis) for all equivalence levels. The dashed diagonal line corresponds to the null condition
where the reference and the alternative are equal. The reference is preferred over the alternative when more pairs fall above
the diagonal. System sensitivity shows a clear preference for the reference with all cases preferring the reference under full
equivalence.
across both methods of analysis. Introducing constraints pression being preferred over an instance without sup-
to achieve full equivalence causes these gains to be equal pression with respect to dynamic levels of antigen and
in both the reference and alternative. The introduction of effector are 32 and 2.3 for Method 1 and 3.9 and 1.6 for
constraints on L1,4 and L1,5 + L1,6 hides the differential Method 2. The strong preference for the design with
preference with respect to effector and antigen gain. Anal- suppression is seen at all levels of equivalence for both
ysis using the incremental introduction of constraints per- Method 1 and Method 2 when considering dynamic levels
mits the observance of these differences. of antigen. A significant preference for the design with
suppression is also seen for dynamic levels of effector. The
Dynamic responses for both antigen peak levels and effec- finding under this measure does lose significance under
tor peak levels demonstrate a preference for the reference full equivalence for Method 1. We conclude that the refer-
design. Even for an independent alternative (no internal ence design is better than or equal to the alternative in all
or external equivalence), the odds of an instance with sup- cases.
Page 15 of 18
Sensitivity to parameter variation, as measured by S(L1,4, Discussion

h2,2) demonstrates a clear preference for the design with The primary purpose of this analysis was to demonstrate
suppression. In population level comparisons (Method proposed enhancements to the method of mathemati-
1), the odds of preference for the system with suppression cally controlled comparisons. Through this analysis we
were 11, 12, 13, and 5.5 for the four levels of equivalence. have demonstrated the incremental introduction of con-
In pairwise comparisons (Method 2) the odds for prefer- straints and have identified that in one measure a
ence of the reference were 2.2, 22, 21 for the independent, preference for the alternative was detected consistently
internally equivalent, and partially equivalent cases. across levels of equivalence that would not have been
Under full equivalence the reference was preferred in identified through comparison using the fully equivalent
every case sampled. Clearly, the design with suppression alternative. The conclusions reached in this analysis,
is preferred, independent of our introduction of which qualitatively match those of Irvine and Savageau,
constraints. are more general in that we demonstrate the existence of
preference for the system with suppression with fewer
Overall, the design with suppression is equal to or pre- constraints and that these preferences additionally appear
ferred in six out of the seven functional effectiveness in population-level comparisons to a common threshold
measures and this conclusion can be generalized to the (Method 1). Our use of methods from categorical data
lowest level of equivalence. We also find that these obser- analysis (logistic regression) and odds (or odds ratios) as
vations apply to both population-based comparisons a measure of preference provides a statistically meaningful
against a common threshold and controlled for group method for comparing the results; and estimation of the
effects (Method 1) and paired comparisons (Method 2). confidence interval for those measures provides a method
We conclude that observations of preference for the for assessing the sufficiency of the sampling used in the
system with suppression over the system without suppres- analysis. The compilation of these results in a table of
sion are very general and are conditioned only on the increasing equivalence provides a convenient display of
sampling space used to draw members of the populations the results and facilitates generalization to the fewest set
studied. We find that these observations differ from those of constraints.
of Irvine and Savageau only with respect to the minimiza-
tion of antigenic gain and to some extent in the The analysis also indicated that unreasonable alternatives
minimization of dynamic effector levels. This difference can result from the application of constraints even when
in finding, as determined by antigenic gain, would be sampling in a small region about reasonable nominal val-
expected as Irvine and Savageau employed a fully equiva- ues. By using the S-system based framework, we are rely-
lent alternative in their comparison that was based on a ing on an equivalence between the canonical S-system
constraint on L1,4. Their study would not have been able representation and the biochemical network structure in
to detect a difference in preference based on this measure. this analysis. The application of constraints that result in
Differences in dynamic measures would also be expected kinetic orders that change sign or approach 0 is, therefore,
as the dynamic data depends on the parameters of the sys- equivalent to a change in the structure of the alternative.
tem and a controlled comparison would, as discussed in This would invalidate our assumption that the alternative
previously, require a sampling of the parameter space. constructed from the reference by application of con-
straints results in a representative of the alternative bio-
Analysis of sensitivity with respect to the sampling distri- chemical system design. The incorporation of a step that
butions indicated that in 6 of the 7 measures, the prefer- explicitly checks the reasonableness of constructed alter-
ences held as described above. For dynamic levels of natives eliminates the potential for such problems.
effector, the preference shifted from the system with sup-
pression (reference) to the system without (alternative). Our proposed enhancement to the method of controlled
Under both methods and across all levels of equivalence comparisons provides results consistent with the currently
the preference shifted away from the reference and either defined method for the fully equivalent case, allows for
became insignificant or shifted slightly in favor of the the incremental introduction of constraints and generali-
alternative. To verify these findings we repeated the entire zation of results, eliminates comparison with unrealistic
analysis with a larger starting sample (5000 groups versus alternatives, and provides a consistent measure of
1000 groups, results not shown). The same shift in prefer- preference that can be compared across levels of equiva-
ence was observed for the larger sample size, and we lence. We find that in this specific analysis, the results of
conclude that our observations with respect to this meas- Irvine and Savageau can be generalized and that some dif-
ure should be conditioned on the chosen sample space. ferential preference is observed for antigenic and effector
gains that would not have been observable in their fully
equivalent analysis. In general, we conclude that there are
different levels of confidence at which we might declare
Page 16 of 18
one design better than another and that the assessment of

these differences requires a more complete exploration of ∂ ln X j ∂ X j Xk
the implications of constraints and sample spaces on the (
L X j , Xk = ) ∂ ln Xk
=
∂ Xk X j
conclusions we reach. As we continue to search for design
∂ ln Vi+
( )
n
principles among the many pathways now being studied
we need to fully characterize the contexts in which a pref-
L Vi+ , Xk =
∂ ln Xk
(
= gi ,k + ∑ gi , j L X j , Xk ) (12)
j =1
erence exists. The method of mathematically controlled
∂ ln Vi−
( )
n
comparisons coupled with the canonical nonlinear repre- L Vi− , Xk =
∂ ln Xk
(
= hi ,k + ∑ hi , j L X j , Xk )
sentations of S-systems and well-chosen statistical meth- j =1
ods offers significant potential to facilitate these searches. System sensitivities measure the degree to which changes
in the system parameters (kinetic orders and rate con-
Competing Interests stants) affect the steady state of the system. Sensitivities
None declared. are often used as a measure of the robustness of the
system.
Author's Contributions
JHS developed the approach, implemented the software, ∂ ln Xk ∂ Xk α i
conducted the tests, and prepared the results. EOV (
S Xk ,α j = ) ∂ ln α i
=
∂α i Xk
provided the theoretical framework, identified the test (13)
∂ ln Xk ∂ X k gi , j
case, and supported the evaluation and interpretation of
the results.
(
S X k , gi , j ) = =
∂ ln gi , j ∂ gi , j Xk
Appendix Logarithmic gains and sensitivities used in this analysis

S-systems are systems of ordinary differential equations of were computed using the implicit differentiation method
the following form [26]. described in Chapter 7 of [9].
n+ m n+ m Acknowledgements
dXi g h
= α i ∏ X j i , j − β i ∏ X j i , j for i ∈ 1...n This research was supported under NLM Training Grant T15LM07438
dt j =1 j =1 (E.O. Voit, PI) and NSF-BES Quantitative Systems Biotechnology research
The S-system representation offers a particular advantage grant 0120288 (E.O. Voit, PI).
in that it allows for the determination of the system's
steady state by solution of a system of linear equations.
References
1. Bhalla US: Understanding complex signaling networks
Setting the time derivatives to 0 and taking logs results is through models and metaphors. Prog Biophys Mol Biol 2003,
a linear system of equations from which the log steady- 81:45-65.
2. Savageau MA: Biochemical systems analysis : a study of function and design
state values of the n dependent variables can be deter- in molecular biology Reading, Mass.: Addison-Wesley Pub. Co.
mined from the values of the m independent variables Advanced Book Program; 1976.
and the system parameters [26]. 3. Savageau MA: Design principles for elementary gene circuits:
Elements, methods, and examples. Chaos 2001, 11:142-159.
4. Savageau MA: Biochemical systems analysis. I. Some mathe-
−1 −1 matical properties of the rate law for the component enzy-
yD = AD b− AD AI yI matic reactions. J Theor Biol 1969, 25:365-369.
bi = ln ( β i ) − ln ( α i ) 5. Savageau MA: Biochemical systems analysis. II. The steady-
state solutions for an n-pool system using a power-law
ai , j = gi , j − hi , j (11) approximation. J Theor Biol 1969, 25:370-379.
6. Savageau MA: Biochemical systems analysis. 3. Dynamic solu-
y D,i = ln ( Xi ) for i ∈ 1...n tions using a power-law approximation. J Theor Biol 1970,
26:215-226.
y I ,i = ln ( Xi + n ) for i ∈ 1...m 7. Torres NV, Voit EO: Pathway analysis and optimization in metabolic
engineering New York: Cambridge University Press; 2002.
S-systems additionally provide for the convenient charac- 8. Voit EO: Canonical nonlinear modeling : S-system approach to understand-
terization of systemic performance local to the steady ing complexity New York: Van Nostrand Reinhold; 1991.
9. Voit EO: Computational analysis of biochemical systems: a practical guide
state. Logarithmic gains that represent the change in the for biochemists and molecular biologists New York: Cambridge Univer-
log value of the steady state of a dependent variable or flux sity Press; 2000.
as a result of a change in the log value of an independent 10. Savageau MA: A theory of alternative designs for biochemical
control systems. Biomed Biochim Acta 1985, 44:875-880.
variable are computed as follows [26]. 11. Alves R, Savageau MA: Extending the method of mathemati-
cally controlled comparison to include numerical
comparisons. Bioinformatics 2000, 16:786-798.
12. Alves R, Savageau MA: Comparing systemic properties of
ensembles of biological networks by graphical and statistical
methods. Bioinformatics 2000, 16:527-533.
Page 17 of 18
13. Alves R, Savageau MA: Irreversibility in unbranched pathways:

preferred positions based on regulatory considerations. Bio-
phys J 2001, 80:1174-1185.
14. Wall ME, Hlavacek WS, Savageau MA: Design principles for regu-
lator gene expression in a repressible gene circuit. J Mol Biol
2003, 332:861-876.
15. Hlavacek WS, Savageau MA: Completely uncoupled and per-
fectly coupled gene expression in repressible systems. J Mol
Biol 1997, 266:538-558.
16. Savageau MA: Optimal design of feedback control by
inhibition. J Mol Evol 1974, 4:139-156.
17. Hlavacek WS, Savageau MA: Rules for coupled expression of reg-
ulator and effector genes in inducible circuits. J Mol Biol 1996,
255:121-139.
18. Irvine DH, Savageau MA: Network regulation of the immune
response: modulation of suppressor lymphocytes by alterna-
tive signals including contrasuppression. J Immunol 1985,
134:2117-2130.
19. Alves R, Savageau MA: Effect of overall feedback inhibition in
unbranched biosynthetic pathways. Biophys J 2000,
79:2290-2304.
20. Savageau MA: Alternative designs for a genetic switch: analysis
of switching times using the piecewise power-law
representation. Math Biosci 2002, 180:237-253.
21. Irvine DH, Savageau MA: Network regulation of the immune
response: alternative control points for suppressor modula-
tion of effector lymphocytes. J Immunol 1985, 134:2100-2116.
22. Agresti A: Categorical data analysis New York: Wiley; 1990.
23. David HA: The method of paired comparisons New York,: Hafner Pub.
Co.; 1963.
24. Schwacke JH, Voit EO: BSTLab: A Matlab Toolbox for Bio-
chemical Systems Theory. In: Eleventh International Conference on
Intelligent Systems for Molecular Biology Brisbane, Australia; 2003.
25. Ihaka R, Gentlemen R: R: A Language for data analysis and
graphics. Journal of Computational and Graphical Statistics 1996,
5:299-314.
26. Savageau MA: The behavior of intact biochemical control
systems. Current Topics in Cellular Regulation 1972, 6:63-130.
Publish with Bio Med Central and every

scientist can read your work free of charge
"BioMed Central will be the most significant development for
disseminating the results of biomedical researc h in our lifetime."
Sir Paul Nurse, Cancer Research UK
Your research papers will be:

available free of charge to the entire biomedical community
peer reviewed and published immediately upon acceptance
cited in PubMed and archived on PubMed Central
yours — you keep the copyright
Submit your manuscript here: BioMedcentral

http://www.biomedcentral.com/info/publishing_adv.asp
Page 18 of 18

Schwake & Voit, 2004 (Improved Methods For Mathematically Controlled Comparisons of Biochemical Systems)

Uploaded by

Copyright:

Available Formats

Schwake & Voit, 2004 (Improved Methods For Mathematically Controlled Comparisons of Biochemical Systems)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Schwake & Voit, 2004 (Improved Methods For Mathematically Controlled Comparisons of Biochemical Systems)

Uploaded by

Copyright:

Available Formats

Theoretical Biology and Medical

Modelling BioMed Central

Research Open Access

Published: 04 June 2004 Received: 18 May 2004

Background the benefits of one design over another and to understand

equation of the form given in Equation 1 where X i indi-

The method of mathematically controlled comparison

to be equal to those of the reference for processes other

In many cases the comparison of these functional effec-

median of the relative measure and its variation across the

Rationale for Improvements

As required by the defined constraints, the steady-state Methods

Let M(Sk,j) be a measure that can be determined from the

magnitude of the difference as measured by M as it specific logarithmic gains [13-15,20,21], combinations of

(for Method 2). The estimated parameters for the regres-

[8]. In particular, we focus on their comparison of systems

g2,1 , g3,2 , h1,2 ~ N ( 0.5, σ2 )

Minimum 0.065 0.137 0.100

Density Density Density Density Density Density Density

Density Density Density Density Density Density Density

Density Density Density Density Density Density Density

Functional Independent Internally Partially Equivalent Fully Equivalent Irvine and

Minimize Basal Level 0.76 0.94 1 1 =

Functional Independent Internally Partially Equivalent Fully Equivalent Irvine and

Minimize Basal Level 0.91 0.98 1 1 =

Functional Independent Internally Partially Equivalent Fully Equivalent Irvine and

Minimize Basal Level 1.2 0.87 1 1 =

Functional Independent Internally Partially Equivalent Fully Equivalent Irvine and

Minimize Basal Level 1.2 0.82* 1 1 =

Independent Internally Partially Fully

S(L14 , h22 ) S(L14 , h22 ) S (L14 , h22 ) S (L14 , h22 )

Figure 6comparison of selected performance measures

Sensitivity to parameter variation, as measured by S(L1,4, Discussion

one design better than another and that the assessment of

Appendix Logarithmic gains and sensitivities used in this analysis

13. Alves R, Savageau MA: Irreversibility in unbranched pathways:

Publish with Bio Med Central and every

Your research papers will be:

Submit your manuscript here: BioMedcentral

You might also like

equation of the form given in Equation 1 where X i indi-