Schwake & Voit, 2004 (Improved Methods For Mathematically Controlled Comparisons of Biochemical Systems)
Schwake & Voit, 2004 (Improved Methods For Mathematically Controlled Comparisons of Biochemical Systems)
Schwake & Voit, 2004 (Improved Methods For Mathematically Controlled Comparisons of Biochemical Systems)
Address: Department of Biometry, Bioinformatics, and Epidemiology Medical University of South Carolina 135 Cannon Street, Suite 303
Charleston, SC 29425, U.S.A
Email: John H Schwacke - [email protected]; Eberhard O Voit* - [email protected]
* Corresponding author
Abstract
The method of mathematically controlled comparison provides a structured approach for the
comparison of alternative biochemical pathways with respect to selected functional effectiveness
measures. Under this approach, alternative implementations of a biochemical pathway are modeled
mathematically, forced to be equivalent through the application of selected constraints, and
compared with respect to selected functional effectiveness measures. While the method has been
applied successfully in a variety of studies, we offer recommendations for improvements to the
method that (1) relax requirements for definition of constraints sufficient to remove all degrees of
freedom in forming the equivalent alternative, (2) facilitate generalization of the results thus
avoiding the need to condition those findings on the selected constraints, and (3) provide additional
insights into the effect of selected constraints on the functional effectiveness measures. We present
improvements to the method and related statistical models, apply the method to a previously
conducted comparison of network regulation in the immune system, and compare our results to
those previously reported.
Page 1 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Page 2 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Page 3 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Page 4 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
the existing approach, there is no explicit evaluation of the alizability or robustness of statements made when point
likelihood or reasonableness of an alternative formed estimates for these parameter values are used in a control-
from a given reference. Constraints on resulting kinetic led comparison. Consider, for example, the immune
orders have been imposed in some previous applications response model described in [8,21]. The referenced study
of controlled comparisons [14,17] but the step has not compares the functional effectiveness of systems with and
been applied in methods using statistical extensions. Con- without suppressor lymphocyte regulation of effector
sider, for example, the analysis of irreversible step posi- lymphocyte production (Figure 1). Antigen and effector
tions in unbranched biosynthetic pathways presented in step responses to a four-fold increase in systemic antigen
[13]. The structure of the reference and alternative are were included as functional effectiveness measures in this
illustrated in Figure 3. As part of the numerical compari- study. The authors developed time courses for both the
sons, parameter values for kinetic orders and rate con- reference (with suppression) and alternative system
stants were drawn from uniform and log-uniform (without suppression) for a specific set of kinetic orders
distributions respectively. Kinetic orders were drawn from and rate constants determined to be reasonable based on
Unif(0,5) for positive or Unif(-5,0) for negative kinetic prior knowledge of the system being studied. They com-
orders and log (base 10) rate constants were drawn from pared time courses and concluded that the system with
Unif(-5,5). Constraint relationships were applied and ref- suppression was superior to one without suppression
erence models with irreversible steps at each position were with respect to the peak antigen and effector levels in
constructed. We repeated the described sampling process response to the step challenge. We repeated their calcula-
and constructed 4-step alternatives with an irreversible tions and reproduce the time courses in Figure 4A. As they
reaction at the first step. The following parameter values observed, the peak levels are lower in the reference system.
were drawn for one of the reference systems in our Next we examined the step response for models drawn
sampling: from a narrow neighborhood about the selected parame-
ters and found that the conclusion does not hold in gen-
eral. Figure 4B illustrates the step response for one such
, = −1.3865 g2,2 = −3.8822
g11 g3,3 = −3.5399 g4,4 = −3.0146 case. We see that for this case the system without suppres-
g1,0 = 0.4422 g2,1 = 1.1487 g3,2 = 2.4753 g4,3 = 0.1503
(2) sion is superior with respect to peak effector level. The
g5,4 = 3.2397 g1,4 = −1.8842 g5,5 = 0.4619 α1 = 1.0103 × 10−2
α 2 = 473.31 α 3 = 6.8966 × 10−2 α 4 = 8.2053 × 103 α5 = 1
analysis described by Irvine and Savageau, which however
preceded the extensions of Alves and Savageau by 15
Applying the constraints from [13] yields the following years, requires statistical methods to fully explore the reg-
alternative: ulatory preferences of the immune system. We provide
this example as reinforcement to the recommendations of
, =0
g11 g2,2 = −3.8822 g3,3 = −3.5399 g4,4 = −3.0146 Alves and Savageau and for reference as we repeat the
g1,0 = 0.4422 g2,1 = 1.1487 g3,2 = 2.4753 g4,3 = 0.1503
(3) comparison of regulatory preferences in the immune sys-
g5,4 = 3.2397 g1,4 = −290.7305 g5,5 = 0.4619 α1 = 4.8260 × 10174
α 2 = 473.31 α 3 = 6.8966 × 10 −2
α 4 = 8.2053 × 10 3
α5 = 1
tem model in the sections that follow.
Page 5 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
A
Step Response (Nominal) Step Response (Nominal)
200 7
Effector Concentration
Antigen Concentration
150
5
100 4
3
50
2
0 1
0 2 4 6 8 10 0 2 4 6 8 10
Time Time
B
Step Response (Group 8) Step Response (Group 8)
50 4.5
4
40
Effector Concentration
Antigen Concentration
3.5
30 3
20 2.5
2
10
1.5
0 1
0 2 4 6 8 10 0 2 4 6 8 10
Time Time
Figure
Step responses
4 to source antigen increase
Step responses to source antigen increase. Step responses to a four-fold increase in source antigen are presented for
both the nominal values (panel A) (from Irvine and Savageau) and for a case in which the values were drawn from a narrow dis-
tribution about those nominal values (panel B). Systemic antigen responses are shown on the left and effector on the right.
Solid lines indicate the response for the reference system (with suppression) and dashed lines are used for the alternative. Step
responses for the nominal values indicate a preference for the system with suppression. The step responses for the sampled
case indicate a preference for the alternative when considering dynamic peaks for effector concentration.
the method are described below. An example of its appli- respect to that outcome. In epidemiological studies, these
cation is given in the Results section. comparisons are supported by the methods of categorical
data analysis where observations are separated into
Statistical Methods for Comparison of Alternatives groups based on common traits (reference and alternative
As described above, a controlled comparison under the in this study). Categorical data analysis has a strong theo-
extensions of Alves and Savageau is similar to a prospec- retical basis, has been applied extensively, provides mean-
tive study in epidemiology. In both cases we sample from ingful measures of preference in the form of odds or odds
a population, construct comparison groups, observe the ratios, and allows for the assessment of statistical signifi-
frequency of outcomes for a given measure of effective- cance in those measures. For these reasons we have cho-
ness, and estimate a relative magnitude of effect that indi- sen to employ the methods of categorical data analysis in
cates the preference for one group over the other with performing controlled comparisons [22].
Page 6 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
We begin by defining the categories of observations however, directly measure the true merit of the design and
important to our analysis. In this analysis we wish to com- must infer it from the measurement of M for samples
pare a reference design to an alternative design at K levels from a population of instances that represent the design.
of equivalence. Each level of equivalence defines a set of We compute M for many instances of the reference design
constraints on the alternative that make it equivalent to and its associated alternatives and compare those results
the reference with respect to one or more properties. There to determine preference for one design over the other.
are, therefore, K + 1 comparison groups in this analysis Estimation of these preferences requires us to define an
where the first group includes all instances of the reference outcome that indicates the direction of preference. We can
design and the k + 1st group contains all instances of alter- either independently compare the effectiveness measures
native designs at equivalence level k. Instances of alterna- for each instance to a common threshold and represent
tive designs at level k are equivalent to their paired the resulting frequency of occurrences as an odds ratio or
references with respect to the same set of constraints. we can perform pairwise comparisons of each reference
Although not a requirement of the method, we generally and its paired alternative and measure the frequency of
order the application of constraints to form increasing lev- occurrence as an odds. Each method has its advantages.
els of equivalence. At the lowest level, the model parame- Consider a comparison in which the reference is always
ters of the alternative design instance and those of the better than the alternative but only by an infinitesimally
paired reference are independent. The reference and the small amount. In the first approach we would probably
alternative share only the values of the independent vari- detect no difference between the two designs because
ables and thus are subjected to the same external when compared to a common threshold both groups
environment. The next level constrains the alternative would demonstrate about the same odds (an odds ratio of
instance to be internally equivalent to its paired reference 1) of exceeding the threshold. In the second approach we
in addition to sharing common values for the independ- would find the odds of preferring the reference design to
ent variables. Increased levels of equivalence successively be infinite as it is always better than the alternative even
apply constraints eventually resulting in full external though only infinitesimally so. As with most applications
equivalence, the highest level of equivalence. The number of statistics, the key to the appropriate choice is in the
of constraints applied determines the number of levels of question to be answered. For applications of controlled
equivalence and thus the number of comparison groups. comparisons we recommend inclusion of both methods
of comparison as they provide both a measure of the mag-
Applying constraints in the construction of alternatives nitude of the difference and allow us to detect strict but
causes the alternative to be statistically dependent on the small differences that may have biological significance.
paired reference because its parameters are determined
from those of the reference and they share a common set Method 1
of values for the independent variables. When comparing Let W be a threshold such that systems for which M >W
the alternative and reference designs we must, in our sta- are taken to be part of a functionally desirable class. Mem-
tistical model, account for systematically high or low bership in this desirable class is therefore represented by a
functional effectiveness resulting from this dependence. dichotomous variable given by the outcome of such a test.
As such we define a second dimension of grouping to We formally define this as follows.
account for this effect. An instance of the reference design
and all alternative instances derived from that reference
are considered to be part of a matched group. If we sample
1 if M Sk , j > W
Yk , j =
( ) (4)
J instances of the reference design and construct K alterna- 0 otherwise
tives from each reference instance we generate a popula-
tion of J·(K + 1) samples in J matched groups. The All alternatives in the same group j are derived from the
resulting set of instances can then be viewed as being part same reference instance, S1,j, and therefore the Yk,j within
of a J by K + 1 table where the K + 1 columns associated a matched group are correlated. We wish to compare the
with the comparison groups and the J rows with the odds of an instance of the reference design being a mem-
matched groups. We label a sample with the indices of ber of the desirable class to the odds of an instance of the
this table, thus Sk + 1,j is an instance of the alternative alternative design, at equivalence level k. The following
design at the kth equivalence level derived from the jth ref- log-linear model is used.
erence instance and S1,j is that paired reference instance.
J
Page 7 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
• Yk,j is the outcome for Sk,j (1 = member of the desirable tribution is assumed for F(·), the linear model is equiva-
class, 0 = not a member of the desirable class) with respect lent to the Bradley-Terry Model for paired comparisons
to M and threshold W, (see description in [23]). The Bradley-Terry model is most
often associated with analysis of orderings of objects in
• Xk,j are indicators taking value 1 if the instance is an alter- paired comparisons such as paired competitions in sports
native at equivalence level k formed from the jth reference or in subjective pairwise comparisons like wine tasting. In
instance. our application we compare, pairwise, the reference
design to several alternative designs under various levels
• ωq,j are a collection of J indicator variables where ωq,j of equivalence. Each new reference and its associated
takes value 1 if q = j and 0 otherwise. alternative instances yields a new set of observations from
matched comparisons of computed measures of effective-
The parameters (θk) are estimated by conditioning out the ness. Currently we consider only one reference and one
nuisance variables (γq) using conditional logistic regres- alternative design under various levels of equivalence. We
sion. The exp(θk) then give the odds ratios for desirable can, however, extend the model to include multiple
class membership comparing alternative structure at designs which could be compared simultaneously. Such a
equivalence level k to the reference structure after model would be useful in Alves and Savageau's study of
controlling for group effects. The methods of categorical preferred irreversible step positions in biosynthetic path-
data analysis and logistic regression are described in many ways [13]. Each possible irreversible step location could
texts on statistics, for example [22]. be included as another alternative in the statistical model.
For our purposes, we continue with the model comparing
This method allows us to address structural preference two designs which we describe as follows:
with respect to M by independently comparing both the
population of reference systems and the population of
alternative systems to a common threshold to determine
(
π1,k = F M ( S1,. ) − M ( Sk ,. ) ) (7)
odds of membership in the desirable class after control- logit ( π1,k ) = βR xR − β A x A + γ1e1 + " + γ K e K
ling for group effects. The odds of membership for the ref- where
erence are compared to the odds for the alternative in the
odds ratios estimated in the regression. Ratios found to be xR is an indicator variable taking value 1 if the reference is
significantly different from 1 indicate a preference with used in the comparison (always 1).
respect to measure M. For this method to be applied we
must choose threshold W. For consistency of comparison xA is an indicator variable taking value 1 if the alternative
with Method 2 we choose W to be the median of the is used in the comparison (always 1).
observed values of M for instances of the reference.
Although this selection for W is somewhat arbitrary, it has ek are indicator variables taking value 1 if the comparison
the desirable effect of making the odds of class member- is being made at equivalence level k.
ship for the reference system equal to 1.
The indicators, ek, representing the equivalence levels of
Method 2 the comparisons are treated as covariates in the model.
The method above provides us with a comparison of the The indicators xR and xA take fixed values for our example
alternative design and reference design based on a com- as we are comparing only two designs. A more general
mon threshold test. In Method 2 we perform a pairwise form of the model can be constructed to compare several
comparison of each alternative design instance and its design alternatives. For the reference instance and each
paired reference and compute the odds that the reference paired alternative instance we compute the effectiveness
is better than its paired alternative with respect to the measure M(·). We perform pairwise comparisons
measure of comparison. For this assessment we consider between the reference and each associated alternative to
the general linear model for paired comparison [23]. yield K outcomes per group and the data is then fit by
Under this model the probability that design Di is pre- logistic regression (without intercept). Under the given
ferred over design Dj is then given by parameterization, the design matrix does not have full
rank and so we employ the constraint βR - βA = 0. In this
πi,j = F(M(Di) - M(Dj)) (6) way, the regression parameter γk gives the log odds of pref-
erence for the reference versus the alternative at the kth
Where F(·) represents a symmetric cumulative distribu- level of equivalence. Performing pairwise comparisons
tion function centered at 0, M measures the true merit of only within matched groups eliminates within group
the design, and πi,j is the probability that Di is preferred dependencies. This method allows us to detect a prefer-
over Dj with respect to measure M. When the logistic dis- ence for the reference (or alternative) independent of the
Page 8 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Page 9 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Step 9 – Identification of Significant Differences To facilitate comparison with the results of Irvine and Sav-
Odds or odds ratios found to be statistically significant ageau, we select the same seven functional performance
indicate differences between the reference and alternative measures. The first two performance measures are the
populations. Odds or odds ratios that are not significantly basal levels of systemic antigen and effector lymphocytes,
different from the null value of 1 are taken as an determined by the steady-state values of X1 and X2. We
indication that in this sampling there is no evidence of a also include the antigenic gain and the effector gain deter-
difference between the reference and alternative design mined by L1,4 and L2,4. Dynamic analysis yields two more
with respect to the given performance measure at the measures given by the magnitude of the overshoot of sys-
given level of equivalence. The ability to detect small dif- temic antigen and effector lymphocytes in response to a
ferences in preference depends on the size of the sample four-fold step increase in source antigen. These values are
used in the analysis. In these studies we have taken determined by integrating the system of equations for
between 1,000 and 5,000 randomly constructed groups each case, initially at steady state, in response to the four-
(one reference and one or more alternatives). We summa- fold increase in source antigen. The difference between
rize these data in the form of analysis tables giving the the peak value of the time course and the new steady state
odds and odds ratios for these comparisons along with as a fraction of the new steady-state value are taken as the
indications of significance and indications of those meas- functional performance measure. Finally we include the
ures fixed by equivalence. sensitivity of the logarithmic gain L1,4 with respect to
parameter h2,2 as a measure of the system sensitivity with
Step 10 – Generalization of Differences respect to parameter variation (S(L1,4, h2,2)). In all cases,
We next examine the homogeneity of conclusions across lower values indicate a more desirable design. The ration-
the levels of equivalence with respect to the direction of ale for the selection of these measures is given in [21].
the effect and with respect to magnitude. Where statisti-
cally meaningful differences are required, contrasts on the Values for each of the parameters are sampled in a neigh-
regression parameters are computed. borhood about the parameter values given in [8] from the
following distributions.
Results
We illustrate the proposed enhancements by repeating the
analysis of network regulation in the immune system per-
(
log10 ( α1 ) ~ N log10 ( 2.02 ) , σ2 )
formed by Irvine and Savageau [21] and summarized in log10 ( β1 ) ~ N ( log10 ( 2 ) , σ )
2
Page 10 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
independent variables X4, X5, and X6 are sampled from the four levels of equivalence. Fourteen cases were identified
following distributions. having values of g2,5 less than 0.1 (the lower bound on our
accepted range for g2,5). Two of those 14 cases had values
log10(X4), log10(X5), log10(X6) ~ N(0, σ2) (10) of g2,5 less than 0 demonstrating that application of the
constraints can lead to a change in the alternative design
The analysis is conducted at σ = 0.1 for baseline results (an activation becomes and inhibition) unless checks for
and again at σ = 0.2 to address sensitivity with respect to reasonableness of constrained parameters are employed.
the sampling space. Groups were also checked for the existence of stable
steady states. A total of 160 groups were eliminated
We define four levels of equivalence for this analysis. The because of unreasonable parameters or because one or
lowest level requires only that the alternative instance more cases in the group did not exhibit a stable steady
adhere to the sampling distributions for parameters state at either the baseline conditions (X4, X5, and X6) or
described above. Instances drawn as such are representa- at the steady state for a four-fold increase in source anti-
tives of the alternative design but do not exhibit internal gen. Summary statistics were computed for each of the
or external equivalence with the paired reference system. constrained parameters (Table 1). Eliminating the groups
The only values shared between the alternative and the identified above resulted in 4200 cases in 840 groups that
paired reference are those of the independent variables. were further analyzed.
We next define an internally equivalent alternative as one
in which the values for α1, α3, β1, β2, β3, g1,1, g1,4, g3,2, g3,6, Functional performance measures were computed for
h1,1, h1,2, h2,2, and h3,3 in the alternative are equal to the each case and analyzed according to both Method 1 and
associated values in the paired reference. We define a par- Method 2 using the statistical package R [25]. Density
tially equivalent alternative as one that is both internally plots for each of the effectiveness measures and associated
equivalent and that also has steady-state values of medians for the reference design are given in Figure 5. For
dependent variables equal to those of the paired reference. Method 1, functional effectiveness measures for all cases
This constraint is used to fix the value of α2 in the alterna- were compared to the median over all reference cases. The
tive instance. Symbolic solutions for the steady-state val- outcome was defined such that odds ratios greater than 1
ues of the dependent variables are computed for both the indicate a preference for the reference design. For Method
reference and alternative, expressions for X2 are set equal 2, functional effectiveness measures for each case were
and solved for α2. To construct a partially equivalent alter- compared to the functional effectiveness measures of the
native we form an internally equivalent alternative and paired reference. Cases in which the performance meas-
additionally compute the value of α2 to satisfy the con- ures were equal were eliminated prior to the regression as
straint. The fourth level of equivalence, referred to as full they contribute no information to an estimate of prefer-
equivalence, incorporates constraints sufficient to remove ence. Outcomes were again defined such that odds greater
all remaining degrees of freedom in the formation of the than 1 indicate a preference for the reference design. The
alternative. The fully equivalent form is constructed by conditional logistic regressions for Method 1 were com-
forcing two additional constraints which fix the values of puted using the clogit function and the logistic regressions
g2,1 and g2,5. The value for g2,1 is determined by requiring for Method 2 were computed using glm with binomial
that L1,4 be equal in both the reference and alternative sys- family and logit link. Regression parameters were
tems and the value for g2,5 is determined by requiring that exponentiated to yield odds ratios (Method 1) and odds
L1,5 + L1,6 be equal in both systems. Symbolic solutions for (Method 2). Results with σ = 0.1 for Method 1 are given
the log gains are computed, set equal and solved for the in Table 2 and for Method 2 in Table 3. Comparisons that
desired parameter. Fully equivalent systems are then were fixed by constraint are indicated in the table. Regres-
formed by forcing internal equivalence and by fixing α2, sion parameters were considered to indicate a preference
g2,1, and g2,5 to satisfy the given constraints. The selected if they were determined to be statistically different from
constraints match those employed by Irvine and Savageau the null value at the α = 0.05 level. The process was
[21]. repeated for σ = 0.2 to examine sensitivity with respect to
the sampling distributions. Those results are given in
Groups of cases were constructed by sampling the param- Tables 4 and 5.
eter distributions defined above. For each group we drew
one set of values for the parameters of the reference, one We find that the first two measures (basal levels of antigen
set of values for the independent variables, and for each and effector lymphocytes) show no statistically significant
level of equivalence we drew a set of values for those difference between the reference and alternative design
parameters not fixed by the associated constraints. Five even when the reference and alternative instances are
cases were constructed for each of 1,000 groups consisting independent. This finding holds for both methods of
of one reference and four alternatives, one for each of the analysis. Examining the pairwise scatter plots for these
Page 11 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Table 1: Summary statistics for model parameters. Summary statistics for all of the parameters fixed by equivalence were computed
across the three alternatives for which equivalence constraints were applied after elimination of cases which violated the requirement
for steady state or lower bounds for g2,1 or g2,5.
Parameter
Property α2 g2,1 g2,5
Dynamic Effector Dynamic Antigen Sensitivity Effector Gain Antigenic Gain Effector Systemic Antigen
Density Density Density Density Density Density Density
0 2 4 6 0.0 1.0 2.0 0.0 1.0 2.0 0.0 0.4 0.8 0.00 0.20 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Reference
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
6
Density Density Density Density Density Density Density
0 2 4 6 0.0 0.4 0.8 0.0 1.0 0.0 0.4 0.8 0.0 0.2 0.4 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
Independent
0 1 2 3 4 5
Alternative
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
0 2 4 6 0.0 0.3 0.6 0.0 1.0 0.0 0.4 0.8 0.0 0.2 0.4 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Equivalent
Internally
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
0 2 4 6 0.0 0.4 0.0 1.0 0.0 0.4 0.8 0.0 0.2 0.4 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Equivalent
Partially
S(L14, h22)
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
0.8
4
2.0
2.0
2.0
0 4 8 0.0 0.4 0.8 0.0 1.0 0.0 0.4 0.8 0.00 0.20 0.0 0.4 0.8 0.0 0.3
0.00
0.0
0.0
0.0
0.0
0 1 2 3 4 5
Equivalent
S(L14, h22)
Fully
0.4
2
max X2
max X1
1.0
1.0
1.0
0.15
L24
L14
X2
X1
4
0.8
2.0
2.0
2.0
Figure 5
Performance measure distributions
Performance measure distributions. Smoothed histograms were computed using R (density function) and are presented
for each of the 7 functional performance measures for each of the equivalence levels. The dashed vertical line indicates the
median of the distribution for the reference system. The alternative is preferred when more of its probability mass is distrib-
uted to the left of the dashed line.
Page 12 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Table 2: Odds ratios for comparison using Method 1. Odds ratios for preference of system with suppression using conditional logistic
regression (Method 1). The data consisted of 840 groups of 5 cases each (reference and four alternatives) sampled as described. Values
shown give the odds that the reference system is preferred over the alternative. Values marked with asterisks are significant at the α =
0.05 level. Kinetic orders and rate constants were sampled with σ = 0.1. Cells marked with an odds ratio of 1 or = indicate cases that
were fixed by equivalence.
Table 3: Odds ratios for comparison using Method 2. Odds for preference of system with suppression using paired comparisons and
logistic regression (Method 2). The data consisted of 840 groups of 5 cases each (reference and four alternatives) sampled as described.
Values shown give the odds that the reference system is preferred over its paired alternative. Values marked with asterisks are
significant at the α = 0.05 level. Kinetic orders and rate constants were sampled with σ = 0.1. Cells marked with an odds ratio of 1 or =
indicate cases that were fixed by equivalence. † Odds of ∞ shown for measures were reference was found to be better than the
alternative in every case sampled.
cases (Figure 6) we see little correlation at this level of tistically significant difference for paired comparisons.
equivalence. The introduction of internal equivalence cre- The introduction of partial and full equivalence requires
ates a much stronger correlation between the values but these measures to be the same and thus contributes no
the correlation follows the null line (same basal level in information to the comparison.
both reference and alternative) and thus still shows no sta-
Page 13 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Table 4: Odds ratios using Method 1 with increased sampling variance. Results from sensitivity analysis using increased variance in
sampled population (σ = 0.2 versus σ = 0.1) giving odds ratios for preference of system with suppression using conditional logistic
regression (Method 1). The data consisted of 460 groups of 5 cases each (reference and four alternatives) sampled as described. Values
shown give the odds that the reference system is preferred over the alternative. Values marked with asterisks are significant at the α =
0.05 level. Kinetic orders and rate constants were sampled with σ = 0.2. Cells marked with an odds ratio of 1 or = indicate cases that
were fixed by equivalence.
Table 5: Odds ratios using Method 2 with increased sampling variance. Results from sensitivity analysis using increased variance in
sampled population (σ = 0.2 versus σ = 0.1) giving odds for preference of system with suppression using paired comparisons and logistic
regression (Method 2). The data consisted of 460 groups of 5 cases each (reference and four alternatives) sampled as described. Values
shown give the odds that the reference system is preferred over its paired alternative. Values marked with asterisks are significant at
the α = 0.05 level. Kinetic orders and rate constants were sampled with σ = 0.2. Cells marked with an odds ratio of 1 or = indicate cases
that were fixed by equivalence. † Odds of ∞ shown for measures were reference was found to be better than the alternative in every
case sampled.
Functional effectiveness measures of antigenic gain and than the reference for the independent, internal, and
effector gain show differing preference for the alternative partial equivalence cases. The reference system, however,
and reference designs. The alternative design shows statis- demonstrates statistically significant preference for
tically significant odds of having a lower antigenic gain reduced effector gain. These findings are again consistent
Page 14 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
6
5
5
Systemic Antigen
4
X1
X1
X1
X1
3
3
2
2
1
1
0
0
0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6
X1 X1 X1 X1
2.0
2.0
2.0
2.0
1.5
1.5
1.5
1.5
Effector
X2
X2
X2
X2
1.0
1.0
1.0
1.0
0.5
0.5
0.5
0.5
0.0
0.0
0.0
0.0
0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0
X2 X2 X2 X2
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
Sensitivity
S (L14 , h22 )
S (L14 , h22 )
S (L14 , h22 )
S (L14 , h22 )
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8
across both methods of analysis. Introducing constraints pression being preferred over an instance without sup-
to achieve full equivalence causes these gains to be equal pression with respect to dynamic levels of antigen and
in both the reference and alternative. The introduction of effector are 32 and 2.3 for Method 1 and 3.9 and 1.6 for
constraints on L1,4 and L1,5 + L1,6 hides the differential Method 2. The strong preference for the design with
preference with respect to effector and antigen gain. Anal- suppression is seen at all levels of equivalence for both
ysis using the incremental introduction of constraints per- Method 1 and Method 2 when considering dynamic levels
mits the observance of these differences. of antigen. A significant preference for the design with
suppression is also seen for dynamic levels of effector. The
Dynamic responses for both antigen peak levels and effec- finding under this measure does lose significance under
tor peak levels demonstrate a preference for the reference full equivalence for Method 1. We conclude that the refer-
design. Even for an independent alternative (no internal ence design is better than or equal to the alternative in all
or external equivalence), the odds of an instance with sup- cases.
Page 15 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Page 16 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
n+ m n+ m Acknowledgements
dXi g h
= α i ∏ X j i , j − β i ∏ X j i , j for i ∈ 1...n This research was supported under NLM Training Grant T15LM07438
dt j =1 j =1 (E.O. Voit, PI) and NSF-BES Quantitative Systems Biotechnology research
The S-system representation offers a particular advantage grant 0120288 (E.O. Voit, PI).
in that it allows for the determination of the system's
steady state by solution of a system of linear equations.
References
1. Bhalla US: Understanding complex signaling networks
Setting the time derivatives to 0 and taking logs results is through models and metaphors. Prog Biophys Mol Biol 2003,
a linear system of equations from which the log steady- 81:45-65.
2. Savageau MA: Biochemical systems analysis : a study of function and design
state values of the n dependent variables can be deter- in molecular biology Reading, Mass.: Addison-Wesley Pub. Co.
mined from the values of the m independent variables Advanced Book Program; 1976.
and the system parameters [26]. 3. Savageau MA: Design principles for elementary gene circuits:
Elements, methods, and examples. Chaos 2001, 11:142-159.
4. Savageau MA: Biochemical systems analysis. I. Some mathe-
−1 −1 matical properties of the rate law for the component enzy-
yD = AD b− AD AI yI matic reactions. J Theor Biol 1969, 25:365-369.
bi = ln ( β i ) − ln ( α i ) 5. Savageau MA: Biochemical systems analysis. II. The steady-
state solutions for an n-pool system using a power-law
ai , j = gi , j − hi , j (11) approximation. J Theor Biol 1969, 25:370-379.
6. Savageau MA: Biochemical systems analysis. 3. Dynamic solu-
y D,i = ln ( Xi ) for i ∈ 1...n tions using a power-law approximation. J Theor Biol 1970,
26:215-226.
y I ,i = ln ( Xi + n ) for i ∈ 1...m 7. Torres NV, Voit EO: Pathway analysis and optimization in metabolic
engineering New York: Cambridge University Press; 2002.
S-systems additionally provide for the convenient charac- 8. Voit EO: Canonical nonlinear modeling : S-system approach to understand-
terization of systemic performance local to the steady ing complexity New York: Van Nostrand Reinhold; 1991.
9. Voit EO: Computational analysis of biochemical systems: a practical guide
state. Logarithmic gains that represent the change in the for biochemists and molecular biologists New York: Cambridge Univer-
log value of the steady state of a dependent variable or flux sity Press; 2000.
as a result of a change in the log value of an independent 10. Savageau MA: A theory of alternative designs for biochemical
control systems. Biomed Biochim Acta 1985, 44:875-880.
variable are computed as follows [26]. 11. Alves R, Savageau MA: Extending the method of mathemati-
cally controlled comparison to include numerical
comparisons. Bioinformatics 2000, 16:786-798.
12. Alves R, Savageau MA: Comparing systemic properties of
ensembles of biological networks by graphical and statistical
methods. Bioinformatics 2000, 16:527-533.
Page 17 of 18
(page number not for citation purposes)
Theoretical Biology and Medical Modelling 2004, 1 http://www.tbiomed.com/content/1/1/1
Page 18 of 18
(page number not for citation purposes)