arXiv:astro-ph/9610044 v1 07 Oct 1996

Theories of the Cosmological Constant

Steven Weinberg
Physics Department, University of Texas
Austin, Texas 78712
[email protected]
August 29, 1996

Abstract This is a talk given at the conference Critical Dialogues in Cosmology at Princeton University, June 2427, 1996. It gives a brief summary of
our present theoretical understanding regarding the value of the cosmological
constant, and describes how to calculate the probability distribution of the observed cosmological constant in cosmological theories with a large number of
subuniverses (i. e., different expanding regions, or different terms in the wave
function of the universe) in which this constant takes different values.

UTTG-10-96

Research supported in part by the Robert A. Welch Foundation and NSF Grants PHY
9009850 and PHY 9511632.

Introduction

The problem of the cosmological constant looks different to astronomers and

particle physicists. Astronomers may prefer the simplicity of a zero cosmological constant, but they are also prepared to admit the possibility of a cosmological
constant in a range extending up to values that would make up most of the critical density required in a spatially flat RobertsonWalker universe. To a particle
physicist, all the values in this observationally allowed range seem ridiculously
implausible.
To see why, it is convenient to consider the effective quantum field theory
that takes into account only degrees of freedom with energy below about 100
GeV, with all higher energy radiative corrections buried in corrections to the
various parameters in the effective Lagrangian. In this effective field theory, the
vacuum energy density that serves as a source of the long-range gravitational
field may be written as
1X

+
h ,
(1)
V =
8G 2
where is the cosmological constant appearing in the Einstein field equations,
and the second term symbolizes the contribution of quantum fluctuations in
the fields of the effective field theory, cut off at particle energies equal to 100
GeV. Now, we know almost everything about this effective field theory it
is what particle physicists call the standard model and we know that the
quantum fluctuations do not cancel, so that on dimensional grounds, in units
with h
= c = 1, they yield
1X
h (100 GeV)4
2

(2)

On the other hand, observations do not allow V to be much greater than the
critical density, which in these units is roughly 1048 GeV4 . Not to worry
just arrange that the Einstein term has a value for which the two terms in
Eq. (1) cancel to fifty-six decimal places. This is the cosmological constant
problem: to understand this cancellation.
Here I will consider three main directions for solving this problem[1]:

Deep Symmetries

Cancellation Mechanisms

Anthropic Constraints

By a deep symmetry I mean some new symmetry of an underlying theory,

which is not an unbroken symmetry of the effective field theory below 100 GeV
(because we know all these symmetries), but which nevertheless requires V to
vanish. In other contexts supersymmetry can sometimes play the role of a deep
symmetry, in the sense that some dimensionless bare constants that are required
1

to vanish by supersymmetry can be shown to vanish to all orders in perturbation

theory even though supersymmetry is spontaneously broken. Unfortunately the
vacuum density is not a constant of this sort it has dimensionality (mass)4
instead of being dimensionless, and it is a renormalized coupling rather than a
bare coupling. Recently Witten has proposed a highly imaginative and speculative mechanism by which some form of supersymmetry makes V vanish[2]. I
am grateful to the organizing committee of this conference for giving me only
15 minutes to talk, so that I dont have to try to explain Wittens idea. I turn
instead to the other two approaches on my list.

Cancellation Mechanisms

The special thing about having V = 0 is that it makes it possible to find

spacetime-independent solutions of the Einstein gravitational field equations.
For such solutions, we have
L/g = 0 ,
(3)
where L is the Lagrangian density for constant fields. The problem occurs
only in the trace of this equation, which receives a contribution from V which
for V 6= 0 prevents a solution. Many theorists have tried to get around this
difficulty by introducing a scalar field in such a way that the trace of L/g
is proportional to L/:
g L/g = f ()L/ ,

(4)

with f () arbitrary, except for being finite. Where this is done, the existence
of a solution of the field equation L/ = 0 for a spacetime-independent
implies that the trace g L/g = 0 of the Einstein field equation for a
spacetime-independent metric is also satisfied. The trouble is that, with these
assumptions, the Lagrangian has such a simple dependence on that it is not
possible to find a solution of the field equation for R. This is because Eq. (4),
together with the general covariance of the action d4 x L, tells us that, when
the action is stationary with respect to variations of all other fields, it has a
symmetry under the transformations
= f () ,

g = 2g ,

(5)

which requires the Lagrangian density for spacetime-independent fields g and

to have the form
!
Z
p
d
,
(6)
L = c det g exp 4
f ( )
where c is a constant whose value depends on the lower limit chosen for the
integral. For c 6= 0, there is no solution at which this is stationary with respect
2

to . The literature is full of proposed solutions of the cosmological constant

problem based on this sort of spontaneous adjustment of one or more scalar
fields, but if you look at them closely, you will see that either they do not satisfy
Eq. (4), in which case there may be a solution for but it does not imply the
vanishing of V , or else they do satisfy Eq. (4), in which case a solution of the
field equation for would imply a vanishing V , but there is no solution of the
field equation for . To the best of my knowledge, no one has found a way out
of this impasse.

Anthropic Considerations

Suppose that the observed subuniverse is only one of many subuniverses, in

which V takes a variety of different values. This is the case for instance in
theories of chaotic inflation[3], in which various scalar fields on which the vacuum
energy depends take different values in different expanding regions of space. In
a somewhat more subtle way, this can also be the case in some versions of
quantum cosmology, where the wave function of the universe is a superposition
of terms in which V takes different values, either because of the presence of
some vacuum field (like the antisymmetric tensor gauge field A introduced
for this purpose by Hawking[4]), or because of wormholes, as in the work of
Coleman[5].
Some authors[4], [6], [7] have argued that in quantum cosmology the distribution of values of V is very sharply peaked at V = 0, which would immediately solve the cosmological constant problem. This conclusion has been
challenged[8], and it will be assumed here that the probability distribution of
V is smooth at V = 0, without any sharp peaks or dips.
In any theory of this general sort the measured effective cosmological constant would be much smaller than the value expected on dimensional grounds
in elementary particle physics, not because there is any physical principle that
makes it small in all subuniverses, but because it is only in the subuniverses
where it is sufficiently small that there would be anyone to measure it. For
negative values of V , this limitation comes from the requirement that the subuniverse must survive long enough to allow for the evolution of life[9]. For
positive values of V (which are observationally more promising) the limitation
comes from the requirement that large gravitational condensations like galaxies must be able to form before the subuniverse begins its final exponential
expansion[10].
If you dont find this sort of anthropic explanation palatable, consider the
following fable. You are an astronaut, sent out to explore a randomly chosen
planet around some distant star, about which nothing is known. Shortly before
you leave you learn that because of budget cuts, NASA has not been able to
supply you with any life-support equipment to use on the planets surface. You
arrive on the planet, and find to your relief that conditions are quite tolerable
3

the air is breathable, the temperature is about 300 K, and the surface gravity
is not very different from what it is on earth. What would you conclude about
the conditions on planets in general? It all depends on how many astronauts
NASA has sent out. If you are the only one then its reasonable to infer that
tolerable conditions must be fairly common, contrary to what planetologists
would have naturally expected. On the other hand, if NASA has sent out a
million astronauts, then all you can conclude about the statistics of planetary
conditions is that the number of planets with tolerable conditions is probably not
much less than one in a million for all you know, almost all of the astronauts
have arrived on planets that cannot support human life. Naturally, the only
astronauts in this program that are in a position to think about the statistics
of planetary conditions are those like you who are lucky enough to have landed
on a planet on which they can live; the others are no longer worrying about it.
In previous work[10] I calculated the anthropic upper bound on the cosmological constant, which arises from the condition that V should not be so large
as to prevent the formation of gravitational condensations on which life could
evolve. This bound is naturally larger than the average value of the cosmological constant that would be measured by typical observers, which obviously gives
a better estimate of what we might find in our subuniverse. (Vilenkin[11] has
advocated this point of view under the name of the principle of mediocrity,
but did not attempt a detailed analysis of its consequences.) The difference
is important, because the anthropic upper bound on V is considerably larger
than the largest value of V allowed by observation.
I will leave the observational limits on the cosmological constant to Dr.
Fukugitas talk, but without going into details, it seems that for a spatially flat
(i.e., k = 0) universe, V is likely to be positive and somewhat larger than the
present mass density 0 , but probably not larger than 30 [12]. On the other
hand, we know that some galaxies were already formed at redshifts z 4, at
which time the density of matter was larger than the present density 0 by a
factor (1 + z)3 125. It therefore seems unlikely that a vacuum energy density
much smaller than 1250 could have completely prevented the formation of
galaxies, so the anthropic upper bound on V cannot be much less than about
1250 , which is much greater than the largest observationally allowed value of
V .
In contrast, we would expect the anthropic mean value of V to be roughly
comparable to the mass density of the universe at the time of the greatest rate
for the accretion of matter by growing galaxies, because it is unlikely for V
to be much greater than this and there is no reason why it should be much
smaller. (I will make this more quantitative soon.) Although there is evidence
that galaxy formation was well under way by a redshift z 3, it is quite possible
that most accretion of matter into galaxies continues to lower redshifts, as seems
to be indicated by cold dark matter models. In this case the anthropic mean
value < V > will be considerably less than the anthropic upper bound, and
perhaps within the range allowed observationally.
4

I would like to present an illustrative example of a calculation of the whole

probability distribution of the cosmological constant that would be measured
by observers, weighted by the likelihood that there are observers to measure
it. Instead of the very simple model[13] of galaxy formation from spherically
symmetric pressureless fluctuations used previously[10], here I will rely on the
well-known model of Gunn and Gott[14], which also assumes spherical symmetry
and zero pressure, but takes into account the infall of matter from outside the
initially overdense core. This is still far from realistic, but it will allow me
to make four points about such calculations, which should be more generally
applicable.
As shown in earlier work[10], the condition for a spherically symmetric fluctuation to recondense is that
500 ()3
> V .
729 2

(7)

where and are the average density and the overdensity in the fluctuation at some early initial time, say the time of recombination. Previously
was assumed to be uniform within a spherical fluctuation, but Eq. (7) actually
applies to any sphere, with understood to be the spatially averaged initial
overdensity within the sphere.
Suppose that the fluctuation at recombination consists of a finite spherical
core of volume V with positive average overdensity , outside of which the
density takes its average value . (This picture is appropriate for well separated
fluctuations. The effects of crowding and underdense regions will be considered
in a future paper.) Then the average overdensity within a larger volume V
centered on this core is = V /V . Assuming that Eq. (7) is satisfied by the
average overdensity within the core,

500 ()3 
> V ,
(8)
729 2 recomb

the average overdensity will satisfy the condition (7) out to a volume
Vmax =

500
729V

1/3

2/3 V

so the total mass that will eventually collapse is

"
1/3 #

500
.
M = V + Vmax = V 1 +
729V

(9)

Once a galaxy forms, the subsequent evolution of stars and planets and life is
essentially independent of the cosmological constant (this is point 1), so the
number of independent observers arising from a given fluctuation at the time of
5

recombination is proportional to the mass (9) for those fluctuations satisfying

Eq. (8), and is otherwise zero. Of course, the value of the cosmological constant
might be correlated with the values of other fundamental constants, on which
the evolution of life does depend, but the range of anthropically allowed cosmological constants is so small compared with the natural scale (2) of densities in
elementary particle physics that within this range it is reasonable to suppose
that all other constants are fixed. (This is point 2.) The range of values of V
for which gravitational condensations are possible is also so much less than the
average density at the time of recombination, that the number of fluctuations
N (, V ) dV d with volume between V and V + dV and average overdensity
between and + d should be nearly independent of V . (This is point 3.)
If P(V ) dV is the a priori probability that a random subuniverse has vacuum
energy density between V and V + dV , then according to the principles of
Bayesian statistics, the probability distribution for observed values of V is
Z
Z
d N (, V )
dV
Pobs (V ) P(V )
(729V 2 /500)1/3

"

500
729V

1/3 #

"

500
729V

1/3 # Z

V 1 +

P(V ) 1 +

where
N ()

(729V 2 /500)1/3

d N ()(10)

dV V N (, V ) .

(11)

Finally, the range of values of V for which gravitational condensations are

possible is so small compared with the natural scale of densities in elementary
particle physics that within this range the a priori probability P(V ) may be
taken as constant. (This is point 4.) The factor P(V ) may therefore be omitted
in the probability distribution (10). Also, all anthropically allowed values of V
are much smaller than the mass density at recombination, so we may neglect
the 1 in the square brackets in Eq. (10), which now becomes
Z
1/3
Pobs (V ) V
d N () .
(12)
(729V 2 /500)1/3

Strictly speaking, this gives the probability distribution only for V > 0.
For V < 0 and k = 0, all mass concentrations that are large enough to allow
pressure to be neglected will undergo gravitational collapse. The number of
astronomers is instead limited[9] for V < 0 by the fact that the subuniverse
itself also collapses, in a time
s
2
3
T (|V |) =
.
(13)
3
8G|V |
6

In contrast, the probability distribution for V > 0 is weighted by an V independent factor, the average time T in which stars provide conditions favorable for intelligent life. The probability distribution for negative values of V is
small except for values of |V | that are small enough so that T (|V |) is less than
or of order T . It will be assumed here that T is very large, so that Pobs (V ) is
negligible for V < 0 except in a small range near zero, and may therefore be
neglected in calculating the mean value of V .
Using the probability distribution (12) and interchanging the order of the
integrals over and V , we easily see that the mean value of observed values
of V is
200 < 6 >
,
(14)
hV i =
729 < 3 > 2
with all quantities on the right-hand side evaluated at the time of recombination,
and the brackets on the right-hand side (unlike those in hV i) indicating averages
over fluctuations:
Z
d N () f () .
(15)
< f () >
0

It remains to use astronomical observations to calculate the fluctuation spectrum N () for the density fluctuations at recombination, which can then be
used in Eq. (12) to calculate the probability distribution for V . Here I will just
give one example of how information about the time of formation of galaxies
can put constraints on < V >. With a positive V , the core of a fluctuation
with average overdensity at recombination will collapse at a time when the
average cosmic density coll is less than it would be at the time of core collapse
for V = 0:[10]
500 3
coll <
,
(16)
243 2 2

with and on the right-hand side evaluated at recombination. Using this in

Eq. (14) gives a mean vacuum density
<V > >

2 2 hcoll i
.
15

(17)

Even if we suppose for example that core collapse occurs for most galaxies at
a redshift as low as z 1, then coll 80 , so Eq. (19) gives < V > > 100 ,
which exceeds current experimantal bounds on V . On the other hand, the
median value of V is less than the mean value, so the discrepancy is less than
this. Even so, it seems that most galaxies must be formed quite late in order
for the value of V in our universe to be close to the value that is anthropically
expected.
***
At the meeting in Princeton I learned of an interesting paper by Efstathiou[15],
in which he calculated the effect of a cosmological constant on the present num7

ber density of L galaxies, which he took as a measure of the distribution function Pobs (V ). In this calculation he adopted a standard cold dark matter
model for matter density fluctuations, with amplitude at long wavelengths fixed
by the measured anisotropy of the cosmic microwave background. Efstathiou
found that for a spatially flat universe the galaxy density falls off rapidly (say,
by a factor 10) for values of V around 7 to 9 times the present mass density
0 , so that < V > /0 should be less than of order 7 to 9, giving a contribution
0 = 0 /(0 + V ) of matter to the total density somewhat greater than around
0.1, which is consistent with lower bounds on the present matter density.
At first sight this seems encouraging, but there are a few problems with
Efstathious calculation. For one thing, as pointed out by Vilenkin[11], the
probability distribution of observed values of V is related to the number of
galaxies (or, more accurately, the amount of matter in galaxies) that ever form,
rather than the number that have formed when the age of the universe is at
any fixed value, as assumed by Efstathiou. However, this will not make much
difference if most galaxy formation is complete in typical subuniverses when
they are as old as our own subuniverse. Efstathiou also encountered another
problem that is endemic to this sort of calculation. The cosmological parameters that can reasonably be assumed to be uncorrelated with the cosmological
constant are the baryontoentropy ratio and the spectrum of density fluctuations at recombination, because these are presumably fixed by events that
happened before recombination, when any anthropically allowed cosmological
constant would have been negligible. But the only way we know about the
spectrum of density fluctuations at recombination is to use observations of the
present microwave background (or possibly the numbers of galaxies at various
redshifts), and unfortunately the results we obtain from this for N () depend
on the value of the cosmological constant in our subuniverse. In calculating
Pobs (V ) one should ideally make some assumption about the value of V in
our subuniverse, then use this value to infer a spectrum of density fluctuations
at recombination from the observed microwave anisotropies, and then calculate
the number of galaxies that ever form as a function of V , with the spectrum of
density fluctuations at recombination held fixed. Instead, Efstathiou calculated
the number of L galaxies as a function of V , with the microwave anisotropies
held fixed, which gave Pobs (V ) an additional spurious dependence on V . This
problem was known to Efstathiou, and apparently did not produce large errors.
There is one other problem, that did have a significant effect in Efstathious
calculation. He relied on the standard method[16] of calculating the evolution
of density fluctuations using linear perturbation theory, and declaring a galaxy
to have formed when the fractional overdensity / reaches a value c , which
is taken as the fractional overdensity of the linear perturbation at a time when a
nonlinear pressureless spherically symmetric fluctuation would recollapse to infinite density. He took the effective critical overdensity for spatially flat cosmologies as c = 1.68/0.28
, with 0 1 V /crit, so that c = 3.2 for 0 = 0.1.
0
But numerical calculations of Martel and Shapiro[17] show that for all fluctua8

tions that result in gravitational recollapse, c is in a range from 1.63 to 1.69.

The upper bound 1.69 is the well-known result c = (3/5)(3/2)3/2 = 1.6865
for V = 0. The lower bound 1.63 can also be understood analytically[10]: it is
the critical overdensity for the case where V has a value that just barely allows
gravitational recollapse
2
(c )min =

729
500

1/3

11
6

 
2
= 1.629 .
3

(18)

With c always between these bounds, it is impossible that the effective value of
c for any ensemble of fluctuations could be greater than 1.69. Overestimating
c biases the calculation toward late galaxy formation, with a corresponding
increased sensitivity to relatively small values of V . Efstathiou has now redone his calculations with c given the constant value 1.68, which should be a
good approximation, and, as I interpret his results, he finds that this change in
c roughly doubles the value of V at which the present density of L galaxies
drops by a factor 10, with a corresponding reduction in the expected value of
0 . It remains to be seen whether this change in his results will lead to a conflict
with observational bounds on 0 and V .
At present Martel and Shapiro are carrying out a numerical calculation of
Pobs using Eq. (12).
I am grateful for helpful discussions with George Efstathiou and Paul Shapiro.

References
[1] For a discussion of these and other possibilities, see Weinberg, S. 1989, Rev.
Mod. Phys. 61, 1
[2] Witten, E. 1995, Int. J. Mod. Phys. 10, 1247; preprint IASNS-HEP-95-51,
hep-th/9506101
[3] Linde, A. D. 1986, Phys. Lett. B 175, 395
[4] Hawking, S. W. 1983, in Shelter Island II Proceedings of the 1983 Shelter
Island Conference on Quantum Field Theory and the Fundamental Problems
of Physics, ed. R. Jackiw et al. (MIT Press, Cambridge, 1985); Phys. Lett. B
134, 403.
[5] Coleman, S. 1988, Nucl. Phys. B 307, 867
[6] Baum, E. 1984 Phys. Lett. B133, 185
[7] Coleman, S. 1988, Nucl. Phys. B 310, 643
[8] Fischler, W., Klebanov, I., Polchinski, J., and Susskind, L. 1989, Nucl. Phys.
B237, 157
9

[9] Barrow, J. D., and Tipler, F. J. 1986, The Anthropic Cosmological Principle
(Clarendon, Oxford)
[10] Weinberg, S. 1987, Phys. Rev. Lett. 59, 2607
[11] Vilenkin, A. 1995, Phys. Rev. Lett. 74, 846; Tufts preprint gr-qc/9507018,
to be published in the Proceedings of the 1995 International School of Astrophysics at Erice; Phys. Rev. D52, 3365; Tufts preprint gr-qc/9512031
[12] For a review and earlier references, see Ostriker, J. P. and Steinhardt, P.
J. 1995, Nature 377, 600
[13] Peebles, P. J. E. 1967, Astrophys. J. 147, 859
[14] Gunn, J. E. and Gott, J. R. 1972, Astrophys. J. 176, 1
[15] Efstathiou, G. 1995, Mon. Not. R. Astron. Soc. 174, L73
[16] Press, W. H. and Schechter, P. 1974, Astrophys. J. 187, 425
[17] Martel, H. and Shapiro, P. R. 1996, paper in preparation

10