In Part One we had a look at some introductory ideas. In this article we will look at one of the ground-breaking papers in chaos theory - Deterministic nonperiodic flow, Edward Lorenz (1963). It has been cited more than 13,500 times.
There might be some introductory books on non-linear dynamics and chaos that don’t include a discussion of this paper – or at least a mention – but they will be in a small minority.
Lorenz was thinking about convection in the atmosphere, or any fluid heated from below, and reduced the problem to just three simple equations. However, the equations were still non-linear and because of this they exhibit chaotic behavior.
Cencini et al describe Lorenz’s problem:
Consider a fluid, initially at rest, constrained by two infinite horizontal plates maintained at constant temperature and at a fixed distance from each other. Gravity acts on the system perpendicular to the plates. If the upper plate is maintained hotter than the lower one, the fluid remains at rest and in a state of conduction, i.e., a linear temperature gradient establishes between the two plates.
If the temperatures are inverted, gravity induced buoyancy forces tend to rise toward the top the hotter, and thus lighter fluid, that is at the bottom. This tendency is contrasted by viscous and dissipative forces of the fluid so that the conduction state may persist.
However, as the temperature differential exceeds a certain amount, the conduction state is replaced by a steady convection state: the fluid motion consists of steady counter-rotating vortices (rolls) which transport upwards the hot/light fluid in contact with the bottom plate and downwards the cold heavy fluid in contact with the upper one.
The steady convection state remains stable up to another critical temperature difference above which it becomes unsteady, very irregular and hardly predictable.
Willem Malkus and Lou Howard of MIT came up with an equivalent system – the simplest version is shown in this video:
Steven Strogatz (1994), an excellent introduction to dynamic and chaotic systems – explains and derives the equivalence between the classic Lorenz equations and this tilted waterwheel.
L63 (as I’ll call these equations) has three variables apart from time: intensity of convection (x), temperature difference between ascending and descending currents (y), deviation of temperature from a linear profile (z).
Here are some calculated results for L63 for the “classic” parameter values and three very slightly different initial conditions (blue, red, green in each plot) over 5,000 seconds, showing the start and end 50 seconds – click to expand:
Figure 2 - click to expand – initial conditions x,y,z = 0, 1, 0; 0, 1.001, 0; 0, 1.002, 0
We can see that quite early on the two conditions diverge, and 5000 seconds later the system still exhibits similar “non-periodic” characteristics.
For interest let’s zoom in on just over 10 seconds of ‘x’ near the start and end:
Going back to an important point from the first post, some chaotic systems will have predictable statistics even if the actual state at any future time is impossible to determine (due to uncertainty over the initial conditions).
So we’ll take a look at the statistics via a running average – click to expand:
Figure 4 - click to expand
Two things stand out – first of all the running average over more than 100 “oscillations” still shows a large amount of variability. So at any one time, if we were to calculate the average from our current and historical experience we could easily end up calculating a value that was far from the “long term average”. Second – the “short term” average, if we can call it that, shows large variation at any given time between our slightly divergent initial conditions.
So we might believe – and be correct – that the long term statistics of slightly different initial conditions are identical, yet be fooled in practice.
Of course, surely it sorts itself out over a longer time scale?
I ran the same simulation (with just the first two starting conditions) for 25,000 seconds and then used a filter window of 1,000 seconds – click to expand:
Figure 5 - click to expand
The total variability is less, but we have a similar problem – it’s just lower in magnitude. Again we see that the statistics of two slightly different initial conditions - if we were to view them by the running average at any one time – are likely to be different even over this much longer time frame.
From this 25,000 second simulation:
- take 10,000 random samples each of 25 second length and plot a histogram of the means of each sample (the sample means)
- same again for 100 seconds
- same again for 500 seconds
- same again for 3,000 seconds
Repeat for the data from the other initial condition.
Here is the result:
To make it easier to see, here is the difference between the two sets of histograms, normalized by the maximum value in each set:
This is a different way of viewing what we saw in figures 4 & 5.
The spread of sample means shrinks as we increase the time period but the difference between the two data sets doesn’t seem to disappear (note 2).
Attractors and Phase Space
The above plots show how variables change with time. There’s another way to view the evolution of system dynamics and that is by “phase space”. It’s a name for a different kind of plot.
So instead of plotting x vs time, y vs time and z vs time – let’s plot x vs y vs z – click to expand:
Figure 8 - Click to expand – the colors blue, red & green represent the same initial conditions as in figure 2
Without some dynamic animation we can’t now tell how fast the system evolves. But we learn something else that turns out to be quite amazing. The system always end up on the same “phase space”. Perhaps that doesn’t seem amazing yet..
Figure 7 was with three initial conditions that are almost identical. Let’s look at three initial conditions that are very different: x,y,z = 0, 1, 0; 5, 5, 5; 20, 8, 1:
Figure 9 - Click to expand
Here’s an example (similar to figure 7) from Strogatz – a set of 10,000 closely separated initial conditions and how they separate at 3, 6, 9 and 15 seconds. The two key points:
- the fast separation of initial conditions
- the long term position of any of the initial conditions is still on the “attractor”
A dynamic visualization on Youtube with 500,000 initial conditions:
There’s lot of theory around all of this as you might expect. But in brief, in a “dissipative system” the “phase volume” contracts exponentially to zero. Yet for the Lorenz system somehow it doesn’t quite manage that. Instead, there are an infinite number of 2-d surfaces. Or something. For the sake of a not overly complex discussion a wide range of initial conditions ends up on something very close to a 2-d surface.
This is known as a strange attractor. And the Lorenz strange attractor looks like a butterfly.
Lorenz 1963 reduced convective flow (e.g., heating an atmosphere from the bottom) to a simple set of equations. Obviously these equations are a massively over-simplified version of anything like the real atmosphere. Yet, even with this very simple set of equations we find chaotic behavior.
Chaotic behavior in this example means:
- very small differences get amplified extremely quickly so that no matter how much you increase your knowledge of your starting conditions it doesn’t help much (note 3)
- starting conditions within certain boundaries will always end up within “attractor” boundaries, even though there might be non-periodic oscillations around this attractor
- the long term (infinite) statistics can be deterministic but over any “smaller” time period the statistics can be highly variable
Deterministic nonperiodic flow, EN Lorenz, Journal of the Atmospheric Sciences (1963)
Chaos: From Simple Models to Complex Systems, Cencini, Cecconi & Vulpiani, Series on Advances in Statistical Mechanics – Vol. 17 (2010)
Non Linear Dynamics and Chaos, Steven H. Strogatz, Perseus Books (1994)
Note 1: The Lorenz equations:
dx/dt = σ (y-x)
dy/dt = rx – y – xz
dz/dt = xy – bz
x = intensity of convection
y = temperature difference between ascending and descending currents
z = devision of temperature from a linear profile
σ = Prandtl number, ratio of momentum diffusivity to thermal diffusivity
r = Rayleigh number
b = “another parameter”
And the “classic parameters” are σ=10, b = 8/3, r = 28
Note 2: Lorenz 1963 has over 13,000 citations so I haven’t been able to find out if this system of equations is transitive or intransitive. Running Matlab on a home Mac reaches some limitations and I maxed out at 25,000 second simulations mapped onto a 0.01 second time step.
However, I’m not trying to prove anything specifically about the Lorenz 1963 equations, more illustrating some important characteristics of chaotic systems
Note 3: Small differences in initial conditions grow exponentially, until we reach the limits of the attractor. So it’s easy to show the “benefit” of more accurate data on initial conditions.
If we increase our precision on initial conditions by 1,000,000 times the increase in prediction time is a massive 2½ times longer.