When the Environment Remembers: Memory Effects in Synthetic Biology

Johannes Falk on January 9, 2020 · 9 min read

A genetic module you characterized in isolation behaves differently inside a cell — not just because of extra noise, but because the environment has a memory. Here is how we computed that memory exactly, by mapping reaction chains onto random walks.

Quicklinks

Paper: Context in Synthetic Biology: Memory Effects of Environments with Mono-molecular Reactions
Related research: Stochastic & dynamical systems

Synthetic biology is based on an exciting idea: You build genetic modules - an oscillator here, a switch there — characterize them precisely under laboratory conditions, and then assemble them into larger circuits, just like electronic components. A resistor remains a resistor no matter where you solder it in. Unfortunately, genetic modules are nowhere near as predictable.

As soon as a module is introduced into a living cell, it finds itself in a sea of other reactions. Proteins bind to sites where they aren’t supposed to, substrates are cleaved, and conformations change. The module suddenly becomes a small subnetwork, embedded in a much larger network, and its behavior changes. In scientific circles, these disruptions are referred to as context effects.

In our paper “Context in Synthetic Biology: Memory Effects of Environments with Mono-molecular Reactions”, we went after one particular and often underappreciated flavor of these context effects: the memory of the environment. In this post I want to walk through what that means and why a deceptively simple chain of reactions can change a module’s dynamics qualitatively.

Noise is not the whole story

It is well known and intuitively understandable that a fluctuating environment introduces noise into a module. However, the environment also has a memory.

The mechanism of this memory is easy to describe. The module disturbs the environment, for example, a protein binds to an external binding site. This disturbance then evolves independently for a while and later feeds back onto the module. The state of the module now therefore depends not only on its current state, but on its history. Mathematically speaking, this is a convolution term — an integral over the history of the subnetwork, weighted by a memory function (the memory kernel) that is characteristic of the environment.

The central question of this work is: What does this memory kernel actually look like? And—the part that’s interesting from a practical standpoint—can we compute it without knowing the exact structure of the environment in which a module will ultimately be located?

Marginalization: projecting out the subnetwork

Methodologically, we rely on the Nakajima–Mori–Zwanzig projection operator formalism. The core idea behind this technique from statistical physics is as follows: We have a large system, but we are only interested in a part of it, so we integrate out the remaining degrees of freedom exactly. What remains is an equation solely for the part that interests us — though at the cost of the integrated-out environment reappearing as memory and noise terms.

A methodological difference from earlier work (e.g., Rubin et al.) is significant here. Earlier approaches were based on the chemical Fokker–Planck or Langevin equation in the limiting case of vanishing noise. Instead, we apply the formalism directly to the chemical master equation (CME), the more fundamental description of a stochastic reaction network. We started from

\frac{\mathrm{d}}{\mathrm{d}t}\, p(t,\mathbf{x}) = \mathsf{T}\, p(t,\mathbf{x}),

where $\mathsf{T}$ is the forward evolution operator (a short notation for the full master-equation evolution). We split the state space into subnetwork and environment and project the environment out. The result is an equation with three parts: a Markovian piece (the immediate local dynamics), a memory term (the convolution over the past), and a noise term (which depends only on the unknown initial conditions of the environment and cannot be influenced by the module).

A side result, which to our knowledge had not been shown before: for linear networks the first-moment equations stay closed, and the marginalized CME yields the same mean equation as the corresponding deterministic treatment via the reaction rate equation (RRE). Stochastic and deterministic marginalization are therefore consistent here, which is what justifies doing the subsequent, purely deterministic calculation.

In this linear case the memory kernel has a compact form:

\mathbf{M}(t-t') = \mathbf{A}^{\mathrm{SE}}\, e^{(t-t')\,\mathbf{A}^{\mathrm{EE}}}\, \mathbf{A}^{\mathrm{ES}}.

It is read from right to left: $\mathbf{A}^{\mathrm{ES}}$ is the coupling from the subnetwork into the environment, $e^{(t-t')\mathbf{A}^{\mathrm{EE}}}$ is the environment’s own dynamics over the elapsed time, and $\mathbf{A}^{\mathrm{SE}}$ couples back into the module. So the term describes exactly this: go in, circulate in the environment for a while, and come back.

Reaction chains are random walks

So far the kernel is a matrix exponential, which in general can only be evaluated numerically. One finding of the paper is that for mono-molecular environments — environments in which every reaction has at most one reactant and one product ( $\varnothing \to X_i$ , $X_i \to \varnothing$ , $X_i \to X_j$ ) — the kernel can be written down analytically.

The environment we consider is a chain of such species:

A subnetwork coupled to a chain of mono-molecular reactions, with transition rates alpha and beta along the chain and inflow sigma and outflow kappa at each node — Generic environment: a chain of mono-molecular reactions. α and β are the transition rates along the chain, σ and κ the constant inflow and linear outflow at each node. The blue arrow marks the coupling to the subnetwork.

In monomolecular reactions, there is no interaction between individual molecules: each particle moves independently of the others. This allows us to view each species in the chain as the position of a random walker. Each reaction rate then becomes a transition probability between neighboring states. The question “How likely is it that a particle will return to the module after time $\tau$ ?” thus becomes the classical question of the first-pass time of a one-dimensional random walk with absorbing boundaries. This mapping is the key that makes the entire system manageable, because the matrix exponential function becomes a path-counting problem.

A collection of memory functions

The most general case for this problem is that of a chain with two absorbing states (at each end). In the publication, we formulate the problem using the backward master equation and use a result by Heathcote and Moyal from 1959 to solve it analytically. This is the most practically useful part: a toolbox of “standard environments” you can stress-test a module against without knowing the real cellular environment.

The complete equation is quite complicated (it includes, for example, the modified Bessel function of the first kind). Here, we will only mention a few special cases for which the memory kernel reduces to simple equations.

A single species. The simplest case — a protein binds reversibly to exactly one external binding site. The kernel is a simple exponential function:

M(\tau) = \beta\, e^{-\beta \tau}\, \alpha.

This is what you would naively expect: exponential decay of the memory.

Directed chain. A simple limiting case of the environment is a direct chain. To reach is, we set the rates of the back-reaction to zero ( $\alpha \to 0$ ). Hence, the particle can only travel one way through the chain before returning. The result is an Erlang distribution:

M(t) = \frac{\beta^{\omega}\, e^{-\beta t}\, t^{\omega-1}}{\Gamma(\omega)},

where $\omega$ is the position in the chain. For $\omega = 1$ this reduces to the single-species case. What is interesting is the behavior: a directed chain produces not a simply decaying but a delayed memory function with a peak at finite time. The environment does not “answer” immediately, but after a typical transit time.

Infinite chain Another limiting case is that of an infinite chain. The longer the chain the less is the influence of the second coupling to the subnetwork. The corresponding memory kernel is:

M(t) = \omega \nu^{-\omega} t^{-1} e^{-\mu t} I_{\omega},

where $\omega$ is the position the walker starts from, $\nu$ and $\mu$ are calculated from the transition rates, and $I_{x}$ is the modified bessel function of the first kind. Since the solution to this equation may not be immediately obvious to most people: When the subnetwork is coupled directly to the environment at $X_1$ ( $\omega = 1$ ), the curve behaves similarly to the case of coupling to a single species. However, since it is now possible to penetrate further into the chain, the memory function declines more slowly.

Finite chains. For more realistic finite chains with a reflective boundary, we used a mirror trick: the dynamics of a particle on a chain of length $\Omega$ with reflecting boundary condition is equivalent to the movement of a particle on a two-ended chain of length $2\Omega + 1$ .

The takeaway of the collection: Even a seemingly simple chain consisting of just a few species produces memory functions that are qualitatively very different (exponential, delayed, non-monotonic), depending on the topology and connection points.

Application Examples

Two modules demonstrate that these memory effects are not merely theoretical.

Retroactivity in the Repressilator

The Repressilator is the classic synthetic oscillator: three genes that mutually repress each other in a cycle, resulting in periodic oscillations. Del Vecchio et al. had shown that adding a downstream module induces feedback via retroactivity, though this assumes a temporal separation between a fast downstream module and a slow upstream module.

We do not need this assumption. We reversibly couple one of the oscillating proteins to a promoter (i.e., to a single-species environment with the exponential kernel derived above) and solve the resulting integral differential equation. The result: The oscillation frequency is drastically reduced, and the amplitudes also shift. A “fully characterized” oscillator therefore behaves measurably differently after coupling, without any changes to its internal parameters.

Oscillations Due to Promiscuity

The second example is less intuitive. A module that represses a promoter through dimerization has, on its own, a constant activation probability, and no oscillations occur.

Now let’s assume that a downstream module, due to promiscuity, produces a protein similar to the input protein $X$ and has a feedback effect on it. The crucial feature is the delay in this feedback: Between promoter activation and the reappearance of the protein lies a chain of intermediate steps: a directed chain with an Erlang kernel. In the limiting case of many steps with a fixed mean, the Erlang distribution transitions into a time-shifted delta function, i.e., into a pure time delay:

\left.\lim_{\omega\to\infty} M(t-\tau)\right|_{\omega/\alpha\,=\,\text{const}} = \delta\!\left(t - \tau - \tfrac{\omega}{\alpha}\right).

Due to this delayed feedback, promoter activity begins to oscillate—a behavior that simply does not occur when analyzed in isolation and that would not be observed at all using conventional methods (fixed environmental concentrations). The memory effect alone creates a new dynamic regime here.

What to take away

In synthetic biology you often do not know which environment a module will be embedded in. Rather than ignoring that, we provide a collection of generic environments with exactly known memory functions, against which a module can be checked for robustness.

The essentials:

Context effects are not just noise. The environment has a memory that reflects past module states back at it.
For linear / mono-molecular environments this memory can be computed exactly and analytically by mapping reaction chains onto random walks with absorbing boundaries.
Even simple chains can change a module’s dynamics qualitatively.