Computation in networks
 James K Peterson^{1}Email author
DOI: 10.1186/s404690150003z
© Peterson; licensee Springer. 2015
Received: 13 November 2014
Accepted: 15 March 2015
Published: 7 July 2015
Abstract
We present an introduction to the modeling of networks of nodes which parse the information presented to them into an output. One example is the nodes are excitable neurons which are collected into a nervous system for an animal whether invertebrate or vertebrate. We will focus on the development of the ideas and tools that might help us understand how to build a model of such a system being careful to explain the many approximations or model errors we make along the way. We start with a discussion of low level biophysical concepts such as the cable equation and the Hodgkin  Huxley model and end with graph based models of computation. We also include motivational arguments that show hardware and software issues in neural models are interwined.
Keywords
Nernst equation Cable equation Ball stick model Hodgkin  Huxley model Cognition models Graphs of computational nodesReview
In this article, we will discuss some of the principles behind models of biological information processing. As usual, we therefore use tools at the interface between science, mathematics and computer science. We want to find the right abstraction of the wealth of biological detail that is available; the right design to guide us in the development of our modeling environment. The details of how proteins interact, how genes interact and how neural modules interact to generate the high level outputs we find both interesting and useful are known to some degree but it is quite difficult to use this detailed information to build models of high level function. In all models of this type, we are asking high level questions and wondering how we might create a model that gives us some insight. All such models have built in assumptions and we must train ourselves to think abut these carefully. We must question the abstractions of the messiness of reality that led to the model and be prepared to adjust the modeling process if the world as experienced is different from what the model leads them to expect. There are three primary sources of error when we build models. First, there is the error we make when we abstract from reality; we make choices about which things we are measuring are important. We make further choices about how these things relate to one another. Perhaps we model this with mathematics, diagrams, words etc; whatever we choose to do, there is error we make. This is called Model error. Then, we make error when we use computational tools to solve our abstract models which is called Truncation error. Finally, since we can not store numbers exactly in any computer system, there is always a loss of accuracy because of this. This is called Round Off error. All three of these errors are always present and so the question is how do we know the solutions our models suggest relate to the real world? We must take the modeling results and go back to original data to make sure the model has relevance.
With all this said, let’s start looking at the fundamental building blocks of information transmission in a living neural system. The neurotransmitters and other signaling molecules to accomplish coordination of effort between individual cells were laid down probably in Cambrian times, so these pathways are very old. However, control of a multicellular organism such as ourselves does not necessarily require a neural architecture like ours. Recent work in the Ctenophore genome has shown us a new neural architecture for control (see (Moroz et al. 2014), (Callaway 2014), (Staff 2013) and (Ryan et al. 2013)). Also, single cells can be controlled in what appears to be the same way a multicellular organism can be as is seen in the video (Nautilus 2014). This implies our framework for understanding how to build useful neural architectures for our goals (understanding cognition, drug treatment, autonomous movement and so forth) does not have to be tied to the architectures we see in invertebrate and vertebrate animals. This implies we are not limited to ideas from invertebrate and vertebrate physiology to solve problems in computational cognition modeling.
The physical laws of ion movement
We start with a simple cell and ion movement in and out of the cell ((Johnson and Wu 1995), (Weiss 1996a) and (Weiss 1996b)). These references provide even more detail and you should feel free to look at these books. However, the amount of detail in them can be overwhelming, so we offer a short version with just enough detail for our mathematical/ biological engineer and computer scientist audience.
An ion c can move across a membrane due to several forces. First, let’s talk about what concentration of a molecule means. For any molecule b, the concentration of the ion is denoted by the symbol [b] and is measured in \(\frac {molecules}{liter}\). Now, we hardly ever measure concentration in molecules per unit volume; instead we use the fact that there are N _{ A }=6.02 × 10^{23} molecules in a Mole and usually measure concentration in the units \(\frac {Moles}{cm^{3}} \: = M\) where for simplicity, the symbol M denotes the concentration in Moles per c m ^{3}. The special number N _{ A } is called Avogadro’s Number. The force that arises from the rate of change of the concentration of molecule b acts on the molecules in the membrane to help move them across. The amount of molecules that move across per unit area due to this force is labeled the diffusion flux as flux is defined to a rate of transfer \(\left (\frac {something}{second}\right)\) per unit area.
There are several basic laws to consider. Ficke’s Law of Diffusion is an empirical law which says the rate of change of the concentration of molecule b is proportional to the diffusion flux and is written in mathematical form as \(J_{\textit {diff}} =  D \: \frac {\partial \: [b]}{\partial x}\) where J _{ diff } is diffusion flux which has units of \(\frac {molecules}{cm^{2}second}\), D is the diffusion coefficient which has units of \(\frac {cm^{2}}{second}\) and [b] is the concentration of molecule b which has units of \(\frac {molecules}{cm^{3}}\). The minus sign implies that flow is from high to low concentration; hence diffusion takes place down the concentration gradient. Note that D is the proportionality constant in this law.
Ohm’s Law of Drift relates the electrical field due to an charged molecule, i.e. an ion, c, across a membrane to the drift of the ion across the membrane where drift is the amount of ions that moves across the membrane per unit area. In mathematical form J _{ drift }=− ∂ _{ el } E where it is important to define our variables and units very carefully. We have J _{ drift } is the drift of the ion which has units of \(\frac {molecules}{cm^{2}second}\) and ∂ _{ el } is electrical conductivity which has units of \(\frac {molecules}{voltcmsecond}\). Now the valence of ion c is the charge on the ion as an integer; i.e. the valence of C l ^{−} is −1 and the valence of C a ^{+2} is +2. We let the valence of the ion c be denoted by z. It is possible to derive the following relation between concentration [c] and the electrical conductivity ∂ _{ el }:∂ _{ el }=μ z [c] where dimensional analysis shows us that the proportionality constant μ, called the mobility of ion c, has units \(\frac {cm^{2}}{voltsecond}\). Hence, we can rewrite Ohm’s Law of Drift as \(J_{\textit {drift}} =  \mu z [c] \: \frac {\partial V}{\partial x}\). We see that the drift of charged particles goes against the electrical gradient.
There is also a relation between the diffusion coefficient D and the mobility μ of an ion which is called Einstein’s Relation. It says \(D = \frac {\kappa T}{q} \: \mu \) where κ is Boltzmann’s constant which is \(1.38 \: 10^{23} \frac {joule}{\circ K}\), T is the temperature in degrees Kelvin and q is the charge of the ion c which has units of coulombs. We note electrical work has units of coulombs  volts. Hence, we see \(\frac {\kappa T}{q} \: \mu \) has units \(\frac {voltcoulomb}{\circ K} \: \frac {\circ K}{coulombs} \: \frac {cm^{2}}{voltsecond} \: = \: \frac {cm^{2}}{sec}\) which reduces to the units of D. Further, we see that Einstein’s Law says that diffusion and drift processes are additive because Ohm’s Law of Drift says J _{ drift } is proportional to μ which by Einstein’s Law is proportional to D and hence J _{ diff }.
The membrane capacitance of a typical cell is one micro fahrad per unit area. Typically, we use F to denote the unit fahrads and the unit of area is c m ^{2}. Since the inside and outside of the cell are separated by a biological membrane of the type, we can ask what if one side of the cell had more or less ions than the other? These uncompensated ions would produce a voltage difference across the membrane because charge is capacitance times voltage (q = c V). Hence, if we wanted to produce a 100 mV potential difference across the membrane, we can compute how many uncompensated ions, δ[c] would be needed: \(\delta [c] = \frac {10^{6}F}{cm^{2}} \: \times \:.1 V = \: 10^{7} \: \frac {coulombs}{cm^{2}}\). This is a typical voltage difference across a biological membrane in an excitable nerve cell. For a typical cell the total number of ions inside the cell for a.5M solution is ≈10^{16} ions. We can show the number of uncompensated ions per cell to get this voltage difference is 4.95 × 10^{7} ions which is only ≈ 10^{−7} % of the total. Hence, the voltage differences relevant to excitable nerve cell are achieved by moving a tiny fraction of the ions available across the membrane.
The Nernst  Planck equation
where we let [ c]_{ out } be the concentration of c outside the cell and [ c]_{ in } be the concentration of c inside. This important equation is called the Nernst equation and is an explicit expression for the equilibrium potential of an ion species in terms of its concentrations inside and outside of the cell membrane. For example, in frog muscle, K ^{+} has an interior concentration of 124.0 mM and an outer concentration of 2.25 mM leading to an equilibrium voltage of −101.52 mV.
Electrical signaling
Hence, if we are given the needed conductance ratios, we can compute the membrane voltage at equilibrium for multiple ions.
Ion flow
Our abstract cell is a spherical ball which encloses a fluid called cytoplasm. The surface of the ball is actually a membrane with an inner and outer part. Outside the cell there is a solution called the extracellular fluid. Both the cytoplasm and extracellular fluid contain many molecules, polypeptides and proteins disassociated into ions as well as sequestered into storage units. We are now interested in what this biological membrane is and how we can model the flow of ionized species through it. This modeling is difficult because some of the ions we are interested in can diffuse or drift across the membrane and others must be allowed entry through specialized holes in the membrane called gates or even escorted, i.e. transported or pumped, through the membrane by specialized helper molecules.
Excitable cells

When the voltage across the membrane is above the resting membrane voltage, we say the cell is depolarized.

When the voltage across the membrane is below the resting membrane voltage, we say the cell is hyperpolarized.
These gates transition from resting to open when the membrane depolarizes due to an incoming voltage pulse. In detail, the probability that the gate opens increases upon membrane depolarization. However, the probability that the gate transitions from open to closed is NOT voltage dependent. Hence, no matter what the membrane voltage, once a gate opens, there is a fixed probability it will close again.
Hence, an action potential can be described as follows: when the cell membrane is sufficiently depolarized, there is an explosive increase in the opening of the sodium gates which causes a huge influx on sodium ions which produces a short lived rapid increase in the voltage across the membrane followed by a rapid return to the rest voltage with a typical overshoot phase which temporarily keeps the cell membrane hyperpolarized.
Next, let’s look at how these depolarizing pulses come about.
The cable model
We begin with a simple model of a biological cell. We can think of a cell as having an input line (this models the dendritic tree), a cell body (this models the soma) and an output line (this models the axon). We could model all these elements with cables – thin ones for the dendrite and axon and a fat one for the soma. To make our model useful, we need to understand how current injected into the dendritic cable propagates a change in the membrane voltage to the soma and then out across the axon. In a uniform isolated cell, the potential difference across the membrane depends on where you are on the cell surface. Now we wish to find a way to model V ^{ m } as a function of the distance downstream from site at which a current is injected into the cable and also in terms of the the time elapsed since current injection. This model will be called the Core Conductor Model.
The core conductor model assumptions

t is time usually measured in milliseconds or mS.

z is position usually measured in cm.

K _{ e }(z,t) is the current per unit length across the outer cylinder due to external sources applied in a cylindrically symmetric fashion. This is usually measured in \(\frac {amp}{cm}\).

K _{ m }(z,t) is the membrane current per unit length from the inner to outer cylinder through the membrane. This is also measure in \(\frac {amp}{cm}\).

V _{ i }(z,t) is the potential in the inner conductor measured in millivolts or mV.

V _{ m }(z,t) is the membrane potential measured in millivolts or mV.

V _{ o }(z,t) is the potential in the outer conductor measured in millivolts or mV.

I _{ o }(z,t) is the total longitudinal current flowing in the +z direction in the outer conductor measured in amps.

I _{ i }(z,t) is the total longitudinal current flowing in the +z direction in the inner conductor measured in amps.
 1.
The cell membrane is a cylindrical boundary separating two conductors of current called the intracellular and extracellular solutions. We assume these solutions are homogeneous, isotropic and obey Ohm’s Law.
 2.
All electrical variables have cylindrical symmetry.
 3.
A circuit theory description of currents and voltages is adequate for our model.
 4.
Inner and outer currents are axial or longitudinal only. Membrane currents are radial only.
 5.
At any given position longitudinally (i.e. along the cylinder) the inner and outer conductors are equipotential. Hence, potential in the inner and outer conductors is constant radially. The only radial potential variation occurs in the membrane.

r _{0} is the resistance per unit length in the outer conductor measured in \(\frac {ohm}{cm}\).

r _{ i } is the resistance per unit length in the inner conductor measured in \(\frac {ohm}{cm}\).

a is the radius of the inner cylinder measured in cm.
It is also convenient to define the current per unit area variable J _{ m }: J _{ m }(z,t) is the membrane current density per unit area measured in \(\frac {amp}{cm^{2}}\).
Building the core conductor model
The transient cable equations

\(v_{m} = V_{m}(z,t) \:  \: {V_{m}^{0}}\) is the deviation of the membrane potential from rest.

\(i_{i} = I_{i}(z,t) \:  \: {I_{i}^{0}}\) is the deviation of the current in the inner fluid from rest.

\(i_{o} = I_{o}(z,t) \:  \: {I_{o}^{0}}\) is the deviation of the current in the outer fluid from rest.

\(v_{i} = V_{i}(z,t) \:  \: {V_{i}^{0}}\) is the deviation of the voltage in the inner fluid from rest.

\(v_{o} = V_{o}(z,t) \:  \: {V_{o}^{0}}\) is the deviation of the voltage in the outer fluid from rest.

\(k_{m} = K_{m}(z,t) \:  \: {K_{m}^{0}}\) is the deviation of the membrane current density from rest.
The new constants τ _{ m } and λ _{ c } are very important to understanding how the solutions to this equation will behave. We call τ _{ m } the time constant and λ _{ c } the space constant of our cable. To understand the Time Constant, consider the ratio \(\frac {c_{m}}{g_{m}}\). Note the ratio has units of seconds and thus, we can interpret this constant as the time constant of the cable, τ _{ m }. Note that τ _{ m } is a constant whose value is independent of the size of the cell; hence it is a membrane property. We can show the time constant can thus be expressed as \(\frac {c_{m}}{g_{m}}\) also. Next consider the Space Constant. Look at the dimensional analysis of the term \(\frac {1}{(r_{i}+r_{o}) g_{m}}\) which has units of cm ^{2}. This is why the square root of the ratio functions as a length parameter. We can look at this more closely. Consider again our approximate model where we divide the cable up into pieces of length Δ z. If we do some standard biophysical analysis, we can show \(\lambda _{c} = \sqrt {\frac {a}{2 \rho _{i} G_{m}}}\). Now ρ _{ i } and G _{ m } are membrane constants independent of cell geometry. So we see that the space constant is proportional to the square root of the fiber radius. Note also that the space constant decreases as the fiber radius shrinks.
Finite cables
The ball and stick model
The calculation of the first Q coefficients is then handled as a linear algebra problem.
Note the differences between the solution we find by using separation of variables techniques where we assume the time and spatial parts of the solution are separated which forces us to use an infinite series approach. This also introduces numerical artifacts and so forth. The solution using transform tools shows a very different time and space dependence in the solution which means the decay of the response to the voltage impulse applied to the cable is actually different than a standard exponential decay. Still, a nice guiding principle is that the response drops roughly e ^{−1} in magnitude every time and space constant we move away from the injection site. Also, please note that we make a large number of simplifying assumptions in order to arrive at these results.
The basic Hodgkin  Huxley model
In terms of membrane current densities, all of the above details come from modeling the simple equation K _{ m }=K _{ c } + K _{ ion } where K _{ m } is the membrane current density, K _{ c } is the current through the capacitative side of the circuit and K _{ ion } is the current that flows through the side of the circuit that is modeled by the conductance term, g _{ m }. We see that in this model \(K_{c} = c_{m} \frac {\partial V_{m}}{\partial t}\) and K _{ ion }=g _{ m } V _{ m }. However, we can come up with a more realistic model of how the membrane activity contributes to the membrane voltage by adding models of ion flow controlled by gates in the membrane. Our models are based on work that Hodgkin and Huxley performed in the 1950’s.
The HodgkinHuxley sodium and potassium model
Hodgkin and Huxley modeled the sodium and potassium gates as
where and are called activation variables and is an inactivation variable which all satisfy the first order Φ kinetics that tell us
where for each ion c
Further, the coefficient functions, α and β for each variable required data fits, such as , as functions of voltage. We will not list the rest here as we do not need that level of detail. Of course these data fits were obtained at a certain temperature and assumed values for all the other constants needed and so they need to be altered if the temperature and ionic concentrations change. Our model of the membrane dynamics here thus consists of the following differential equations:
We note that at equilibrium there is no current across the membrane. Hence, the sodium and potassium currents are zero and the activation and inactivation variables should achieve their steady state values which would be m _{ ∞ }, h _{ ∞ } and n _{ ∞ } computed at the equilibrium membrane potential which is here denoted by V _{0}.
At this point, we see a general model of how to generate an action potential. We do not model the transmission of the action potential to its target neurons; suffice it to say, there are molecular mechanisms that send the pulse to its targets without change. At the interface to the target neuron called the synapse, the incoming voltage pulse generates a Ca ^{+2} current which in turn generates discrete packets of neurotransmitter which are released for processing by the target neurons dendritic subsystem. Note the signal transduction pathway here: the axonal voltage pulse is converted into a discrete release of molecules which bind with the dendritic arbor and thereby shape the next axonal pulse in very nonlinear ways. We are also not interested here in how information might be encoded in collections of axonal spikes. Our interest at the moment is in the low level bones. In general, we separate the effects of these triggers into two classes: first messengers which are essentially voltage activated gates and second messengers which enter through the membrane and initiate a protein transcription event which can rebuild the actual hardware of the cell. An abstract analysis of such a trigger event is presented in (Peterson 2014a) in order to build trigger approximations. Here, we will primarily be interested in second messengers which are neurotransmitters even though the triggers can be more general. There is a vast literature on signaling in a biological neural system and we will point you to just a few relevant sources. We prefer papers and texts that help us understand an overarching theory of the signaling process. Most of these sources are actually fairly old but are full of the technical detail needed to understand the processes being studied. In particular, the use of mathematical points of view is not shied away from and newer references tend to downplay that point of view. Two basic and very useful texts on signaling are (Gerhart and Kirschner 1997) and (Wilkins 2002) which look at these ideas very theoretically. Two other very good resources for this material are (Hille 1992) and (Bray 1998). Bray’s paper is very important and is very useful when you try to understand signaling principles; again, the material is not really dated and it is worth a strong push towards assimilating its ideas. Remember that understanding signaling principles allows us to see how to build reasonable approximations. The paper (Sneppen et al. 2010) is really about protein networks but it has lots of good advice about how to approximate complicated biophysics and then glue together the models into systems. A good overview of how neurons work together to generate high level behavior (which we do not really understand) can be found in (Roberts et al. 2010) which will help you see Black’s points discussed in the next section again phrased in terms of what we currently understand. There is a lot of combinatorial complexity here too and modeling that is hard. Even 4 things taken 2 at a time at each node leads to incredible explosions of computation. A really interesting article on dealing with that is found in (Borisov et al. 2006) which is well worth a read. Another look at the notion of discrete synaptic states and resulting computational complexity is in (Montgomery and Madison 2004) and (HarrisWarrick and Johnson 2010). With all this said, for simplicity, let’s look now at the class of catecholamine neurotransmitters in the nervous system. We will focus on only a few types: DA, dopamine; NE, norepinephrine; and E, epinephrine and lump them together into the category called CAs because they all share a common core biochemical structure, the catechol group.
Modeling issues
“More generally, regulation of transmitter synthesis shares important commonalities in neurons that differ functionally, anatomically, and embryologically. This point is worthy of emphasis. Simply stated, the common biochemical and genomic organization of these diverse populations determines how environmental, epigenetic information, through altered impulse activity, is translated into neural information... cellular biochemical organization, not behavioral modality, is a key determinant of how external stimuli are converted into neural language. In this domain, modes of information storage are biochemically specific, not modality specific, indicating that synaptic systems subserving entirely different behavioral and cognitive functions may share common modes of information processing (boldface our choice)”
“...angiotensin II receptors on the [presynaptic] membrane also modulate norepinephrine release. Angiotensin is a potent vasoconstrictor, derived from... an enzyme secreted by the kidney...the principle is startling: the kidney can communicate with sympathetic neurons through nonsynaptic mechanisms (boldface choice ours)...Circulating hormone regulates transmitter release at the synapse. Synaptic communication, then, may be modulated by nonsynaptic mechanisms, and distant structures may talk to receptive neurons. Consequently aspects of communication with the nervous system are freed from hard–wiring constraints (boldface choice author’s).”
Clearly, dendritic–axonal software interaction should be mediated via pathways of both local scope objects) and global scope, using additional objects which could be modeled after hormones.

Dopamine packets are released into the synaptic cleft and bind to receptors on the dendrite. So, the number of receptors per unit area of dendritic membrane provides a control of dopamine concentration in the cleft.

Dopamine is broken down by enzymes in the cleft all the time which also control dopamine concentration.

Dopamine is pumped back into the presynaptic bulb for reuse providing another mechanism.
“Briefly, certain types of molecules in the nervous system occupy a unique functional niche. These molecules subserve multiple functions simultaneously.... [They] incorporate environmental information into the cell and nervous system. Consequently these molecules simultaneously function as biochemical intermediates and as symbols representing specific environmental conditions.... The principle of multiple function implies that there is no clear distinction among the processes of cellular metabolism, intercellular communication, and symbolic function in the nervous system. Representation of information and communication are part of the functioning fabric of the nervous system.... the brain can no longer be regarded as the hardware underlying the separate software of the mind. Scrutiny will indicate that these categories are ill framed and that hardware and software are one in the nervous system.”
“Shorn of all detail, the software–hardware dichotomy is artificial.... software and hardware are one and the same in the nervous system. To the degree that these terms have any meaning in the nervous system, software changes the hardware upon which the software is based. For example, experience changes the structure of neurons, changes the signals that neurons send, changes the circuitry of the brain, and thereby changes output and any analogue of neural software.”
“Increasing evidence indicates that ongoing function, that is, communication itself, alters the structure of the nervous system. In turn altered structure changes ongoing function, which continues to alter structure. The essential unity of structure and function is a major theme...... In this system, then, signal communication, growth, altered architecture, altered neural function, and memory are causally interrelated; there is no easy divide between hardware and software. The rules of function are the rules of architecture, and function governs architecture, which governs function.... The essential unity of structure and function, of hardware and software, is not restricted to mammals; it is evident in invertebrate nervous systems as well.”
“Any set of elements is relevant only insofar as it processes information and simultaneously participates in ongoing neural function; these dual roles require the neural context. What structural elements may be usefully examined?... First, neural elements of interest must change with environment. That is, environmental stimuli must, in some sense, regulate the function of these particular units such that the units actually serve to represent conditions of the real world. The potential units, or elements of interest, thereby function as symbols representing external or internal reality. The symbols, then, are actual physical structures that constitute neural language representing the real world (boldface our choice). Second, the symbols must govern the function of the nervous system such that the representation itself constitutes a change in neural state. Consequently symbols do not serve as indifferent repositories of information but govern ongoing function of the nervous system (boldface our choice). Symbols in the nervous system simultaneously dictate the rules of operation of the system.... The syntax of symbol operation is the syntax of neural function (boldface our choice).”
“Two related strategies are employed by the nervous system in the manipulation of the transmitter molecular symbols. First, individual neurons use multiple transmitter signal types at any given time. Second, each transmitter type may respond independently of others to environmental stimuli.... The neuron appears to use a combinatorial strategy, a simple and elegant process used repeatedly in nature in a variety of guises. A series of distinct elements, of relatively restricted number, are used in a wide variety of combinations and permutations.”
In our mind, although there is an architecture that specifies connection information between processing nodes, what really counts is how the computation is organized into overlapping computational modules. It is dangerous to think that these neural systems are organized hierarchically. In (Wagner 2014), there is a compelling discussion of hierarchical schemes which is targeted towards problems in homology, the study of how characteristics in different species are similar and could even have evolved from a common ancestor. Wagner says it best:
“We have to emancipate our thinking from the hierarchical concept of how bodies of organisms are organized. In fact, hierarchy never made sense if one thinks of the body as an integrated system that contains differentiated parts. Integration is primary, differentiation is secondary and how the body becomes parceled into modular units does not follow a hierarchical logic (boldface our choice)”
“there is increasing evidence that the gene regulatory network state of a cell is governed not by one core network, but by a mosaic of densely interconnected network modules each of which, in isolation, might look like a core network.”
He goes on further to note that some cell types can be understood as different combinations of gene regulatory modules which is reminiscent of the kind of combinatorial machinations we see in the immune system to generate a response to an intruder ((Shastri et al. 2002) and (Rot and von Andrian 2004)) and in the nervous system ((Black 1991) and (Montgomery and Madison 2004)). Each IOMAP module must therefore be capable of a certain number of active states.
Assembling neurons into networks
Chained architecture details
Each of our directed graphs also has node and edge functions associated with it and these functions are time dependent as what they do depends on first and second messenger triggers, the hardware structure of the output neuron and so forth. We therefore could model neural circuitry using a directed graph architecture consisting of computational nodes N and edge functions E which mediate the transfer of information between two nodes. Hence, if N _{ i } and N _{ j } are two computational nodes, then E _{ i→j } would be the corresponding edge function that handles information transfer from node N _{ i } and node N _{ j }. For our purposes, we will assume here that the neural circuitry architecture we describe is fixed, although dynamic architectures can be handled as sequence of directed graphs. We then organize the directed graph using interactions between neural modules (visual cortex, thalamus etc) which are themselves subgraphs of the entire circuit. Once we have chosen a graph to represent the neural circuitry, note the addition of a new neural module is easily handled by adding it and its connections to other modules as a subgraph addition. Hence, at a given level of complexity, if we have the graph \(\boldsymbol {\mathcal {G}(N,E)}\) that encodes the connectivity we wish to model, then the addition of a new module or modules simply generates a new graph \(\boldsymbol {\mathcal {G^{\prime }}(N^{\prime },E^{\prime })}\) for which there are straightforward equations for explaining how G ^{ ′ } relates to G which are easy to implement.
Conclusions
From our discussions, it should now be clear that modeling information transmission in a neural system is complicated and any attempt to do so requires many approximations. The cable equation arises from many compromises and from the explicit modeling of the cellular processes using approximations based on circuit theory. The HodgkinHuxley model also comes from many approximations and we have only focused on the simplest model: one that contains only sodium and potassium gates. Many more complicated models have been built, of course, but this simple model gets across the gist of it all. We have also focused on one dimensional approximations to biological information transfer and we can also redo all of these approximations in a three dimensional way but that does not add any real additional information. Further, the dendritic arbor can be subdivided into compartments thereby generating what are called compartment models and we do not discuss that level of detail here. However, from the discussions from (Black 1991)(do not be put off by the date of Black’s small book. He identifies many of the problems we still face and reading his book gives one extraordinary perspective), it is very clear that the circuit theory approach is not at all what is really going on.
From one point of view, each neural system is a graph of interacting computational nodes, which in a typical model of neurobiological interest would be neurons, but need not be. Systems from autonomous robotics and gene regulatory networks are of a similar type although the nodal and edge processing are different from what we describe here. See again the book level treatment of homology in (Wagner 2014) and the chemokine grammar discussed in (Rot and von Andrian 2004). In this issue, (Peterson 2014a) explores various ways to approximate the nodal and edge processing functions in these graphs in such a way that they could be efficiently implemented in a functional language such as Erlang which does not have shared memory. Much work has been devoted to trying to understand the information processing principles of various subgraphs in such a model. A general model of brain function can be seen in (Friston 2005) and (Friston 2010) and some general thoughts on fundamental building blocks and motifs in a neural system are discussed in (PereiraLeal et al. 2006), (Lesne 2008) and (Somel et al. 2013). There are many more, but our favorites tend to be those that focus on theoretical principles. For example, there is a nice paper on motifs in (Huang et al. 2007) which we have found very useful.
In addition, there is much to be learned by looking at neural modules and systems that do not come from human neurophysiology. The function of the cortex in humans is a good example. Primary sensory signals are parsed by neural circuitry and associated into neural signals that have a functional use. The mushroom body in insects is a great example of an alternate way to perform these calculations. Also, the size of the mushroom body varies as the insect gets physiologically larger and one can perform studies to see how the animals functions improve with larger mushroom body size (i.e. more neurons and more connections allocated for use in the body). There is a wealth of information on this topic and reading various papers in this area shows us alternate ways to think of information processing – different solutions to the primary problems. We think this is very useful in designing ways to do autonomous robotics for example. There is really no need to think our designs in that area must be based on human neurophysiology. Some good references here are (Farris and Sinakevitch 2003) (basic theoretical background), (Sjöholm et al. 2005) (very good on mushroom body discussions) and (Haehnel and Menzel 2010). Also, birds have an analogue of prefrontal cortex which is useful to think about: see (Herrold et al. 2011) and (Wang et al. 2010). The point here is that trying to answer high level questions about how a neural system leads to behavior and some level of cognition (again useful for autonomous robotics) can be helped by looking at other models. The honeybee is a compact model of cognition and studying it from that point of view is illuminating: see (Menzel 2012).
To try to understand how neural modules hook together, it is also very helpful to look at the principles of brain development across multiple species. The papers (Murakami et al. 2005) and (Baslow 2011) are good examples of this kind of work. Understanding these principles also helps shape our view of how to build approximations to neural processing machinery. For example, what is a good minimal neural system which will exhibit some of the cognitive behaviors we see in small animals? Do we need models of working memory? Do we need emotional subcircuits?
To close, we note the graph model which has individual processing nodes and edge functions allows for each node to be different. At the moment we do not even know how many different types of neuron are in the cortex, thalamus and other structures. The graph models we mention here can deal with many different neuronal nodes and different edge processing approaches in principle. However, we will always have to do lots of approximation. Consider a simple model of a purkinje neuron. It integrates as many as 10,000 or more individual inputs into its dendritic arbor and the individual dendritic fibers can be extraordinarily long as they stretch laterally across the cortical layers. In a standard cable model or compartment cable model, we know there is strong attenuation on the order of e ^{−1} in terms of the space and time constant of the cable. Signals arrive thousands of space constants away from the soma yet the purkinje node can process these handily. Numerical modeling tools can not do this as the attenuation amounts to multiplicative factors on the order of e ^{−N } for large N which gives numerical values below machine zero. We must develop approximate ways to do these calculations which is still a work in progress.

Once we fix the graph G and the node and edge functions, what kind of cognitive systems can we build? This is a nuts and bolts engineering type of question. Our hardware will undoubtedly limit our choice of node and edge function possibilities and so what can we do with that limitation?

In the graph, some edges will be thought of as inhibitory and some as excitatory. Hence, for our choice of node and edge functions where should inhibitory connections be placed to maximize cognitive function?

Since the graph model could be based on mushroom bodies for the associative functionality or even a novel architecture with different signaling systems as in the Ctenophore, what functionality can we achieve for a given choice of node and edge functions in neural systems with different types of association machinery? Are there optimal choices we can capitalize on?

The graph model can easily incorporate hormone influences on the computational node interactions. A nice treatment of how hormones influence brain plasticity is in (GarciaSegura 2009) and reading that makes us realize that triggers on the order of 10^{−4} to 10^{−3} smaller in magnitude than other triggers can influence the neural systems global responses over time frames that exceed weeks. Note the very large time scale range here: a trigger at the mS level is introduced and through a cascade of multiplicative reactions leads to macro level changes at time scales 10^{7} orders of magnitude higher. This range of computation is very difficult to simulate and indicates how important it is for us to design architectures that do well at multiple time scales.

A similar problem occurs if we add glial modifications to the model and it is probably true that changes in glial functioning effect mood and influence some aspects of mental dysfunction (Rajkowska 2000). However, adding glial interaction requires more levels of nonlocal interaction. We do not address that in (Peterson 2014a) but we believe approximations for these effects should be based on low level modeling insights such as we have been discussing here.

In the neural organization that subserves associative function, why is a laminar structure so useful? A clue to that comes for mathematical homology which allows us to find a type of structure which is conserved in graph structure. We can not present any of those details here, but suffice it to say that perhaps the conserved structure that is most efficient occurs when the graph we start with has the right kind of laminar structure. The best tantalizing glimpse into this is in (Ghrist 2007).
Declarations
Authors’ Affiliations
References
 Ansari, A, & Mapp, A (2002). Modular design of artificial transcription factors. Current Opinion in Chemical Biology, 6, 765–772.View ArticleGoogle Scholar
 Baslow, M (2011). The vertebrate brain, evidence of its modular organization and operating system: insights into the brain’s basic units of structure, function, and operation and how they influence neuronal signaling and behavior. Frontiers in Behavioral Neuroscience, 5, 1–7.View ArticleGoogle Scholar
 Black, I. (1991). Information in the Brain: A Molecular Perspective. A Bradford Book. Boston, MA: MIT Press.Google Scholar
 Borisov, N, Markevich, N, Hoek, J, Kholodenko, B (2006). Trading the microworld of combinatorial complexity for the macroworld of protein interaction domains. Biosystems, 83, 152–166.View ArticleGoogle Scholar
 Bray, D (1998). Signalling complexes: Biophysical constraints on intracellular communication. Annual Review of Biophysics and Biomolecular Structure, 27, 59–75.View ArticleGoogle Scholar
 Callaway, E (2014). Jelly genome mystery. Nature, 509, 411.View ArticleGoogle Scholar
 Capuron, L, & Miller, A (2004). Cytokines and psychopathology: Lessons from interferon α. Biol. Psychiatry, 56, 819–824.View ArticleGoogle Scholar
 Edwards, L, Zamitsyna, V, Hood, J, Evavold, B, Zhu, C (2012). Insights into t cell recognition of antigen: significance of twodimensional parameters. Frontiers of Immunology, 3, 1–9.Google Scholar
 Esvelt, K, Mali, P, Braff, J, Moosburner, M, Yaung, S (2013). Orthogonal cas9 proteins for rnaguided gene regulation and editing. Nature Methods, 10, 1116–1121.View ArticleGoogle Scholar
 Farris, S, & Sinakevitch, I (2003). Development and evolution of the insect mushroom bodies: towards the understanding of conserved developmental mechanisms in a higher brain center. Arthopod Structure and Development, 32, 79–101.View ArticleGoogle Scholar
 Friston, K (2005). Models of brain function in neuroimaging. In: Schacter, D, & Kazdin, A (Eds.) In Annual Review of Psychology vol. 56. www.annualreviews.org. Annual Reviews, Palo Alto, CA, New York, (pp. 57–87).Google Scholar
 Friston, K (2010). The freeenergy principle: a unified brain theory?Nature Reviews: Neuroscience, 11, 127–138.View ArticleGoogle Scholar
 GarciaSegura, L (2009). Hormones and Brain Plasticity.Google Scholar
 Gerhart, J, & Kirschner, M. (1997). Cells, Embryos and Evolution: Towards a Cellular and Developmental Understanding of Phenotypic Variation and Evolutionary Adaptability. USA: Blackwell Science.Google Scholar
 Ghrist, R (2007). Barcodes: The persistent topology of data. Bulletin (New Series) of the American Mathematical Society, 45(1), 61–75.MathSciNetView ArticleGoogle Scholar
 Haehnel, M, & Menzel, R (2010). Sensory representation and learningrelated plasticity in mushroom body extrinsic feedback neurons of the protocerebral tract. Frontiers in Systems Neuroscience, 4, 1–13.View ArticleGoogle Scholar
 HarrisWarrick, R, & Johnson, B (2010). Checks and balances in neuromodulation. Frontiers in Behavioral Neuroscience, 4, 1–9.Google Scholar
 Herrold, C, PalomeroGallagher, N, Hellman, B, Kröner, S, Theiss, C, Güntürkün, O, Zilles, K (2011). The receptor architecture of the pigeons nidopalladium caudolaterale: and avian analague to the mammalian prefrontal cortex. Brain Structure Function, 216, 239–254.View ArticleGoogle Scholar
 Hille, B (1992). Ionic Channels of Excitable Membranes.Google Scholar
 Hodgkin, A, & Huxley, A (1952). Currents carried by sodium and potassium ions through the membrane of the giant axon of loligo. Journal of Physiology (London), 116, 449–472.View ArticleGoogle Scholar
 Hodgkin, A (1952). The components of membrane conductance in the giant axon of loligo. Journal of Physiology (London), 116, 473–496.View ArticleGoogle Scholar
 Hodgkin, A (1954). The ionic basis of electrical activity in nerve and muscle. Biological Reviews, 26, 339–409.View ArticleGoogle Scholar
 Huang, C, Cheng, C, Sun, C (2007). Bridge and brick network motifs: Identifying significant building blocks from complex biological systems. Artificial Intelligence in Medicine, 41, 117–127.View ArticleGoogle Scholar
 Johnson, D, & Wu, M (1995). Foundations of Cellular Neurophysiology.Google Scholar
 Lesne, A (2008). Robustness: confronting lessons from physics and biology. Biological Reviews, 83, 509–532.Google Scholar
 Menzel, R (2012). The honeybee as a model for understanding the basis of cognition. Nature Reviews: Neuroscience, 13, 758–768.View ArticleGoogle Scholar
 Moroz, L, Kocot, K, Citarella, M, Dohung, S, Norekian, T, Povolotskaya, I, Grigorenko, A, Dailey, C, Berezikov, E, Buckley, K, Ptitsyn, A, Reshetov, D, Reshetov, D, Swore, J, Girado, D, Fodor, A, Gusev, F, andd R. Bruders, RS, Kittler, E, Mills, C, Rast, J, Derelle, R, Solovyev, V, Kondrashov, F, Swalla, B, Sweedler, J, Rogaev, E, Halanych, K, Kohn, A (2014). The ctenophore genome and the evolutionary origins of neural systems. Nature, 510, 109–120.View ArticleGoogle Scholar
 Montgomery, J, & Madison, D (2004). Discrete synaptic states define a major mechanism of synapse plasticity. Trends in Neuroscience, 27, 1–7.View ArticleGoogle Scholar
 Murakami, Y, Uchida, K, Rijli, F, Kuratani, S (2005). Evolution of the brain development plan: Insights form agnathans. Developmental Biology, 280, 249–259.View ArticleGoogle Scholar
 Nautilus, E (2014). Stunning Siphonophore. https://www.youtube.com/watch?v=8KZsrDGLUJQ, Published: Jun 27, 2014, Accessed: October 28, 2014.
 Peterson, J (1991). Nonlinear iterative methods for approximating the solution to the learning problem in arbitrary feed forward networks. In The Proceedings of the 2 ^{ n d } Workshop on Neural Networks: Academic/Industrial/NASA/Defense. www.ces.clemson.edu/~petersj/CurrentPapers/nonlin2.pdf. NASA, (pp. 143–151).
 Peterson, J (2014a). Nodal computation approximations in asynchronous cognitive models. Computational Cognitive Science, 1. doi:10.1186/s404690150003z.Google Scholar
 Peterson, J (2014b). Subgraph training algorithms and a cognitive dysfunction modeling framework. Technical report, Clemson University, Clemson, SC USA. www.ces.clemson.edu/~petersj/CurrentPapers/ALCAI201509072014.pdf.
 PereiraLeal, J, Levy, E, Teichmann, S (2006). The origins and evolution of functional modules: lessons from protein complexes. Phil. Trans. Royal Soc. B, 31, 507–517.View ArticleGoogle Scholar
 Rajkowska, G (2000). Postmortem studies in mood disorders indicate altered numbers of neurons and glial cells. Biological Psychiatry, 48, 766–777.View ArticleGoogle Scholar
 Rall, W. (1977). 3. Core Conductor Theory and Cable Properties of Neurons, (pp. 39–67). London: Oxford University Press.Google Scholar
 Roberts, A, Li, W, Soffe, S (2010). How neurons generate behavior in a hatchling amphibian tadpole: an outline. Frontiers in Behavioral Neuroscience, 4, 1–11.Google Scholar
 Rot, A, & von Andrian, U (2004). Chemokines in innate and adaptive host defense: basic chemokinese grammar for immune cells. Annual Review Immunology, 22, 891–928.View ArticleGoogle Scholar
 Rusk, N (2014). Orthogonal logic gates. Nature Methods, 11, 130.Google Scholar
 Ryan, J, Pang, K, Schnitzler, C, Nguyen, A, Moreland, R, Simmons, D, Koch, B, Francis, W, Havlak, P, Smith, S, Putnam, N, Hannock, S, Dunn, C, Wolfsberg, T, Mullikin, J, Martindale, M, Baxevanis, A (2013). The genome of the ctenophore mnemiopsis leidyi and its implications for cell type evolution. Science, 342, 1336–1344.View ArticleGoogle Scholar
 Shastri, N, Schwab, S, Serwold, T (2002). Producing nature’s gene chips: the generation of peptides for display by mhc class i molecules. Annual Review Immunology, 20, 463–493.View ArticleGoogle Scholar
 Sneppen, K, Krishna, S, Semsey, S (2010). Simplified models of biological networks. Annual Review of Biophysics, 39, 43–59.View ArticleGoogle Scholar
 Somel, M, Liu, X, Khaitovich, P (2013). Human brain evolution: transcripts, metabolites and their regulators. Nature Reivews: Neuroscience, 112, 112–127.View ArticleGoogle Scholar
 Staff, S (2013). Nervous system may have evolved twice. Science, 339, 391.Google Scholar
 Sjöholm, M, Sinakevitch, I, Ignell, R, Strausfeld, N, Hansson, B (2005). Organization of kenyon cells in subdivisions of the mushroom bodies of a lepidopteran insect. The Journal of Comparative Neurology, 491, 290–304.View ArticleGoogle Scholar
 Wagner, G. (2014). Homology, Genes, and Evolutionary Innovation. Princeton, NJ: Princeton University Press.View ArticleGoogle Scholar
 Wang, Y, BrzozowshaPrechtl, A, Karten, H (2010). Laminary and columnar auditory cortex in avian brain. Proceedings of the National Academy of Sciences, 107(28), 12676–12681.View ArticleGoogle Scholar
 Werbos, P (1987). Learning how the world works. In Proceedings of the 1987 IEEE International Conference on Systems, Man and Cybernetics, (Vol. Volume 1. IEEE Press, New York, pp. 302–310).Google Scholar
 Weiss, T. (1996a). Cellular Biophysics: Volume 1, Transport. Boston, MA: MIT Press.Google Scholar
 Weiss, T. (1996b). Cellular Biophysics: Volume 2, Electrical Properties. Boston, MA: MIT Press.Google Scholar
 Wilkins, A. (2002). The Evolution of Developmental Pathways. New York: Sinauer Associates, Inc.Google Scholar
Copyright
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.