THE COMMUNICATION AND INFORMATION TliEORY ASPECTS OF THE NERVOUS SYSTEM By Eugene Agalides Head of the Biophysical Communications Laboratory Paper presented at the 17th Annual Co2nference in Engineering in Medicine and Biology, November 18, 1964, Cleveland, Ohio Appro-ved f or DatO General Dynamics/Electronics Biophysical Communications Laboratory Research Departmen4t Rochester, New York THE COMMUNICATION AND INFORMATION THEORY ASPECTS OF THE NERVOUS SYSTEM by Eugene Agalides Paper presented at the 17th Annual Confereiice in Engineering in Medicine and Biology, November 18, 1964, Cleveland, Ohio. The quest for a quantitati2ve expression of natural phenomena has lead, in communication theory, to the definition of a quantity of infor- mation. The fact that information theory is today applied to several fields other than electrical communications underlines its importance. Information theory fulfills the needs of these various fields for quanti- tative organization because they present characteristics very similar 2 to those of telecommunication. In the case of the nervous system, for instance, communication is present in the form of impulses (stimuli) conveyed to the central nervous system (CNS) for integration, detection, and computation, or from the CNS to the effectors (muscles, internal glands, etc. Having established the link between information theory and the 2 biophysical sciences, we are led to regard the central nervous system as a gigantic information processing device. There are two ways to consider the phenomenon of nervous communi- cation. First, it may be thought of as a problem in communication itself *This research is sponsore jointly by the Air Force Office of Scientific AF 849(638) Research, Information System Directorate under Contract -1185, by the Office of Naval Research, Biology Branch under Contract Nonr 3993(00); and by General Dynamics/Electronics, Research Department. 2 in which noise is to be eliminated from the message content through the application of coding and redundancy. Second, nervous communication may be thought of as a problem in data processing where the messages must be stripped of their redundant parts in, order to be accommodated in the CNS. These two problems are not cont2ra@ictory. The first one concerns the retrieval of information from noise; i.e. , the coding of information to be transmitted, with a consequent increase in reliability. The second problem concerns the processing of data or the coding of the source information so that redundancy is eliminated. As an analogy, the transmission of English language over a noisy channel can be 2 considered. Before messages are transmitted on the line, all re - dundancies pertaining to the language have to be removed; when this is done, the messages could be coded to combat noise. The duality of this problem might raise the question: Why does an organism need an elaborate coding scheme if only the pertinent infor- mation is used by its CNS? The answer could lie in t2he possibility that the data proce ssing is not done only at the CNS but also in the ganglions present along the transmission path. Thus, there would be a great need for efficient coding close to the periphery of the sensorial mechanism. As the information travels up to the CNS, this need might diminish as far as the processing is concerned. But, processing is sti6ll needed as far as the noise is concerned s@-nce it is reasonable to 3 assume that the noise increases as the information approaches the CNS. (Noise, as used here, means all unwanted information con- verging on the CNS at the same time as the wanted information.) To illustrate this attributed quality of the ga'nglion, it is only necessary to recall that a constant s2timulus, after a certain period of time, does not elicit a response in the neuron. We may postulate that this saving in transmission is accomplished by an accommodation phenomenon in the nervous system. In order to understand the processes involved in neuron firing, various mathematical models have been evolved such as those by Goldstein,l Rosenblith,2 and Agalides.3 Goldstein and Rosenblith have represented neuron activity as a resultant of the summation of responses from a whole population of nervous fibers exhibiting the same characteristics. This approach leads to a microscopic view of neural mechanism.4 Thu's, a modified shot noise effect is brought in the picture. To organize the amount 2of data available and to extricate from the apparently random nature of the data the statistical parameters necessary for the understanding of the processes at hand, constitute another scientific challenge. We might mention along these lines the contribution already given by spectral analysis to the interpretation of data, as expounded by Wiener5 and Brazier,6 w1here autocorrelation 4 techniques have helped to detect low frequency phenomena in Electro- Encephalography and to assign different locking frequencies for the various rhythmic activities of the brain (a -rhythm - 10 cps, P -rhythm 20-25 cps, 'Y -rhythm= 40 -60 cps, 5 -rhythm = 1-2 cps, e-rhythm 4-7 cps, k (kappa) -rhythm -L, 10 cps). Throughout the experimental work on the nervous system, both macroscopic and microscopic considerations tend to influence the development of theoretical explanations in nervous transmission in separate, although well-defined, directions. In both cases, we are led to consider the problem of coding and redundancy. These concepts arise when we study the ability of a rece2iver to detect accurately the nature of the message being sent in the presence of interferring noise. In applying information theory to the nervous system, this problem becomes complicated by two factors not usually encountered in usual telecommunication practice. One, already mentioned above, reflects the difference between macroscopic and microscopic phenomena. The other, which is6 perhaps more complex involves the presence of both spatial and temporal effects in coding. It may be said that the presence of both macroscopic and micro- scopic points of view should not necessarily lead to a conflict. This is true in thermodynamics where the former leads to analysis on the basis of ensemble averages, whereas the latter illustrates the dynamics 5 of individual particles. The study of the nervous system is virtually carried out on this dual basis. However, if we want to analyze com- munication in the nervous system from the point of view of coding, a conceptual difficulty arises because coding and redundancy do not by necessity belong to either a macroscop2ic or a microscopic mode. 6 Dr. Brazier, in affect, has suggested that the existence of a multi- coding scheme in the nervous system might explain the conveyance of both temporal and spatial information. The existence of multicoding in the transmission of stimuli to the central nervous system and the macroscopic and microscopic aspects 2 of stimuli transmission are studied. The approaches to the study of these aspects of stimuli transmission are: 1. Microscopic Approach. The transmission of equal stimuli along different individual fibers of the same nervous bundle are studied, and a signal is observed as it travels along the path of the nervous bundle and as it is interpreted by2 the central nervous system or an integrating organ. 2. MacroscoLic Approac The transmission of equal stimuli along different nervous bundles is studied again with observation of a signal as it travels along an integrating path, and an analysis is made of the signal as it is interpreted by the central ne8rvous system or an integrating organ. 6 Two types of coding are investigated. In the microscopic approach we attempt to verify that the signal traveling along similar fibers is not necessarily transmitted by similar coding or modulating schemes, but that a multitype of coding is used to provide better noise recording properties. In the macroscopic approach we are con2cerned with the integration of possible different codes along a complex nervous path, again with a possible gain in information capacity through improved noise contributing properties. The two approaches have been defined above in broad terms. Note that we have not mentioned the problem of distinguishing between temporal and spatial types of coding. This is because we ar2e only concerned here with the problem of "how" the stimuli are being coded, not with "what" is being coded. This latter problem could only be studied after some definite forms of coding have been established. To find these coding forr@s we neglect, as a first approximation, another aspect of nervous communication; i. e. , the orientation of neural struc8tures. It is evident that the outlined approaches are of a general nature. As we consider the vast array of neural structures, we find differences in functions. In the nervous system we are faced with two problems derived from the specific needs of information transmission. One is to find how 7 to obtain maximum efficiency. The other is how to obtain minimum equivocation. The first problem calls for making maximum use of the capacity of the transmission channels. The second is to find the way to -minimize the noise effect, environment, and threshold vari- ation in order to overcome equivocation or misunderstanding. 2 In the first case, a multiple coding system would be beneficial. Here the term "multiple" means not only the coding in itself but also the system as such. For example, the use of pulse position modulation, pulse duration modulation, and pulse amplitude modulation could be one way to solve the problem of transmitting three different messages over one and the same channel. The effect o2f noise and the resulting equivocation could be determined by using three or more types of coding systems for one and the same message. Each coded message could then be transmitted over separate channels or over a single channel, depending on the reliability required. See Fig. 1. A particular noise at a time "t" will not affect all three different modulation system2s in the same way and to the same degree. Three types of modulation and two types of noise are shown in Fig. 2. Any one of the noise types will not affect in the same way a pulse duration modulation system, a pulse position modulation system, and a pulse amplitude modulation system. Figure 3 shows a modulation system with three degrees of freedom. For a modulation syst8em with three degrees of freedom, the modulated wave can be represented by: pplp- 8 Filtering and Detection Coincidence and Channel x 2 Decision Element DI Transmission Point Noise In Channel z Information N c OUT 10 2 c D 3 System I Fiitei-ing and 2 Detection Coincidence and Decision Element Transmission D, Point Combined Channel Infc x +z +k c Information 10 IN 2 OUT System II Fig. 1. Multichannel multicoding and unichannel-multicoding transmission and reception systems. (PDM) x (PPM) z K3V3 Kiv K zvz 2 (PAM) k N.oise (Type A) L T Noise n n n (Type B) Fig. 2. PDM, PPM, PAM, and two Fig. 3. Pulse 7modulation with types of noise transmitted three degrees of freedom simultaneously. (applying PDM, PPM, and PAM at the same time). 9 t 0 t K V (T n) 2An M (t) + 0 3 3 + - T T T 2AnK3V3 (Tn) o 2 2 + + z [i + K3V3 (Tn)] T M=l M -rr sin mw + A ] Cos (nu") t niw B c n c c n where 2An -KIV, (Tn- K2V2 (T n) 2 KIV, (Tn) + K2Vz (T n) Bn - 2 * +B K IV, (T 2 n n n *n -An K?.Va (T n) 1 d a,, (n) - [I + K3V3 (T dt Ty n c 2 1 + K3V3 (Tn) to + 2An T 2 [1 + K3V (n) .1 (Tn)] m MT2r sin n-iw [to + A cos niw B c 2 c n 2 [1 + K3V3 (T b (n) m MTr 8 t s i rk,)cBnsin rnwc 0 +A n] 2 Popp' 10 The unmodulated pulses are ideal rectangular pulses with time dur- ation top amplitude unity, and repetition period T. The leading edge is modulated with modulation factor K, and the trailing edge with a modulation factor Ka. VI (t) and V. (t) are independent modulating wa2ves; n = 0, ± 1, ± 2; and Tn, and Tnz denote the respective times in the nth period that VI(t) and V. (t) are sampled. We may assume that Tn, = Tn. = Tn, With uniform sampling, Tn, P Tn, , . . . Tnn can be replaced by Tn. The terms am(n) and bm(n) are constants over any one period T, but may vary from period to period. Also 2 w c = 2 Tr / T. F inally, K. 3is a modulation factor applied to the amplitude of the pulse. A modulation form prevalent in living organisms is Pulse Repetition Rate Modulation (PRRM). This type of modulation is a digital equivalent of the frequency modulation of an analog signal. A form of pulse repetition rate modulation is found in the 2sp@nal cord. In the skin receptors, transmission of stimuli seems to occur in the form of a pulse position modulation system. Graded signals in the nervous system seem to belong to an amplitude modulation system. We can now state that in complex living organism like vertebrates there are many modulation systems used for transmitting information. In 6addition there are different types of coding. The name of multi- dimensional coding could be applied to the form in which different coded signals are transmitted over one and the same channel. at the same time. Brazier mentioned the multitude of codes existing in the nervous sys- tem at the 1960 Moscow Conference on Higher Nervous Activity. It can be shown that the multiple coding system represents an economy in the trans- mission channels in the case of different §timuli. On the other hand,.if the same stimulus is transmitted by a multi-channel multi-coding system, there 2 is a definite advantage as compared with the transmission of the stimulus by a single coding system. This is true regardless of how good a single coding system can be because it can be one of the multiple coding systems. To ascertain the existence of different types of codes, we investigated the response of different stimuli on the skin sensory receptors. The first receptor to be -studied was the Pacinian Corpu2scle (Fig. 4). Agalides com- pared the elicited responses to mechanical and acoustic stimuli.7 The f7 Fig. 4. Photomicrograph of a Pacinian Corpuscle, 4OX. (Nikon Interference Phase Microscope, Nikon Microflex Camera, Tri-Pan-X Kodak Film; exposure time2, 0. 5 sec.) ppo- 12 difference between the formamplitude and latency involved for each response was shown (Figs. 5, 6, 7). The power and frequency curves of the trans - ducers were plotted. There is a definite difference between the response of the same receptor to mechanical and to acoustical stimuli. This was an 2 investigation in vitro. If the coding in the nervous system in vivo should be studied, then either gross recording electrodes or microelectrodes can be used. Him ME-E, Fig. 5. Impulses from a Pacinian Corpuscle; stimulus, mechanical pressure; vertical scale, 220 @Lv/division (large scale); horizontal scale, 10 ms/division (large scale). Stimulus St ulus Figs. 6 and 7. Effect of acoustical wave pressure on Pacinian Corpuscle. B and K heterodyne oscillator and amplifier; 70 cps, 2 keyed; 100 v rms on the piezo-electric acoustic transducer, one electrode grounded. Recorded between the Pacinian Corpuscle and first node of Ranvier and the axon of the sensory receptor at a distance of 7 mm from the corpuscle. Vertical scale, 10 @Lv/division (large scale); horizontal 0 scale, I ms/division (large scale). 1 3 The damage done to the surrounding cells by inserting electrodes into the nervous system cannot be overlooked. Another approach for studying in vivo coding processes in living organisms is via an in- vestigation of the electrical activity of electric fishes. Figure 8 shows some representatives of gymnotid fishes from trop2ical and equatorial South America and mormyrid fishes from tropical and equatorial Africa. Some of the fishes having electric organs use them for offensive and defensive purposes. These are strong electric fishes whose discharge can paralize prey or at least scare away enemies. Torpedo Nobiliana, a marine electric fish, can discharge electric pulses of some 220 volts. The short-circuit c2urrent of an adult Torpedo can reach 50 amperes. The output capabilities of Electrophorus Electricus, the Electric Eel, (Fig. 9), a fresh-water electric fish, were studied. Voltage discharges of over 600 volts were mea@ured. The maximum current at 10 ohms load resistance was close to one ampere. The pulse peak power was around 100 watts. There are 2other electric fishes, called weak electric fishes, which use electric coded impulses for navigation and communication purposes. Pulse shape and pulse duration are very different de- pending on species. The electric eel has three electric organs, each of which per- forms different functions. One is called the main electric organ and 14 Mormyridie GymnotWas HyperoP sus bebe hmemani rmyrus vmrftn kannume 2 Stemarchorhynchus cu airyrhynchus Gymnorhamphichd7s t*rmyrops ngert hyposwmus 2 carapo attenuatus Eigenmannis macrops Gym niloticus Figure 8. Representative types of the Mormyridae and Gymnotidae. Th5e convergent evolution between these unrelated families is expressed in terms of their electric discharges, reduction of the tail fin, propulsion through an elongated unpaired fin, development of long snouts, and several other features. 1 5 Fig. 9. Electrophorus Electricus (electric eel). produces high voltage pulses used for stunning purposes. Another electric organ, called the organ of Hunter, is used for communication and navi- gation purposes. A third electric organ, called the organ of Sachs, is thought to have an auxiliary function and its electric discharge is 2much weaker than that of the main electric organ. By studying the form and shape of the coded impulses, we can dis- tinguish a large variety of coding systems (Figs. 10, 11). Gymnarchus Niloticus, an electric fish from equatorial and tropical Africa, studied 8 in detail by Lissman and Machin , is one of the most sensitive electric f2ishes. It can sense any disturbance of the surrounding water produced by objects. It can distinguish between electric conductors and micro- conductors. Charges of the electric field of 0. 03 @tv/cm produced by direct currents can be detected by Gymnarchus Niloticus. Its receptor system operates in the second derivative mode. The change in current in the electroreceptors was calculated to about 0.0 003 44a for 1 ms pulses at a frequency of 300 cps. 1 6 0 > 0 @-m5s 1-d 800 Ms Torpedo and Astroscopus Raia + + 0 > 0 3 2 resp. Ms Ms Electrophorus Eigenma2nri,@a and Sternopygus 0 > 0 3 5 Ms Ms Gymnotus (other tha.-. Electrophorus) Gnathonemus 0 0 z 5 d 2 Ms I- rn s NIormyrus 3 Malaptei,urus Fig@ 10. Pulse shape and pulse duration of the discharges of some species of electric and magnetic sensitive fish. (After H. Grundfest, 1952) 17 100 Ms Fig. ii. Cross communication between two electric eels in two separate aquaria; optical galvanometer recorder, dc to 10 kc frequency range, speed 76 mm per second. Gnathonemus Petersii (Figs. 12, 13), an African electric Mormyrid, uti2lizes a pulse position plus pulse amplitude modulation. The slope of the pulse is less than 50 4sec, and the entire pulse duration is 250 FLsec. Sternarchus Albifrons (Figs. 14, 15, 16), a South American Ster- narchid, utilizes a permuted phase plus 'an amplitude modulation coding system. Electrophorus Electricus utilizes a very complicated coding method 5 in conjunct-ion with a modulation system with three degrees of freedom. It was verified that the fish can decode the electric signals emitted by other fish of the same species. The experiment proving this fact was reported at the 1964 Rochester Conference on Data Acquisition and Processing 9 in -Biology and Medicine. 18 By studying the types of coding used by fishes for perhaps twenty or thirty million years, we can once more observe that evolution has solved biochemical, electrical, and mechanical problems of energy transformation by always finding the way to utilize minimum energy combined with excel- lent efficiency. In the case reported on here, evolution f2ound the way to extract signals from noise with little power utilization. The study of the natural phenomena and of living organisms still can provide the scientist with new clues and means to further the general progress of science. Acknowledgement I wish to express my appreciation for the support offered by the Air Force Office of Scientific Research, Information Syste2m Directorate; the Office of Naval Research, Biology Branch; and General Dynamics/Electronics. In particular I wish to thank Dr. H. C. Nedderman, Director of Research for GD/E, for his managerial assistance. Mr. Bernardini was responsible for the excellent microsurgery required to dissect the Pacinian Corpuscle, the electric organs of electric fishes, and their brains. Mr. R. Zinsmeister set u0p the electric equipment and skillfully recorded the electrical activity of electric fishes and that of the skin sensory receptors. Finally I would like to thank Mr. R. Doughty for his help in editing this article. 19 Fig. 12. Gnathonemus Petersii (African electric fish). Fig. 13. Electrical signals emitted by Gnathonemus Petersii. rO 0 20 000-1 Fig. 14. Sternarchus Albifrons (South American electric fish). MUMML Fig. 15. Electrical signals emitted by Sternarct-lus Albifrons. 4 Scales: vertical, 2 mv/div; horizontal, I ms/div. Recording made with two stainless steel electrodes, 7 inches long, placed in the water tank. Recorder: Midwestern Optical Recorder. Amplifiers: (1) Tektronix and (2) specially constructed units. Date of recording: 2 February 1963 Time: 2:00 PM Recorder speed: 60 ips Voltage at electrode: 1 M'v (equals 1. 6-inch deflection on recording paper). i I Voltage at output of second amplifier: 1. 1 v (equals 1-inch deflection on recording paper). Fig. 16. Recording of the electrical activity of Sternarchus Albifrons. 22 BIBLIOGRAPHY 1. Goldstein, M. H. Jr., "A Statistical Model for Interpreting Neuroelectric Responses, " Information and Control, 3, pp. 1-17 (1960). 2. Rosenblith, W. A editor, "Sensory Communication, " MIT Press and John Wiley and Sons, Inc., p. 196 (12961).. 3. Agalides, E., "The Mathematical Model of the Neuron, Rochester Conference on Data Acquisition and Data Processing in Medicine and Biology, Pergamon Press 1962. 4. Adolph, A. R., "Sensory Neurology and Analog" Technical Document LMSD 49752, Lockheed Aircraft Corp. (June 1959). 5. Wiener, N., "Cybernetics, " or "Control and Comm2unication in the Animal and the Machine, " MIT Press and John Wiley and Sons, Inc., Chapt. 10 (1961). 6. Brazier, M.A. B..l " Long-persisting Electrical Traces in the Brain of Man and Their Possible Relationship to Higher Nervous Activity, " Moscow Symposium on Higher Nervous Sensitivity, 1960. 72. Agalides, E., "The Effect of Acoustical Waves on the Pacinian Corpuscle, " Transaction of the New York Academy of Sciences, Vol. , No. April 1964. 8. Lissman, H. W. and K. E. Machin, "The Mechanism of Object Location in Gymnarchus Niloticus and similar Fishes, J. Ex-per. Biology, 15, 451 (1958). 23 Agalides, J. Bernardini, and R, Zirlsrneister, if Infor rnation Processing by Electric Fislies " ., Proceedings Of the 1964 Rochester Conference oil Data Acquis ition and Proce:§sing in ATedicizie and Biology, ed., by 1,@. Elislei n, in press: Pelga,o,l Press, London and New York.