Nonlinear Dynamics of Operant Behavior: A New Approach via the ...

Nonlinear Dynamics of Operant Behavior: A New Approach via the ...

This paper was published in the journal:“Reviews in the Neurosciences”, 13(2002), 31-57.

Nonlinear Dynamics of Operant Behavior: A New Approach viathe Extended Return MapJay-Shake Li* and Joseph. P. HustonInstitute of Physiological Psychology and Center for Biological and MedicalResearch, University of Düsseldorf, GermanyKey words:operant, Skinner-box, reinforcement, rat, nonlinear dynamics, Extended Return Map.Corresponding Author:Jay-Shake LiInstitute of Physiological Psychology 1, University of DüsseldorfUniversität Str. 1, 40225 Düsseldorf, GermanyTel.: 49-211-8113493Fax: 49-211-8112024e-mail:lijay@uni-duesseldorf.de2

SynopsisPrevious efforts to apply nonlinear dynamical tools to the analysis of operant behaviorrevealed some promise for this kind of approach, but also some doubts, since the complexityof animal behavior seemed to be beyond the analyzing ability of the available tools. We hereoutline a series of studies based on a novel approach. We modified the so-called “ReturnMap” and developed a new method, namely, the Extended Return Map (ERM) to extractinformation from the highly irregular time series data, the inter-response-time (IRT) generatedby Skinner-box experiments. We applied the ERM to operant lever pressing data from ratsusing the four fundamental reinforcement schedules: Fixed-Interval (FI), Fixed-Ratio (FR),Variable-Interval (VI) and Variable-Ratio (VR). Our results revealed interesting patterns inall experiment groups. Especially, the FI and VI groups exhibited well organized clusters ofdata points. We calculated the fractal dimension out of these patterns and comparedexperimental data with surrogate data sets, which were generated by randomly shuffling thesequential order of original IRTs. This comparison supported the finding that patterns inERM reflect dynamics of the operant behaviors under study. We then built two models tosimulate the functional mechanisms of the FI schedule. Both models can produce similardistributions of IRTs and the stereotypical “scalloped” curve characteristic of FI responding.However, they differ in one important feature in their formulation: While one model uses acontinuous function to describe the probability of occurrence of an operant behavior, the otherone employs an abrupt switch of behavioral state. Comparison of ERMs showed that only thelatter one was able to produce patterns similar to the experimental results, indicative of theoperation of an abrupt switch from one behavioral state to another over the course of the interreinforcementperiod. This example demonstrated the ERM to be an useful tool for theanalysis of IRT accompanying intermittent reinforcement schedules and for the study ofnonlinear dynamics of operant behavior.IntroductionSince the pioneering works of Skinner /33/, a number of efforts have been made toexamine functional mechanisms underlying reinforced behavior from different aspects. Mostof the theoretical models dealt with influences of different schedules of reinforcement onbehavior. For example: Herrnstein’s hyperbola equation describes the relationship betweenresponse rate and the rate of reinforcement /15,16/. Killeen and Fetterman introduced a“Behavioral Theory of Timing (BeT)”/19/ to describe behavior under the control of intervalschedules of reinforcement. Killeen formulated a mathematical principle for reinforcement asa whole /20/, and developed in his recent work a mathematical model of operant behaviorcontrolled by ratio-schedules /2/. Parallel to these efforts, Gibbon introduced the “ScalarExpectancy Theory (SET)“ /5,10/ for timing in operant behavior.These models described how operant behavior − in the case of Skinner-boxexperiments, the rate of lever pressing − changed with parameters defined in reinforcementschedules. They predicted the averaged output of behavior over a session or across severalsessions. Key assumptions underlying these kinds of approaches were: firstly, the existenceof a stationary state of behavior, and secondly, the arrival to such a state at the moment ofmeasurement. Without the existence and arrival of such a stationary state, averaging oversessions would not make sense. Machado and his coworkers noted this point and madeobservations on changes of behaviors from session to session /24,25/. The model theyproposed was based on Killeen’s BeT, and was able to describe changes of behavior over timewithin the inter-reinforcement period for the fixed-interval (FI) schedule. Their workrepresented one of the rare attempts to actually deal with dynamics of operant behavior. The3

sparsity of such studies is astonishing considering that the learning process itself is defined bychanges of behavior over time as a function of experience, and, hence, is a dynamic process.However, Machado’s approach used values averaged over the whole or half of asession. This made his model very different from the stringent motion equations known inphysics. Earlier, several studies did go into the realm of studying real time dynamics. Forexample, as early as Skinner’s time, it was well known that stereotypical behavioral patternscould be found in operant experiments, and that the type of behavioral pattern depends on thereinforcement schedule in use. For example, a stereotypical pattern called “scalloped” curveappeared in the cumulative records of behavior controlled by fixed-interval (FI) reinforcementschedules. Parallel to this stereotypical behavior, rate of operant responding varied withinone reinforcement interval, as well as from one interval to the next one. A large variability inrate of responding was also found from session to session /33/. Numerous efforts, includingthe analysis of inter-response-time (IRT), were made to study the stereotypical “scalloped”curve and to tackle this “variability”-problem (e.g. /6,7,31,38/). However, an “equation ofmotion” in the sense of Newton’s classical dynamics has not yet been found.In the second half of the past century new findings in the study of chaos and nonlineardynamics opened a new field and achieved deeper insight into the nature of variability (for anintroduction see /11/). Similar to traditional models, the nonlinear systems approach viewedbehavior as being caused by the combined deterministic influences of a set of controllingvariables. However, in a nonlinear system these variables did not work the same way asstandard independent variables described in traditional models. The effect of each variabledepended on the simultaneous states of all other variables. As a result, the combination ofjust a few variables in a nonlinear system was capable of generating highly irregular patternsthat were indistinguishable from behavior of a real complex system with a huge number ofvariables /26/. These findings provided a new aspect to the “variability”-problem found inbehavioral experiments.Furthermore, nonlinear dynamics implied that separately studying each variable of asystem in isolation might not necessarily lead to an understanding of that system as a whole.The effects of variations of a single variable in the context of a system might be different thanwhen it was altered in isolation /17/. Thus, experimenters were left with an infinitelycomplicated system, whose understanding required studying all of the potential factors andtheir interactions simultaneously. The nonlinear systems approach made use of tools thatexplored how multiple variables might interact without identifying and measuring themdirectly /29/. As a result, systems could be understood as a whole, even if their componentparts had not been identified.Analysis of nonlinear dynamics relied heavily on graphical methods. One of the mostfundamental tools is the construction of phase space (for an introduction to nonlineardynamical tools see /1/). The phase space of a dynamic system is a mathematical space withorthogonal coordinate directions representing each of the variables needed to specify theinstantaneous state of that system. To construct such a portrait, the position of each variablemust be known in infinite precision through time. Unfortunately, in most behavioralexperiments, only few of the variables involved in a particular experimental situation can beknown. Even less understood is how these variables change over time. In order tocircumvent this problem, we can reconstruct a multi-dimensional space with delay-coordinatevectors, which are vectors of time series measurement of one observable of a system.Analyses of this type were suggested in physics (e.g. /29/) and their validity has beendemonstrated mathematically /35/. The delay-coordinate method is widely used in all fieldsof studies employing nonlinear dynamics.4

The Skinner-box experiment faces another problem when applying the delayedcoordinatemethod. The data set recorded here, the inter-response-time (IRT), is in fact a timeseries with time itself as the observable, and it is sampled in an irregular period along thetime-coordinate. This problem is common for all systems that can be classified as pointprocess, such as the inter-spike-interval of neuronal firing (e. g. /34/), or the R-R interval fromelectrocardiogram (ECG) studies (e. g. /13/). Although it is possible to first convert the IRTdata into response rate, thus making it uniform along the time axis, there are many settingswhere it is more interesting to analyze the IRT data directly /18/. Instead of time-coordinate,the basic idea of a delayed-coordinate method can be extended to the order-coordinate.Diagrams drawn in this manner are the so-called “return maps” (e.g. /4,32/). In some earlyapplications to operant experiments similar tools were in use, and such diagrams were said toshow the sequential dependence of IRTs /30,38/. If IRTs are bounded by some kind ofdynamic mechanism with low dimensionality, the variation of IRTs should be deterministicand the return map should be able to capture the structure of the function that describes thedynamic mechanism /18/.Previous applications of the return map to operant data indicated some promise of thenonlinear systems approach, but also some doubts, since the complexity of behavioralexperiments seemed to demand improvement of the analyzing ability of this tool /30,38/.Therefore, we modified the original definition of the return map and developed the “ExtendedReturn Map (ERM)” /21,22/. Instead of using single IRTs directly, we first calculated therunning average of several IRTs and then used them to plot a diagram. We applied thismethod to study operant behavior of rats controlled by Fixed-Interval (FI) , Fixed-Ratio (FR),Variable-Interval (VI) and Variable-Ratio (VR) schedules, and found interesting patterns inthe resulting ERMs. We further calculated the correlation dimension and informationdimension from these patterns and compared them with maps generated using surrogate datasets, which were generated by randomly shuffling the sequential order of the original IRTs.The random shuffling procedure destroyed the sequential dependency of the original timeseries data. This dependency is essential for the dynamics of the system. If the results fromthe surrogate data sets do not match the experimental results, then the patterns shown in theERM taken from the experiment must reflect the dynamics of the system /23,37/. Our resultsshowed that this is indeed the case. Dynamics of operant behavior are of deterministic nature,and this can be analyzed using ERM.We then developed two models to simulate the operant behavior controlled by FIschedules and generated artificial IRT sets using a computer program. We adjusted theparameters of the models so that both sets of IRTs showed curves similar to the experimentaldata in the frequency distribution of IRT. Furthermore, both IRT sets could successfullyreproduce the “scalloped” patterns of response shown in experimental data as represented incumulative records. However, both models differed in one feature of their formulation.While one model produced a probability function that changed continuously after eachreward, the other one employed an “abrupt switch of responding modus”. The two modelsshowed different patterns in their ERMs, and only the one employing the abrupt switch led toERMs that were like those of the experimental results. This indicated that an abrupt switch ofresponding modus was essential for the dynamics of operant behavior under thesereinforcement schedules. These corresponded to a change of behavioral states during the timebetween two adjacent rewards. This idea has been suspected in early operant conditioningstudies (e.g. /9,31/), and Killeen and Fetterman also indicated switching between behavioralstates in their BeT /19/. However, it has not been possible so far to quantitatively identify thisphenomena. Now, with the ERM as an analyzing tool, we can easily monitor the procedureof this switching between behavioral states.5

Mathematical definitionDefinition of the Original and the Extended Return MapData suitable for the application of the return map consist of sequences of timeintervals between events, the inter-event-time (IET), or, in the case of Skinner-Boxexperiments, the inter-response-time (IRT). In the simplest case the nth inter-event-time(IRT n ) is plotted against the value that precedes it (IRT n-1 ), resulting in a 2-dimensional, lag 1return map. In the more general case, a d-dimensional lag L return map is plotted by usingpoint set:{ (IRT n , IRT n-L , IRT n-2L , ... IRT n-(d-1)L ) | where n indices the order of data points } (1)In order to improve the analyzing ability of return maps for data strongly contaminatedwith noise, and for systems with both deterministic and probabilistic features, we havemodified the above definition and have invented the “extended return map” /21,22/. In thefirst step a new data set consisting of the running average (RV) of the original IRT wasconstructed according to the following equation:fRVn=n∑ + f − 1k = nIRTk/ f (2)Here a new parameter “f” was introduced in order to specify the range within whichthe running average was computed. We call this new parameter the “fold” of the extendedreturn map. Then a procedure similar to the original return maps can be applied. A d-dimensional, lag L extended return map with fold f, is constructed by the following point set:{( f RV n , f RV n-L , f RV n-2L , ... f RV n-(d-1)L ) | where n indexes the order of data points } (3)Fractal DimensionOne way to specify the property of patterns found in a phase diagram is to calculatethe fractal dimension. Unlike the topological dimension, which is always an integer value,the fractal dimension can be an integer or a fraction. This value gives a quantitative measureof the fractal properties, including self-similarity and scaling /23/. Several mathematicaldefinitions of fractal dimension have been suggested, and procedures were developed tocalculate them from experimental data /12,14,36/. In the present study, we used the“correlation dimension” and the “information dimension” to interpret our experimental data.We chose these because the distribution of our data points in the extended return map was notuniform and these two definitions can take this factor into account.Suppose the data points from an experiment consist of a time series {X 1 , X 2 , ..., X M },then the correlation dimension of this set as defined by Grassberger is /12/:ν=ln C( ε )(4)ln ( ε )limε → 0Where C(ε) is defined as2C(ε) = limM− × { number of pairs (X i , X j ) with | X i - X j | < ε} (5)M→∞M is the amount of data in the time series; ε is a value predefined to calculate C(ε).6

The information dimension is defined according to Grassberger as /12/:=E(ε)(6)ln(ε)δ limε→0Where E(ε) is defined asN( ε )E(ε) = ∑ Pi ln(P i)(7)i=lHere P i is the probability for a data point to fall into the cube i of the partitioning with size ε.For the analysis of experimental data, the frequency of occurrence in each cube of thepartitioning was counted and was taken as the probability P.In practice, several ε values between 0.01 sec and 10 sec were chosen. The former isthe resolution of the IRT and the latter is in the order of the size of the patterns in the ERMs.The corresponding ln C(ε) and E(ε) were calculated, and then plotted against ln (ε.) Theslope of the resulting regression line was taken as the correlation dimension ν and theinformation dimension δ.Although it is not certain, whether or not the extended return map is a topologicallyequivalent reconstruction of the phase diagram, it is still possible to calculate the fractaldimension, because the mathematical definition in equations (4), (5), (6) and (7) is applicablefor any time series. The only constraint is the number of data points. Theoretically infinitedata points are required in order to obtain an exact value. In practice, the amount of datashould be reasonably large /28/. For Skinner-Box experiments, this condition is relativelyeasy to fulfill.Even when the meaning of this value calculated from the extended return map differsfrom the “dimension” of attractors found in a real phase diagram, it is still interesting todetermine, whether or not it can serve as a quantitative measure of behavioral patterns inanimals receiving different reinforcement schedules and/or physiological treatments.Testing the Results with Surrogate DataSince experimental data is always contaminated with noise, the differentiation ofinformation of systems dynamics from disturbance by noise becomes a critical point in a nonlinearanalysis. We wish to determine whether our findings in the extended return map reallyreflect the system’s characteristics, or whether they are simply caused by the analyzingprocedure. One way to clarify this, is to test the results with surrogate data sets which havethe same statistical properties as the original data, such as mean, variance and power spectra,while the dynamic information, that is, the deterministic correlation of the system’s presentstate with its history, is destroyed. If the patterns in the extended return map from surrogatedata sets differ from those of the original data, then the differences must be due to the fact thatthe dynamic information presented in the original data was removed.There are several different ways to generate surrogate data sets. The simplest one is torandomly reorganize the sequential order of the original data set. This reorganization is verysevere. Theiler suggested that it might be enough to generate surrogate data sets which havethe same first order correlation as the original one, but all higher order correlations areremoved /23,37/. For simplicity, only the reorganization of the original data was applied inthe present study.7

In the present study, 5000 and 2500 surrogate data sets were generated for thecalculation of the surrogate information and correlation dimensions respectively. Thus afrequency distribution of the surrogate fractal dimensions could be established. The locationof the experimental fractal dimensions on the diagram of the surrogate distributionquantitatively shows the probability of acquiring the experimental IRT data by chance.ExperimentsAnimalsFive male, experimentally naive white Wistar rats obtained from theTierversuchsanlage, University of Düsseldorf, were used in the present study. They were 2months old and weighed 220 ~ 260 g at the beginning of the experiment. The animals werehoused in individual cages with free access to water. Illumination was controlled artificiallyunder a 12hours day/night period with light on at 7:00 AM and off at 7:00 PM. Experimentswere conducted between 10:00 AM and 5:00 PM. During shaping and test sessions, theanimals were given about 50 % of the food they consumed during the free feeding days. Thefood pellets consumed through rewards in the Skinner-box study were also taken into account.ApparatusThree Skinner-boxes (28 cm × 27 cm × 29 cm) made of metal were placed into threeseparated wooden sound attenuating chambers (130 cm × 59 cm × 103 cm). A 40 W bulbilluminated each chamber, resulting in about 12 LUX measured in the middle of the Skinnerbox.Experiments were executed and controlled by a personal computer. A program runningunder MS DOS 6.2 was developed to control the hardware and to record data. With the aid ofthis software, one PC can run up to four Skinner-Boxes at a time. IRTs were saved in a textfile with tabular format, and a second program running under Microsoft Windows 95/98 waswritten to perform the nonlinear dynamical analysis. This program was also capable of doingsimulations using models discussed later. The DOS-program was written and compiled withthe programming language “Pascal”. Design and development of the apparatus, including thesoftware, was accomplished by Manfred Mittelstaedt, and one of the authors (J.-S. Li). TheWindows 95/98-program was written by J.-S. Li and was compiled with the programminglanguage “C++ Builder”.ProcedureAfter five days (15 minutes per day) of manual shaping followed by a three day pause,the effect of changing reinforcement schedules on the dynamics of operant lever pressingbehavior was observed for 8 days (details see. Table 1). One daily session was run for eachanimal. In the first session, every rat received 90 minutes continuous reinforcement (CRF).In the following 7 sessions, each animal received CRF during the first minute, then thereinforcement schedule was changed abruptly to one of the following schedules: FI20, FI40,FR20, VI40/20, and VR20/20. Each rat received the same intermittent reinforcementschedule in all 7 sessions during the testing periods. Data from the first day are indicatedbelow as “CRF”. They served as a kind of baseline behavior before different reinforcementschedules were applied. Data from the remaining 7 days are designated as “sessions 1through 7” throughout this paper. Before the beginning of shaping and during the 3 daypause, the weight of food consumed by each rat was recorded individually and served asreference for food deprivation.8

Table 1Experimental proceduresDays Reinforcement schedules Length of Run time1 ~ 5 Shaping 15 min6 ~ 8 Pause9 CRF 90 min10 ~ 16(Session 1 ~ 7)CRF⇓VI, FI or FR1 min⇓90 minFor variable ratio and interval schedules, a random number generator provided by thecompiler “Borland Turbo Pascal 7.0” was used to construct a homogeneous probabilitydistribution. The numbers x/y in the notation of variable interval and ratio schedules stand forthe mean (the first number x) and the range of variation (the second number y). That is, theintervals and ratios used in the VI40/20 and VR20/20 schedules varied within the range (30,50) and (10, 30). The probability of occurrence of each value within the range was the same.ResultBefore presenting the results of nonlinear dynamic analysis, some traditional analysis,including the overall lever pressing behavior across sessions, the frequency distribution ofIRTs, and the cumulative records were examined in order to ensure that it conformed toconventional responding generated by Skinner-box experiments. The nonlinear dynamicanalysis began with a comparison between the original “Return Map (RM)” and the“Extended Return Map (ERM)” from session 7. The results confirmed the expectedimprovement of the analyzing ability of the ERM over the RM. The comparison of fractaldimensions of the ERMs between the experimental and the surrogate data sets demonstratedthe consequence of destroying the sequential order of the IRT data. Since the animals underthe control of interval-schedules showed very special patterns in the ERMs of session 7, wedocumented the ERMs of their behavior from other sessions, in order to investigate thedevelopment of these special patterns.Lever pressing behavior across sessionsFig 1 shows the total number of lever presses per session throughout the wholeexperiment. Data points in the first column show the number of lever presses over 90minutes. On this day the animals were under a CRF schedule. As indicated by the diagram,all animals produced roughly the same number of lever presses. For sessions 1-7 only thelever presses from the 2 nd to the 91 th minute were taken into evaluation. Records of the firstminute were discarded, because all animals were still under CRF during this time. From thesecond minute on different reinforcement schedules were introduced. Thus, only behaviorunder the influence of testing schedules in sessions 1-7 were shown in the diagram, and thelength of recording time was also 90 minutes, as in the first column. As indicated by thefigure, number of lever presses increased rapidly over the first session as the scheduleschanged, and they remained more or less constant over the following six sessions. Only thedata from the FR20 rat exhibited another rapid increase in responding in sessions 4 and 5.9

Fig. 1: The total number of lever presses per session throughout the whole experiment.The first column (CRF) shows the number of lever presses over 90 minutes. Forsessions 1-7 only the lever presses from the second to the 91th minute wereevaluated.Frequency distribution of IRTsThe frequency distributions of IRTs from session 7 are shown in figure 2. Each bar inthe diagrams was 0.1 second wide. All IRTs longer than 12 seconds were accumulated in thelast bar. Roughly speaking, the IRT distributions can be divided into three parts: firstly, anextremely sharp and narrow peak consisting of very short IRTs (

Fig. 2: The frequency distributions of IRTs from session 7. Width of bars = 0.1 sec.; allIRTs longer than 12 sec. are accumulated in the last bar. The IRT distribution curvesare similar to the distribution curve of a Poisson process.Fig. 3: Part of the cumulative records in session 7, reconstructed using IRT data. Typicalcharacteristics for operant behavior under intermittent reinforcement schedules can beseen.11

Fig. 4: The averaged rate of responding after rewards of FI20, FI40 and VI40/20 rats insessions 1. Column width = 1 second for FI20; 2 seconds for FI40 and VI40/20.The last column summarizes all responses that appeared later than 20, 40 and 50seconds after rewards.Fig. 5: The averaged rate of responding after rewards of FI20, FI40 and VI40/20 rats insession 7. Setup for the diagrams is the same as in the fig. 4.The high variability shown in fig. 3 made it difficult to generalize any findings to be acommon characteristic of the operant behavior for a given animal. One possible way toovercome this problem was to average the responses across rewards. Although different,previous studies have indicated that this kind of analysis might be problematic and should notbe used to represent the “average” of scalloped patterns found in cumulative records under FIschedules /3,7,9/, this procedure is a convenient way to produce an easily recognized overallcumulative record.Figures 4 and 5 show the averaged rate of responding over time after rewards underFI20, FI40 and VI40/20 schedules from sessions 1 and 7. The width of all but the lastcolumns is 1 sec for FI20 and 2 sec for FI40 and VI40/20. The last column summarizes allresponses that appeared later than 20, 40 and 50 sec after rewards. In the first session theaveraged response rate was roughly constant except shortly (less than 4 sec) after rewards, asindicated by the shorter second columns in the records of the FI20 and FI40 rats, and the firstcolumn in the records of the VI40 animal. The patterns of responding in records of session 7show typical characteristics of operant behavior under interval-schedules: except for theovershooting in the first column, rate of responding dropped abruptly after reward, and thenincreased gradually until the administration of the next reward. The maximum response rate12

Fig. 6: The original “Return Maps (RMs)” from session 7. Here only the region with Xand Y values smaller than 10 seconds is shown. Most of the data points are locatednear the zero point and along the two axes. No structure can be clearly identified.Fig. 7: The “Extended Return Maps (ERMs)” from session 7. “RA of IRT(k)” and “RA ofIRT(k+L)” represent running average of IRT at position k and k+L, where L is thetime lag. Data used was the same data as in fig. 6. Parameters: F=15, L=15.Lattice structure can be seen in the maps on the left side. Arrows indicate thecluster with highest concentration of data points.13

in the record of VI40/20 was about 36-38 seconds, which was slightly lower than the mean ofthe pre-defined inter-reinforcement-interval. These patterns of responding are typical foroperant behavior controlled by an interval-schedule. Also the differences between behavior insession 1 and in the final session 7 are consistent with previous reports (e.g. /25/).Results of the original “Return Maps” and the “Extended Return Map”Figures 6 and 7 show the original “Return Maps (RMs)” and the “Extended ReturnMaps (ERMs)” respectively. They were constructed using IRTs from session 7. In order tohave a closer look at the fine structures and to have a better comparison between the twodiagrams, only the region with X and Y values smaller than 10 seconds is shown. The fold(F) and time lag (L) used for the construction of the ERMs were F=15 and L=15.In figure 6, most of the data points are located near the zero point and along the twoaxes. No other structure could be found in any of the RMs. Neither could we find cleardifferences between behavior controlled under different schedules. This finding is consistentwith a previous study using pigeon /30/. In figure 7, we can easily differentiate betweenanimals under control of interval- and ratio-schedules by comparison of the ERMs. The 3ERMs in the left column show results of the rats under interval-schedule control. Severalclusters of points form well organized lattice structures. The arrows indicate the cluster withmaximal concentration of data points. The two ERMs in the right column show results of theanimals under ratio-schedule control. A very dense cluster, which contains most of the datapoints, is located in the region bounded by the four coordinates: (0.5, 0.5), (2, 0.5), (0.5, 2)and (2, 2) for FR20 rat: (1, 1), (3, 1), (1, 3) and (3, 3) for VR20/20 rat. Other data pointsformed several small clusters which line up along both axes. Data points were very rare in thecentral region of the maps.Fig. 8: The surrogate ERMs from session 7. Compared with fig. 7, the lattice structures havealmost vanished.14

Comparison between experimental and surrogate data setsThe ERMs of surrogate data sets from session 7 are shown in figure 8. The data setswere generated by randomly shuffling the sequential order of the experimental IRT data. Theparameters used for the construction of the surrogate ERMs were F=15 and L=15, which werethe same as in figure 7. The comparison between figures 7 and 8 gives a clear impression ofthe change of patterns caused by the random shuffling. The well organized lattice structure inthe ERMs of animals under interval-schedule control almost vanished in the ERMs of thesurrogate data. Although most of the data points still concentrated in roughly the same regionas in the ERMs of the experimental data, the distribution of data points was morehomogeneous. The ERMs of the surrogate data from animals under ratio-schedule control arevery similar to those from the experimental data, except that for the FR20 rat, the smallclusters aligned along the axes in the experimental ERM become more diffuse in the surrogatedata. Hardly any difference can be identified in the surrogate ERM of VR20 rat.Since the direct comparison of the maps cannot quantitatively describe the differencebetween the experimental and the surrogate data sets, we first had to calculate an entity thatcould reflect the topological features of the patterns found in ERMs, and then compare thisvalue. We chose the correlation dimension and the information dimension to accomplish this.We repeated the random shuffling procedure many times and calculated the correlation andinformation dimensions from each of the resulting surrogate data set; hence, after partitioning,we could count the occurring frequency of the surrogate dimension in each partition and forma frequency distribution diagram for each rat. These are shown in figure 9 for the correlationdimension and figure 10 for the information dimension. The number of surrogate data setsgenerated in the random shuffling procedure was 2500 for the correlation dimension and 5000for the information dimension. The parameters used to construct the ERMs were F=15 andFig. 9: The frequency distribution of the surrogate correlation dimension. The bar indicates theposition of the experimental value. The percentage value represents the relative size ofthe dark gray area (outside the experimental dimension ). Except for the VR20 rat, allother experimental values are clearly far from the central area of the distribution.15

Fig. 10: The frequency distribution of the surrogate information dimension. The barindicates the position of the experimental value. The percentage value representsthe relative size of the dark gray area. There are two peaks in each distributioncurve. The experimental FI40 and VI40 values are clearly apart from the centralarea. The experimental FR20 value is very close to the center of the surrogatedistribution. Owing to the existence of the second peak, it is not so easy to evaluatethe situation for FI20 and VR20.L=15. The experimental correlation and information dimensions are shown in figs. 9 and 10as light gray bars. The percentage shown in the diagrams represents the relative surfacecovered by the dark gray area.It can be clearly seen that the frequency distribution of the surrogate correlationdimension of all rats was more or less normally distributed. Except for the VR20 rat, all otherexperimental correlation dimensions were located far apart from the center maximum. Thisindicated that the sequential order of IRTs was important for the operant behavior of theserats, so that its randomization resulted in data sets that generated ERMs that had distinctivelydifferent values of correlation dimension. The VR20 rat, on the contrary, produced IRTswhose sequential order could be formed by chance as well.The frequency distribution curves of information dimension did not look like a normaldistribution, because there were two maxima in the occurring frequency. For the FR 20 rat,one of the two peaks was extremely small. Although they were not normally distributed, wecould nevertheless judge from the percentage of the gray area how probable it was to obtainthe sequential order of the experimental IRTs simply be chance. According to the presentresults, the rats under the interval-schedules showed operant behavior that is farther fromrandom than that of the ratio-schedule controlled rats.16

Formation of the lattice structures in the ERMs of rats under interval-schedulesThe ERMs constructed using IRT-data of the FI20, FI40 and VI40 rats from all testingsessions are shown in figs. 11, 12 and 13. The gradual formation of the lattice structures canbe seen. The parameters used in the construction of ERMs were for FI20: F=12, L=12; forFI40: F=15, L=15; for VI40/20: F=15, L=15. In the beginning of the learning process theERMs of all three rats (left diagram in the first line) looked very similar: A dense cluster ofpoints near the bottom-left corner of the diagram can be seen, and the density of pointsdecreases when we move from bottom-left to top-right. The data points in the ERM of theFI20 rat are slightly more concentrated on the bottom-left side of the cluster as revealed bythe slightly denser tip of the cluster. The lattice structure in the ERMs from the FI20 ratbegins to appear in session 3 (right diagram in the first line) and becomes very clear insessions 4 and 5 (left and middle diagrams in the second line). Thereafter, the lattice structurein the ERMs becomes slightly diffuse in sessions 6 and 7 (right diagram in the second lineand left diagram in the third line). The formation of lattice structure in the ERMs of the othertwo animals begins at about session 6 (right diagram in the second line). In comparison to theFI20 rat, this occurred much later. Prior to the formation of the lattice structure all the ERMslooked very similar to that of session 1. The only exceptions were the ERMs from session 4of the FI40 rat and session 5 of the VI40/20 rat, which showed more diffuse distributions andhad fewer data points.DiscussionSummery of the traditional analysisThe goal of the traditional analysis was to confirm that the behavior generated in thepresent study corresponded to the conventional responding generated by similar Skinner-boxexperiments. This object was accomplished through the following findings: Firstly, theoverall number of lever presses across sessions showed that the shaping of the operantbehavior and the training of different reinforcement schedules were successful. Secondly: theFig.11 The ERMs of the FI20 rat from all sessions. There is already weak organization in theERMs from sessions 2 and 3. Clear lattice patterns appear from session 4 on.17

Fig.12 The ERMs of the FI40 rat from all sessions. Weak organization appears in sessions 5and 6. Clear lattice patterns do not form until session 7. In session 4 the distribution ofdata points is very diffuse.Fig.13 The ERMs of the VI40 rat from all sessions. Clear lattice patterns appear in sessions 6and 7. In session 5 the distribution of data points is very diffuse.analysis of the frequency distribution of IRTs revealed characteristics of a Poisson process,which is consistent with previous reports (e.g. /38/). Finally, both the reconstructedcumulative records from individual reinforcements and the averaged responses over allrewards showed the expected “scalloped” curve and the post reinforcement pause that aretypical for behavior under interval-schedules. In conclusion, it is justified to regard the18

present study as a standard conventional Skinner-box experiment. Thus, the results of thenonlinear dynamic analysis can be generalized to other operant studies under comparableexperimental conditions.Summery of the nonlinear dynamic analysisOne achievement of the present work is the improved analyzing ability of the“Extended Return Map” over the original “Return Map”. Comparison between figures 6 and7 reveals that the well organized lattice structures in the ERMs of rats under the control ofinterval-schedules are not visible in RM. Also the difference of behavioral dynamics betweenanimals under interval- and ratio-schedules becomes visible only in the ERMs. The patternsseen in the maps is comparable to the trajectory in the phase space and reflects dynamicproperties of the system under study /18,32/. Thus, we can state that the patterns in ERM aretraces of the behavioral state of the animals during the experiment. The regular latticestructures in the ERMs of interval-schedule controlled rats suggest a kind of periodicity of theoperant behavior.However, since time series data from an experimental study contain only limitednumber of elements and are often contaminated with noise, the absolute results of a nonlineardynamic analysis could be erroneous and might indicate false deterministic relationships ofthe system’s dynamics. To exclude this possibility, it is necessary to look at the surrogatedata sets, which share the same first order correlation with the experimental data, but have thehigher order correlation destroyed /23,37/. In the present work, we produced surrogate datasets using exactly the same elements as the experiment. We simply randomized theirsequential order. Hence, both the surrogate and the experimental data sets had the samemean and frequency distribution. However, if there were any sequential dependency in theexperimental IRTs, this relationship would be destroyed.The direct comparison between the surrogate and the experimental ERMs shows that,after randomization, the patterns found in the interval-schedule controlled rats almostvanished. This indicates that the lattice structures seen in the ERMs of these rats really reflectdynamics of operant behavior under this condition. There were also differences betweensurrogate and experimental ERMs of the FR rat, however, these are not so obvious. Althoughthe direct graphical comparison is very impressive, we cannot be certain, that the surrogateERMs shown in fig. 8 are representative, nor can we make any statistical test to quantify thedifference. To overcome this problem, we first had to calculate an entity able to representsome geometrical features of the patterns found in the ERMs, and then obtain the same entityfor the surrogate ERMs in order to perform a statistical comparison. In the present work, wechose the correlation and the information dimensions as entities to be calculated andcompared. The results showed that the probability to obtain the experimental sequence of IRTdata from interval-schedule controlled rats as well as the FR20 rat is very low, since theexperimental correlation dimensions are very far apart from the central area of the distributioncurve of the surrogate dimension. Similar results in the information dimension were alsofound for the interval-schedule controlled rats. The VR20 schedule, on the contrary,produced an IRT sequence which was indistinguishable from the surrogate data from thestandpoint of both the correlation and information dimensions of the ERM.Fractal dimensions reflect very important geometrical features of the patterns understudy. In phase spaces of systems with complicated dynamics, they usually indicate thechaotic properties of the attractors generated by the systems. However, it has not been provenmathematically that the reconstructed diagrams using ERMs are equivalent to the originalphase space. Thus, the meaning of the correlation and the information dimensions calculatedin the surrogate studies is unclear. However, this deficiency does not impair the goal of the19

surrogate study, because the fractal dimensions themselves still reflect the differencesbetween patterns seen in the experimental and the surrogate ERMs.Another interesting application of the ERM is to trace the course of the learningprocess in an operant experiment. As mentioned previously, the learning process is bydefinition a dynamic process. With the ERM as an analyzing tool, we can follow the changeof the animal’s behavioral dynamics over time. In figs. 11-13 we can see when thestereotypical lattice patterns in the ERM of an interval-schedule controlled rat began to occur,and we can further observe how these patterns gradually developed, and which intermediatestate would appear before the final formation of these patterns.Models for operant behavior under fixed-interval reinforcement schedulesOur next step was to construct a modelthat can produce artificial IRTs which match theexperimental results found in session 7 from thefixed-interval schedule controlled rats. Themodel was to be basically a kind of “pointprocess”; that is, it was intended to describe thetime dependent changes of the probability oflever pressing behavior. The first idea for theconstruction of such models came from the“scalloped” curve in the cumulative records.Similar to the lattice structures in the ERMs,this stereotypical pattern in the cumulativerecords also developed over the course of thelearning process, as indicated in figs. 4 and 5.In addition to that, the IRT distribution is animportant condition that a point process modelmust fulfill. Theoretically, there are an infinitenumber of different ways to build models thatcan generate IRTs to match such experimentalcumulative records and IRT distributions. Thequestion is, whether these two conditions aresufficient to completely describe the operantbehavior under the interval-schedule. If, forexample, we can build a model that cansuccessfully simulate the cumulative recordsand IRT distributions, but fails to generateERMs that can match the experimental results,then we can be sure that the ERM is able toshow some properties that are not visible in thecumulative records and the IRT distributions.Furthermore, if a small modification can makethis model fulfill all the three conditions, thenthis modification might represent a veryimportant feature for the dynamics of theoperant behavior under this conditions.Table 2Parameters for the simulationsModel IModel IIA 00.0042 0.0044M 00.2 0.2O 00.0016 0.00005A 10.7 0.9M 10.65 0.16O 10.015 0.02A 20.45 0.62M 20.31 0.166O 20.02 0.03A 30.02 0.06M 30.01 0.05O 30.002 0.01K 2 ×10 - 9 6 ×10 - 11ReinforcementScheduleTotal RunTime (sec.)FI 40 FI 405400 5400Following this strategy, we designed two models, both of which were able to generateIRT data sets that matched the IRT distributions and the cumulative records. The mainconcept for the design of the models can be divided into two parts. The first part describesthe probability for an operant behavior to occur, and it can be further divided into two20

Fig.14 Results of the simulations. Upper panel: model I; lower panel: model II; leftcolumn: IRT distributions; middle column: averaged responding rate over timeafter rewards; right column: ERMs. Parameters used for ERM: fold = 15, time lag=15. Compared to figs. 2, 5 and 7, both models I and II can produce IRTdistribution and averaged response after rewards similar to the experimental results,but only model II produces ERM that can match the experimental results.subunits. The first subunit is a function that depends on the time after the delivery and theconsummation of rewards. The second subunit consists of three sigmoid functions which aredependent on the time after each response. How these two subunits are interconnected,represents the main difference between the two models. While in the first model the twosubunits are interconnected with a multiplication-operation, in the second model they areinterconnected with a switch-operation. That is, after the reward is consumed and before thefirst operant response occurs, the final probability for an operant behavior to occur is decidedby the first subunit. After the first response, it is decided by the second subunit. We canregard this switch between reacting modus as a switch between behavioral states. The firstmodel, on the contrary, is a continuous function.The second part of the two models is designed to account for the “overshooting”phenomena. Since the feeding machine required some time to execute the command for“reinforce” from the software, there were always small delays between the time of reward andthe time at which the animals could actually consume the reward. Besides, the animalssometimes did not respond for the presence of the reward at once, and continued to leverpress for a while. This resulted in a number of operant responses immediately after rewards,as indicated in the first column of the averaged cumulative records (fig. 4). Therefore wedesigned a time dependent function to describe the probability that the animal actually took or21

consumed the reward after it was recorded as “available” by the software. This part wasactive only during the time between the availability and the consuming of a reward.The detailed formulation of the models is outlined in the appendix. The parametersused in the simulations are outlined in table 2. The parameters were carefully adjusted so thatthe simulated IRT distributions and the averaged rate of responding after rewards could matchthe experimental results by visual examination. The simulated IRT distributions, the averagedrate of responding after rewards, and the ERMs are shown in figure 14, from which it isapparent that both models could produce simulations by which the IRT distributions and theaveraged response rate after rewards were well matched with the experimental results. Thefirst model (left column), however, produced an ERM that is completely different from that ofthe experimental data. The second model (right column), on the contrary, produced an ERMthat looks very similar to the experimental ERM in figure 7.In summary, this simulation reveals that the cumulative records and the IRTdistribution alone are not sufficient to completely describe the dynamics of the operantbehavior under fixed-interval schedule, whereas the ERM accomplishes this task.Furthermore, the simulation also suggests that a switch between two behavioral states playsan important role in the operant behavior under interval-schedules.ConclusionsThe present work shows that the ERM has, in comparison with the original RM, animproved analyzing ability for the dynamics of operant behavior. Especially in the datagenerated by interval-schedules, there are patterns in the ERMs that were not visible in theoriginal RMs. The examination with the surrogate data procedure in combination with thecomputation of correlation and information dimensions furthermore revealed that the patternsfound in the ERMs characterize dynamics of the operant behavior under interval- and fix-ratioschedules, because if the sequential order of the IRT data was destroyed, the patterns shownin the ERMs vanished and the ERMs had different correlation as well as informationdimensions.We furthermore studied the time course of the whole learning process involved inestablishing behavioral control with intermittent reinforcement schedules with ERMs. Cleardifferences could be seen in the ERMs from different sessions. Of course, since there wasonly one rat used under each condition, it is not proven whether or not this finding can begeneralized to operant behavior under these conditions as a whole. From another point ofview, the ERM is designed to study the behavioral dynamics of individual animals. Thus, itshould not be surprising if we should find individual differences in the behavior of animalsunder the same experimental conditions, including a distinctively different ERM. Thequestion is, whether we can find an entity that can quantify the patterns found in the ERM andwhether this entity can be used to represent the behavioral dynamics of an experimentalgroup. The fractal dimension used in the surrogate data study might be suitable for thispurpose. In the present work, we did not compare between fractal dimensions from rats underdifferent reinforcement schedules, because there was only one rat run under each conditionand this kind of comparison would not have made much sense.A next possible improvement would be a systematic study of the parameters used inthe reinforcement schedule; e.g., the length of the inter-reinforcement interval defined in theFI schedule. In the present study, we implemented only 20 and 40 seconds for the FIschedule, and the simulation for the FI40 schedule using model II suggests that the switchbetween two behavioral states is essential for the dynamics of operant behavior under thiscondition. If we use a longer period in the FI schedule, for example 120 seconds, will therebe a third or even more behavioral states in the period between two adjacent rewards?22

The study on simulation models also revealed very interesting results. As mentionedpreviously, the comparison between the two models suggests that a nonlinear switch betweentwo behavioral states plays an essential role in the operant behavior under the control ofinterval-schedules. Although similar observations were reported previously /9,19,27,31/, it isthe ERM and the study using two simulation models that definitely demonstrated thenecessity of this nonlinear switch of behavioral states. In the present work we performed onlysimulations for the FI40 rat. It might be easy to simulate the FI20 and VI40 rats as well byusing different sets of parameters, since the cumulative records, the IRT distributions, and theERMs of these rats looked very similar.Another direction for further study is to implement drug and/or lesion manipulations.Although there are many reports about the influence of drug and lesion treatments, thereexists hardly any work on such effects on dynamics of operant behavior. Most availablestudies have dealt solely with changes in “averaged” rate of responding. Since the averagedoperant response is the consequence of the behavioral dynamics of the animals, it will be veryinteresting to see what kind of change in the dynamics can be produced through these kinds oftreatments.In addition to Skinner-box experiments, the ERM can also be applied to otherparadigms in behavior research. For example, self stimulation of the brain using electricalstimulations as a reinforcer is a good candidate, since the type of data from this experiment isalso a kind of point process. There would be no need to make any modification of the ERM,and the possibility for the experimental setups is even more manifold. Theoretically, theERM can be applied to any kind of point process. Of course, the number of data pointsrepresents an important constraint for this method. For a reliable analysis, the number of datapoints required is in the order of hundreds.Finally, a thorough study of the mathematical properties of the ERM itself would bean important further step. As mentioned in the introduction, the mathematical foundation ofthe delay-coordinate method is based mainly on the work by Takens /35/. The assumptionsmade in his theory, such as the continuity of the function, cannot be fulfilled, stringentlyspeaking, by any real experiments. Owing to the irregular measurements, the extension ofthis method to the RM presents even more problems regarding the mathematical validity ofTakens theory /18/. Here, in the definition of ERM, we introduce the computation of therunning average, and thereby complicate the situation even more. In the present work, wesimply empirically demonstrated how to use the ERM in the analysis of behavioralexperiments. To what extent we can still, mathematically, rely on the Takens theory, was notdiscussed at all, and this would be a very important question for the future application of theERM.Appendix:The detailed formulation of the models is outlined as following:Model I : Φ (t) = F 0 (t a ) × F 1 (t b ) (8)Model II: Φ (t) =F 0 (t a ) after the taking or consummation of a reward andbefore the occurrence of the first responseor F 1 (t b ) after the first response (9)Where Φ (t) is the probability for an operant response to occur. The definition of the twomodels differs only in the equations (8) and (9). All other equations below are valid for bothmodels. The functions F 0 (t a ) and F 1 (t b ) are defined as follows:23

F 0 (t a ) = F 0 (t a - 1) +A 0 × F 0 (t a - 1) × [M 0 - F 0 (t a - 1)] (10)with initial condition:F 0 (0) = O 0 (11)andF 1 (t b ) = R 1 (t b ) – R 2 (t b ) + R 3 (t b ) (12)The functions R 1 (t b ), R 2 (t b ) and R 3 (t b ) are defined as follows:R 1 (t b ) = R 1 (t b - 1) +A 1 × R 1 (t b - 1) × [M 1 - R 1 (t b - 1)] (13)R 2 (t b ) = R 2 (t b - 1) + A 2 × R 2 (t b - 1) × [M 2 - R 2 (t b - 1)] (14)R 3 (t b ) = R 3 (t b - 1) + A 3 × R 3 (t b - 1) × [M 3 - R 3 (t b - 1)] (15)with initial conditions:R 1 (0) = O 1 , R 2 (0) = O 2 and R 3 (0) = O 3 (16)Here t a and t b were discrete counters which represented the time after the taking of or theconsummation of a reward (t a ), and the time after an operant response (t b ), respectively.When a reward was consumed, and when an operant response was emitted, the counters t a aswell as t b would be reset to 0. The probability for a reward to be taken was determinedthrough the following formula:Ψ(t) = k × t c3 × tb2where the Ψ (t) was the probability that a reward would be taken after it was available. Herethe t c was again a discrete counter which represented the time after the presentation of areward. The definition for the counter t b was the same as above. The units of all timecounters t a , t b and t c was 0.01 second. The parameters A 0-3 , M 0-3 , O 0-3 and k can be adjusted,so that the IRT distribution and the averaged cumulative records generated from the simulatedIRT data sets can match the experimental results.Acknowledgement:References:Supported by a grant (Hu.306/11-3) from the German National Science Foundation.1. Baker GL, Gollub JP. Chaotic dynamics - an introduction. New York: CambridgeUniversity Press,1990.2. Bizo LA, Killeen PR. Models of ratio schedule performance. Journal of ExperimentalPsychology: Animal Behavior Processes 1997; 23(3):351-367.3. Branch MN, Gollub LR. A detailed analysis of the effects of d-amphetamine onbehavior under fixed-interval schedules. Journal of the Experimental Analysis ofBehavior 1974; 21:519-539.4. Braun T, Lisbôa JA. Characterization of homoclinic chaos in a glow discharge throughreturn maps. International Journal of Bifurcation and Chaos 1994; 4(6):1483-1493.(17)24

5. Church RM, Meck WH, Gibbon J. Application of scalar timing theory to individualtrials. Journal of Experimental Psychology: Animal Behavior Processes 1994;20(2):135-155.6. Dews PB. The theory of fixed-interval responding. In: Schoenfeld WN, editors. Thetheory of reinforcement schedules. New York: Appleton-Century-Crofts, EducationalDivision, Meredith Corporation, 1970; 43-61.7. Dews PB. Studies on responding under fixed-interval schedules of reinforcement: IIThe scalloped pattern of the cumulative record. Journal of the Experimental Analysis ofBehavior 1978; 29:67-75.8. Ferster CB, Skinner BF. Schedules of reinforcement. New York: Appleton-Centrury-Crofts, 1957.9. Gentry GD, Weiss B, Laties VG. The microanalysis of fixed-interval responding,Journal of the Experimental Analysis of Behavior 1983; 39:327-343.10. Gibbon J. Scalar expectancy theory and Weber’s Law in animal timing. PsychologicalReview 1977; 84(3):279-325.11. Gleick J. Chaos - making a new science. New York: Penguin Books, 1987.12. Grassberger P. Generalized dimensions of strange attractors. Physics Letters, 1983;97A(6):227-230.13. Guzzetti S, Signorini MG, Cogliati C, Mezzetti S, Porta A, Cerutti S, Malliani A. Nonlineardynamics and chaotic indices in heart rate variability of normal subjects and theheart-transplanted patients. Cardiovascular Research 1996; 31:441-446.14. Hentschel HGE, Procaccia I. The infinite number of generalized dimensions of fractalsand strange attractors. Physica D 1983; 8:435-444.15. Herrnstein RJ. On the law of effect. Journal of the Experimental Analysis of Behavior1970; 13:243-266.16. Herrnstein RJ. Formal properties of the matching law. Journal of the ExperimentalAnalysis of Behavior 1974; 21:159-164.17. Hoyert MS. Order and chaos in fixed-interval schedules of reinforcement. Journal ofthe Experimental Analysis of Behavior 1992; 57:339-336.18. Kantz H, Schreiber T. Nonlinear time series analysis. New York: CambridgeUniversity Press, 1997.19. Killeen P, Fetterman G. A behavioral theory of timing. Psychological Review 1988;95(2): 274-295.20. Killeen P. Mathematical principles of reinforcement, Behavioral and Brain Sciences1994; 17:105-172.21. Li JS, Huston JP. The extended return map: a new method to analyse the non-lineardynamics of inter-response-intervals in Skinner-box experiments. In: Fourth EuropeanMeeting for the Experimental Analysis of Behaviour, Amiens, France, July 9-13, 2000.22. Li JS. The extended return map: a new method to analyze the nonlinear dynamics ofinter-response-intervals in operant conditioning studies. In: Tenth Annual InternationalConference of the Society For Chaos Theory in Psychology & Life Sciences, Universityof Pennsylvania; Philadelphia, Pennsylvania; July 20-23, 2000.25

23. Liebovitch LS. Fractals and chaos: simplified for the life sciences. New York: OxfordUniversity Press, 1998.24. Machado A. Learning the temporal dynamics of behavior. Psychological Review 1997;104(2):241-265.25. Machado A, Cevik M. Acquisition and extinction under periodic reinforcement.Behavioural Processes 1998; 44:237-262.26. May RM. Simple mathematical models with very complicated dynamics. Nature 1976;261:459-467.27. Nevin JA, Baum WM. Feedback functions for variable-interval reinforcement, Journalof the Experimental Analysis of Behavior 1980; 34:207-217.28. Ott E, Sauer T, Yorke JA. Coping with chaos: analysis of chaotic data and theexploitation of chaotic systems. New York: John Wiley & Sons, Inc., 1994.29. Packard NH, Crutchfiel JP, Farmer JD, Shaw RS. Geometry from a time series.Physical Review Letters 1980; 45:712-716.30. Palya WL. Dynamics in the fine structure of schedule – controlled behavior. Journal ofthe Experimental Analysis of Behavior 1992; 57:267 – 287.31. Schneider AB. A two-state analysis of fixed-interval responding in the pigeon. Journalof the Experimental Analysis of Behavior 1969; 12:677-687.32. Shaw RS. The dripping faucet as a model chaotic system. Santa Cruz CA: Aerial Press,1984.33. Skinner BF. The behavior of organisms. New York: Appleton-Century-Crofts 1938.34. Suzuki H, Aihara K, Murakami J, Shimozawa T. Analysis of neural spike withinterspike interval reconstruction. Biological Cybernetics 2000; 82:305-31135. Takens F. Detecting strange attractors in turbulence, In: Rand DA, Young LS, editors.Lecture notes in mathematics: dynamical systems and turbulence 898. Berlin: Springer–Verlag, 1981; 336-381.36. Takens F. On the numerical determination of the dimension of an attractor. In:Braaksma Boele LJ, editors. Lecture notes in mathematics: dynamical systems andbifurcations 1125. Berlin: Springer–Verlag, 1985; 99-106.37. Theiler J, Eubank S, Longtin A, Garldrikian B, Farmer JD. Testing for nonlinearity intime series: the method of surrogate data. Physica D 1992; 58:77-94.38. Weiss B. The fine structure of operant behavior during transition states, In: SchoenfeldWN, editor. The theory of reinforcement schedules. New York: Appleton-Century-Crofts, Educational Division, Meredith Corporation, 1970; 277-311.26

More magazines by this user
Similar magazines