Skip to main content
  • Other Publications
  • Subscribe
  • Contact Us
Advertisement
JCORE Reference
this is the JCORE Reference site slogan
  • Home
  • Most Read
  • About Us
    • About Us
    • Editorial Board
  • More
    • Advertising
    • Alerts
    • Feedback
    • Folders
    • Help
  • Patients
  • Reference Site Links
    • View Regions
  • Archive

The Complexity of Childhood Development: Variability in Perspective

Beatrix Vereijken
DOI: 10.2522/ptj.20100019 Published 1 December 2010
Beatrix Vereijken
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • PDF
Loading

Abstract

The complexity of childhood development is exemplified in the variability of development that is seen across tasks and individuals. Furthermore, variability in performance is omnipresent within individuals across repetitions of a task and across individuals performing the same task. Previously, this variability was thought to reflect error of measurement or error of execution. On this account, variability reflects noise that should be filtered or averaged out of the data in order to reveal the “true” underlying characteristics of the performance. Although errors of measurement and execution indeed contribute to variability in movements, research in the last 2 decades has revealed characteristics of variability that are far more interesting than just noise. These characteristics can be deeply informative about underlying control processes and point to directions for clinical practice. This perspective article reviews different ways of characterizing variability, illustrates changes in variability as a result of development and learning, and discusses different theoretical perspectives on the role of variability that give clues about how to understand changes in variability and how to deal with variability in clinical settings.

Children's motor development typically is described as an organized sequence of developmental milestones that children neatly pass through on their way to adulthood. The numerous milestones in this orderly march were described in great detail by pioneering researchers such as Gesell,1 McGraw,2 and Shirley.3 Their early work provided the foundation for the subsequent deduction of normative charts that carefully depict the “typical” ages and stages when children who are developing “typically” achieve the major motor milestones of posture, dexterity, and locomotion.4,5 Behind the simplicity of these orderly charts, however, lies a reality that is far more complex and variable, even for “the typically developing child.” Not only do the ages vary at which individual children achieve the various milestones—thus, the age bands in the normative timetables—but the sequence of milestones itself is not obligatory and can vary considerably from child to child.6 Infants can skip stages temporarily or altogether, they can briefly sample a more advanced milestone but not converge on the new skill consistently until several days or even weeks later, and they can skip back and forth between the various stages, achieving more advanced milestones before the less advanced milestones.

As soon as one is willing to move away from normative data and sample averages, the complexity of childhood development unveils itself in the many individual pathways children weave through development. Close observation of the development of reaching, for example, reveals that in order to grasp a toy, some infants need to damp down the excessive force of their arm movements, whereas other, quieter infants need to increase force generation.7 Observation of the same infants in unconstrained free play illustrates the need for postural control in order for manual skills to emerge, but here, as well, individual infants craft individual pathways.8 Research on the development of locomotion reflects similar complexity. Although the various achievements of locomotor forms are neatly ordered in normative timetables, real infants in the real world hardly ever show a clean break between a less advanced and a more advanced locomotor skill. On the contrary, different infants travel different paths, from precursors such as rolling over and rocking back and forth to different forms of crawling, supported locomotion, and finally independent locomotion.6,9 Along the way, they can demonstrate several precursor skills or none, brief instantiations of a later skill, repeated returns to old skills, but ultimately more and more consistent use—and subsequent improvement—of upright locomotion.10

In the past, all this variability between infants and repetitions was preferably dismissed as reflecting errors of some kind that should be filtered or averaged out of the data in order to reveal the “true” underlying characteristics that were considered invariable across infants and situations. Nowadays, however, we realize more and more that the variability we observe between infants developing the same skill and within infants repeating the same skill—over and over—is not merely an error of development, of immature executive function, or of our measuring instruments. Although such errors do indeed contribute to variability in behavior, research in the last 2 decades has revealed characteristics of variability that are far more interesting than just error and deeply informative about the underlying developmental process. This article takes a closer look at these other characteristics of variability and discusses their implications for childhood development and clinical applications. The first section takes a closer look at different ways of characterizing variability. The subsequent section illustrates how both the amount and quality of variability change as a result of learning and development. The final section discusses different theoretical perspectives on the role of variability and how variability can be dealt with and exploited in clinical settings.

Characterizing Variability

Variability in behavior is not something we observe on the odd occasion. On the contrary, we have come to realize that variability is an inherent characteristic of every move we make and repeat. Variability is the very stuff that evolution is made of, with individual variation, so-called biodiversity, forming the cornerstone of Darwin's evolutionary theory.11 Likewise, when different children find different solutions to the same challenge, inter-individual variability or diversity is present.12 When the same child tries out different solutions to the same challenge, we speak of variety of strategies or dispersion,12 which is one form of intra-individual variability. Another form of intra-individual variability is observed when the same person repeats the same movement several times. As Bernstein pointed out several decades ago, repetition of the same movement never leads to exactly the same movement trajectory, regardless of the amount of practice, experience, or skill level.13 These intra-individual variations in repetitions, therefore, can be said to reflect inconsistency of performance. Variability thus comes in many guises and can be found at many different levels. Below, we will take a closer look at different attempts to characterize variability.

The Good, the Bad, and the Neutral

Although variability can represent inconsistency and errors, not all variability in performance is necessarily detrimental to the outcome. Consider, for example, the tasks of picking up an object or having to navigate a sloping surface. In order to grasp a delicate object without dropping or crushing it, children need to accurately scale their load and grip forces to the characteristics of the object.14 Similarly, in order to navigate a sloping surface safely without picking up uncontrollable momentum, children need to adapt step frequency and step length to find the proper balance between their skill level and the steepness of the slope.15 These kinds of variations in performance reflect necessary adaptations to changes in the biomechanical properties of the growing musculoskeletal system or changes in the environmental conditions. The resulting variability in performance is essential to preserve the outcome and thus can be considered functional or “good” variability.

Another example of functional variability is having a variety of strategies available to accomplish the same task, thereby increasing the flexibility of performance. Examples of this can be found in locomotion, where children can choose to crawl, walk, or run across a room,9,10,16 or in throwing a ball, which can be done bimanually, unimanually, underhand, or overhand,17 depending on, for example, skill level or motivation. Functional variability also is found when different components that contribute to performance counterbalance each other's variations so as to secure the outcome. An example of this type of variability is found in expert marksmen, where variable shoulder and wrist movements complement each other, thereby stabilizing the pistol and thus the aim on a target.18 Good variability in performance, therefore, is useful, and it is not necessary or even desirable to try to minimize it.

In contrast, variability in performance also can hurt the outcome and even cause insurmountable problems. This is the case in most precision tasks such as aiming and microsurgery, but it also can happen in everyday tasks such as locomotion. A newly walking child can easily become overwhelmed with the complexity of the new undertaking, and variability in postural control, muscular activity, and limb movements inevitably adds up to repeated falls.19,20 Several months earlier, when this child was on the brink of reaching for an object, variability in movement control likewise presented severe problems. The child's flailing arm movements typically ended up colliding with the object or missing it all together.21 This kind of “bad” variability in movement control impairs performance and can result in falls, missed targets, or other failures to achieve the desired outcome. Minimizing bad variability is thus a must for reliable performance.

Finally, we can observe and measure a good deal of variability in performance that neither helps nor hurts the outcome. There may be small variations in effectors used, muscles activated, or forces generated, without any of these variations improving the performance or decreasing the consistency of the outcome. This kind of variability thus can be classified as being “neutral” to the performance and can be ignored without compromising the outcome. The big question is: When we come across variability in our observations and measurements, how do we know which kind of variability it is?

The Structure of Variability

There are several methods available for measuring the amount of variability, such as the standard deviation of a variable or coefficient of variation, but how can we judge the quality of variability? Preferably, we should have an analysis method that can allocate instances of variability to the appropriate good, bad, and neutral bins. Ideally, we should be able to do this without additional concern or knowledge about the person, the context, and the task. Unfortunately, no such method exists, not in the least because most variability we come across is person-specific, task-specific, and context-specific. Unless we know who was trying to do what and under which circumstances, how could we judge either the performance or its variability? When looking at a trail of footprints, for example, lack of step-to-step variability might be indicative of the exquisite skill of a tight-rope artist, or the rigid stereotypy of a clinical patient. Likewise, the presence of step-to-step variability may reflect the flexible adaptation of a hiker using stepping stones to cross a river, or the clumsy first steps of a wobbling, newly walking infant. What methods do we have available to help us judge the quality of variability?

An obvious place to start when trying to understand the quality, or structure, of variability is by separating variability in execution variables from variability in outcome variables. As long as the outcome is preserved and the task is accomplished, variability in execution is either functional or harmless—in other words, good or neutral. Furthermore, when execution variables co-vary so as to stabilize the outcome, they share a functional relationship called a “synergy.”22 This is the starting point for the so-called uncontrolled manifold concept, a computational method for distinguishing good variability from bad variability in a given task by identifying controlled and uncontrolled variables.23 Consider, for example, the simple task of exerting a specific amount of force using 2 fingers. As long as force changes in one finger are complemented by equal but opposite force changes in the other finger, the 2 fingers together will produce a stable force output. Similarly, pointing at a target can be achieved by many different configurations of the shoulder, elbow, and wrist joints. The combination of all configurations that lead to accurate pointing form the task solution or manifold. Variability that occurs along the manifold provides flexibility without increasing outcome variability or decreasing accuracy of pointing.22,23 In contrast, variability away from the manifold leads to increased outcome variability and errors in pointing. The uncontrolled manifold concept is an elegant and promising tool that enables quantification of synergies and has led to important insights regarding the structure of variability and strategies of motor control. However, its application requires that the functional relationship between execution and outcome variables be known, implying major simplifications and the use of rather artificial tasks.22

Clinical applications so far have been limited as well but produced some promising results. For example, the higher variability in motor patterns displayed by people with Down syndrome generally is interpreted as a bad thing. However, using the uncontrolled manifold concept, Black and colleagues24 were able to show that part of this increased variability was surprisingly functional and contributed to the body's stability during treadmill walking. Similarly, underneath the difficulty people with poststroke hemiparesis have in reaching with their affected arm, strong multijoint synergies are nevertheless present to stabilize the path of the hand across reaches.25 In contrast, the higher variability seen in elderly people in a 4-finger force production task turned out to reflect an impaired ability to coordinate individual digits, which may be a factor contributing to generally degraded performance of everyday manual tasks.26

As the previous examples illustrate, increased variability can reflect an underlying problem but also the solution to a problem, stressing again the need to consider the structure of variability and not just the amount. A second approach highlighted here focuses on the structure of variability in terms of its complexity. Two such complexity measures in particular have provided valuable results with respect to clinical assessment and differentiation of healthy from pathological conditions: the Lyapunov exponent and approximate entropy.27,28 The Lyapunov exponent measures stability by quantifying the rate of separation or divergence of initially close movement trajectories. Rigid, repetitive movements are characterized by overlapping trajectories that do not diverge, indicated by a maximal Lyapunov exponent close to zero. This finding points to minimal change in the structure of variability and a highly regular, predictable system with little or no flexibility. Lyapunov exponents for random white noise, on the other hand, are considerably larger, whereas those for complex, flexible systems are in between these 2 extremes.

The second measure, approximate entropy, quantifies the degree of regularity or predictability in a time series. Values close to 2 represent maximum irregularity and independence between data points, such as in white noise, whereas values close to zero reflect maximum regularity and dependence between data points, such as in a periodic signal. Again, values in between these 2 extremes reflect complex systems with some, but not complete, dependence between data points, thereby allowing the system to balance between flexibility and stability.

Both measures have provided important new insights about movement function and dysfunction. For example, the typical initial instability of infants learning to sit was shown to be not just noise and random wobbling, but to contain elements of orderly patterns and early strategies for postural control.29 In a similar study of infants with typical versus delayed development, the only measure of several that could differentiate between the 2 groups was the Lyapunov exponent. The exponent was smaller for infants with delayed development, indicating that their swaying patterns while sitting were more repetitive and thus less adaptive.30 Studies such as these illustrate that complexity measures can quantify patterns in the variability of performance that range from rigid to chaotic and that a certain amount of variability can be crucial and necessary for the development of new skills.31

The third and final approach highlighted here is the fractal analysis approach. Like approximate entropy, it is used to investigate the structure of variability by focusing on complex, interdependent fluctuations in time series. However, unlike approximate entropy, which can be calculated from relatively short time series, fractal analyses require extended time series in order to reveal the hidden structure of variability. Several examples of fractal analysis can be found in the extensive literature on stride-to-stride variability in gait.32 Analyzing the fluctuations of successive stride intervals indicated that these intervals contained long-range, power-law correlations.33 This finding points to a fascinating characteristic of normal gait: variations in the current step are influenced not only by the immediately preceding step or 2, but by steps at remote previous times as well.33 In other words, instabilities and perturbations in the current step still reverberate in our gait pattern hundreds of steps later.32 The power-law signature indicates that this influence decays across steps, with nearby steps having stronger influence than faraway steps.

Fractal analyses of stride interval fluctuations have been applied successfully in studies of aging and neurological diseases, leading to important new insights about changes in underlying control processes. For example, gait patterns of elderly people who were healthy were shown to be less fractal and less long-range correlated than gait in younger adults who were healthy. This finding suggests that even healthy aging is associated with gait fluctuations becoming more random and less structured.34 In people with Huntington disease, a similar change toward less correlated and more random gait fluctuations was observed that was directly related to the degree of functional impairment.34 These findings indicate that the functional integrity of the entire system should be an important part of clinical assessment and intervention.

Examples of long-range, power-law correlations also are found in the literature on variability in repeated responses in simple response tasks such as reaction time, interval estimation, mental rotation, and serial and parallel searches.35,36 Just as stride intervals in gait are correlated over hundreds of steps, so are response times over a thousand key strokes. Recent extensions of this work used wavelet-based multifractal analyses to show that in addition to the long-range correlations, most response series contained a combination of periods with small, regular fluctuations and periods with large, irregular fluctuations that aligned across timescales. This nonhomogeneous structure indicates that variability is multifractal, which is caused by underlying interactions between timescales.37 These interactions and alignment of variability become especially strong when performance is about to deteriorate beyond a critical level, thereby enabling functional adaptations of the system in order to rectify performance.37 In a subsequent study on prolonged, relaxed standing in young adults who were healthy, similar alignment of postural sway variability and interactions between timescales were shown to coincide with shifts in posture that restored balance (Ihlen EAF, Lund S, Vereijken B; unpublished research). These findings suggest that in a healthy system, posture is controlled in an intermittent, ballistic-like fashion rather than continuously.

Current work applies these analyses of (multi)fractal structure and temporal alignment of fluctuations across timescales to different clinical populations. Questions addressed include whether we observe a change from intermittent postural control to more continuous postural control with age and disease, which would inform about underlying changes in motor control. Similarly, the buildup of variability alignment across timescales and the subsequent reaction of the system to rectify performance, or its breakdown, might be changed with age and disease, again pointing to underlying changes in motor function and control.

Developmental Changes in Variability

During the processes of learning and development, both the amount and the quality of variability can change. Typically, variability in some variables decreases and reflects increased consistency in outcome while variability in other variables increases and reflects increased flexibility of performance. In addition, aging is associated with increased variability and may reflect increased individuality of performance. With advancing age, older adults increasingly face changes in physical functioning. Although some skills continue to develop and improve with age, muscle strength (force-generating capacity) and functional mobility steadily decline38 and bodily organs and systems become increasingly affected by disease. All the strengths and weaknesses of each individual add up to a unique constellation of possibilities and limitations. Performance will reflect the influence of these individual constraints and may reflect individual solutions and thus high variability among elderly people. This section takes a closer look at the 2 processes occurring during childhood—increased consistency and increased flexibility—and provides examples that highlight these changes in variability.

Increased Consistency in Performance

The best-known change in variability with increased experience and skill is an increase in consistency of performance. With practice, we learn which solutions fit a given challenge, and we learn to further improve upon these solutions. Our movements typically become smoother, more economical, and more proficient, and these improvements, in turn, lead to more consistent outcomes. Newborn infants face a profusion of challenges ahead that need to be tackled with an initially very limited repertoire of skills. Not in the least, they have to figure out that they have a body, what its characteristics are, and how to control its movements.39 The first attempts at any movement are notoriously awkward and often unsuccessful, but infants are nothing if not very persistent. Repeated attempts that literally amount to thousands of trials eventually lead to proficient skills, just as repeated practice improves skills in adult athletes, musicians, and chess players. When 3-month-old infants are supported on a treadmill, their initial stepping movements are irregular and infrequent, and exhibit inconsistent timing between the 2 legs.40,41 However, even a month of repeated practice induces regular, frequent stepping movements, with consistent alternative phasing between the legs.41

Around the end of the first year, children taking their first independent steps present an imposing picture of thrashing legs, waving arms, and wobbling upper bodies that repeatedly move outside their limits of stability, resulting in endless crashes to the floor or into loving arms of parents. With increased experience, the upper body is gradually stabilized, the arms increasingly contribute to the task rather than creating disruptive forces, and subsequent steps become more regular and unswerving in a progressively straighter line to the target.10,42 Looking at the level of the outcome, developmental progression thus is clearly characterized by a decrease in variability. However, underneath the steady decrease in outcome variability and increase in outcome consistency, variability across infants persists. This variability is partly caused by infants having different body anthropometrics and, therefore, different constraints that influence their set of possible solutions.43 In addition, infants have different prior experiences, temperaments, levels of motivation, and so on, each contributing to a range of idiosyncratic solutions across infants and situations. Getting rid of this kind of variability is not a feature of developmental progress, as will be illustrated next.

Increased Flexibility of Performance

As skill level increases as a result of practice and experience, so does the ability to withstand internal or external perturbations. For example, newly walking infants typically do not cope well with sloping surfaces, but repeated practice gradually equips them with the skills and wherewithal to prospectively adapt their posture and gait to these changes in the environment.15 Similarly, obstacles in the environment are not reliably avoided at first, but children learn to adjust their trunk orientation and footsteps while approaching the obstacle, and later to clear around it in one fluent chain of adjustments.44 In adult studies as well, repeated practice leads to increased flexibility of performance. If a top table tennis player initiates an attacking forehand drive slightly later, he slightly accelerates the drive so as to stabilize temporal accuracy of ball/bat contact.45 If the shoulder moves too far to the left in expert marksmen, the wrist compensates to the right to secure a steady aim on the target.18 As was outlined above, in order for this kind of flexibility to emerge, subcomponents contributing to task performance need to be coordinated so that variation in one subcomponent is counteracted by variation in other subcomponents.

The above examples reflect how variability can be functional and increase flexibility at the execution level. In addition, there is functional variability that increases flexibility at the outcome level by having multiple solutions available to achieve the same goal. My colleagues and I came across a striking example of this in an earlier study on crawling, in which we observed no fewer than 25 different patterns of belly crawling.9 Across multiple weeks of belly crawling, infants kept coming up with new combinations of body parts to be used for propulsion and support. Even after 10 weeks of crawling on their belly, infants still discovered new patterns they had not displayed before. In sharp contrast to this wealth of available patterns of belly crawling, all infants converged almost instantly to the standard pattern of diagonal hands-and-knees crawling once they lifted their bellies off the ground. This consistency of hands-and-knees crawling may have been induced by the biomechanical constraints of keeping balance in mid-air while moving forward, with a diagonal crawling pattern representing the least destabilizing movement pattern.9 By the time we reach adulthood, we have fully developed bodies, a plethora of available skills, and plenty of experience that we can draw upon. Although adults are perfectly capable of displaying variability in outcome, performance often reflects the influence of similar constraints, leading to similar solutions and thus low inter-individual variability.

Variability and Timescales of Analyses

There is an important relationship between issues of variability and timescales of analysis. As was described above, averaging across multiple repetitions of a task deletes variability and all information that is contained in that variability, including its structure. A similar loss of information happens when we study processes of change resulting from development or disease by sampling at coarse intervals. The coarsest interval is the before-and-after snapshot, indicating at best that a change has happened, but stripped bare of all information concerning the trajectory of that change. U-shaped development would be blatantly missed by the before-and-after sampling interval, as would any other information about the shape of developmental change.46 The same word of caution applies to studies about variability. Obviously, sampling over increasingly coarser intervals reduces precision and sensitivity to fluctuations in the data. More importantly though, sensitivity is affected in power-like fashion and drops off surprisingly rapidly,46 indicating that sampling frequency may need to be as high as costs for the researcher and taxation for the subjects permit. Furthermore, the relationship among variability at different timescales can be highly informative. When measures of variability (such as standard deviation or variance) across increasingly frequent sampling intervals reach an asymptote, the sampling frequency is sufficient to capture the phenomenon of interest, and further increases in sampling frequency will not reveal further detail. When measures of variability increase linearly with increasing sampling frequency, the underlying structure is characterized by a 1/f power relationship.47

Finally, when measures of variability do not converge to a limit with increasing sampling frequency, the underlying structure is fractal.37 Looking at the rate at which variability changes across different sampling frequencies thus provides valuable information about the complexity of the underlying control system and how densely spaced data collections should be in order to capture it.

The Role of Variability

So far, we have seen that variability in performance can be both functional and disruptive and that variability can both decrease and increase with development and learning, reflecting increased consistency and increased flexibility, respectively. How should we deal with variability when we come across it in clinical settings? Should we encourage variability or try to minimize it? Can we somehow benefit from it? This final section will review 3 theoretical perspectives that deal explicitly with the role of variability in development and learning that can help formulate answers to these questions.

Variability and Selection

The first theoretical perspective draws upon Darwin's insight that variation is necessary for selection to take place and for species to evolve. Edelman based his theory of brain development and function, or neuronal group selection, on the same idea of variation and selection.48,49 The cornerstone of Edelman's theory is that organisms start with enormous neural diversity in terms of cell type, number, and connectivity. This neural diversity forms the raw material for subsequent experience-based selection. Neurons that fire together wire together,50 and dense, reciprocal connections between neurons gradually lead to the formation of neuronal groups.48 The temporal and spatial coherence of sensory input signals leads to sensations that are correlated across perceptual domains. Continued perceptual and motor experience molds the brain by selecting and strengthening certain neuronal groups while deselecting and weakening other groups of neurons that are less adaptive for the task at hand.48 Sporns and Edelman51 applied this framework to early motor development, postulating that the brain's initial structural variability allows for the dynamic variability that is seen in the many different possible outputs of the system. The infant's initial spontaneous movements form a basic repertoire from which movements can be selected as a solution to a given task. With continued selection and subsequent feedback, movements become more and more adaptive to ongoing changes in biomechanical characteristics or environmental demands.51

The application of Edelman's theory of neuronal group selection, particularly the ideas about variation and selection, has led to interesting hypotheses and new insights in clinical research. For example, the reduced variation typically seen in children with cerebral palsy or developmental coordination disorder might be caused by a limited repertoire of neuronal networks or impaired selection.52 Furthermore, recent developments in studies of preterm infants suggest that diffuse damage of the cerebral white matter might be associated with an overall reduction in movement variability and an increase in stereotyped behavior.53 On the account of variation and selection, variability is thus necessary and a hallmark of normal development, whereas the lack of variability can be indicative of neurological and developmental disorders. Clinical interventions in these cases should be geared toward helping children to increase the variability of their repertoire and stimulating them to create and explore a variety of possible solutions.

Variability and Transitions

Few theoretical approaches are as well equipped to handle processes of change in complex systems as dynamic systems theory. Originally developed in physics and mathematics to study the emergence of organized patterns in complex systems, it has found broad application in fields as diverse as biology, weather systems, and the stock market. The basic tenet of dynamic systems theory is that interactions among the elements of a system give rise to patterns that are organized in space and time without the need for instructions or prescriptions, so-called self-organization. The formed patterns can be envisioned as attractors in a landscape of possible organizations.54 Changes from one attractor to another are made possible by destabilization of the current attractor and a subsequent transition to a new attractor. Most importantly in the current context, decreasing stability of an attractor is accompanied by an increase in the variability of that pattern, which subsequently facilitates change to a different pattern. An upcoming transition thus is signaled by increased variability. In contrast, lack of variability traps the system in a specific attractor, leading to rigid stability.

One of the first proponents of a dynamic systems approach to child development was Thelen.40,55 She used the ideas and principles of dynamic systems theory as an inspiration for unusual study designs and alternative interpretations of well-known phenomena. For example, by adding little weights to the legs of newborn infants or submerging their legs in water, she was able to reinterpret the U-shaped development of infant stepping as a reflection of asynchronous growth of leg mass and muscle strength rather than cortical inhibition of infantile reflexes.56 Similarly, a series of reaching experiments led to a reinterpretation of the classic “A-not-B error”57 in terms of reaching dynamics, particularly perseveration, rather than invoking the concept of object permanence.58 Dynamic systems approaches to development acknowledge the complexity of childhood by placing environmental, physical, and biomechanical factors, as well as the child's prior history and experiences, on a par with neural and genetic factors as potential explanatory factors. This feature in particular has led many empirical and applied developmental researchers to embrace dynamic systems approaches, and their work shows that reduced variability and increased rigidity can indeed be indicative of disease and disorders.59,60 Again, clinical practice in this spirit would welcome variability, and seek to increase it, as a necessary condition for flexibility and developmental progress.

Controlling and Exploiting Variability

As indicated above, Bernstein13 noted that variability was part and parcel of human movement, irrespective of amount of experience or level of skill. He realized that all movements take place in an environment, and as the details of the environment change from one repetition to the next, so must the details of the repeated performance. He labeled these performance changes “context-conditioned variability” and made it one of the central issues in his work on motor control.

Another central issue that Bernstein13 highlighted was the abundance of degrees of freedom or movement possibilities of the musculoskeletal system. Many different configurations can be used to accomplish a task, but is this a curse or rather a blessing? Bernstein13 formulated explicit hypotheses about how the nervous system can handle the abundant degrees of freedom so that they do not pose a problem at the beginning of a learning process and can be utilized later when expertise has increased. He postulated 3 learning stages that were later corroborated by empirical evidence. In the first stage of learning, the control problem is simplified by reducing the number of degrees of freedom by freezing them or coupling them rigidly together.61 This reduction in degrees of freedom often is visible as stiffening of posture and excessive levels of coactivation during movement, as in a child trying to use new skates for the first time.

As learning progresses, the learner can explore additional degrees of freedom and increasingly incorporate them in the task solution, thereby adding flexibility to the performance.62 Stiffening and coactivation decrease, and movements become more fluent and integrated. When multiple degrees of freedom become functionally linked together so that they behave as a single unit in order to stabilize task performance, they are referred to as a “coordinative structure.”13

In the final stage of learning, expertise has progressed to a level where reactive forces are no longer fought or resisted but actively exploited within the coordinative structures.63 Although most empirical research on learning observes evidence for one or more of these stages, the stages are not obligatory. Whether or not a learner freezes degrees of freedom is influenced by the need or desire to stay in control over reactive forces,6 and we do not necessarily reach the expert stage of exploitation in every single task we set out to learn. Depending on where one is in the learning process, the abundance of degrees of freedom thus can be either a curse that needs to be minimized or a blessing that can be exploited.

The decreases and increases in degrees of freedom closely parallel the observed decreases and increases in variability across development and learning. The child's initial lack of control over an abundance of degrees of freedom leads to high variability in performance and little consistency.64 When degrees of freedom are eliminated in order to minimize disruptive reactive forces, the variability of performance likewise is reduced.65 Exploration of degrees of freedom leads to an increase in performance variability,9 whereas successful exploitation of degrees of freedom leads to increased flexibility and reduced outcome variability. Variability thus can be both a curse and a blessing, depending on one's ability to control it. On this account, developmental progress is not aimed toward reducing variability, but toward learning to control and exploit variability.

Although this framework has not been applied extensively in clinical settings, findings on 2-handed ball catching in children with developmental coordination disorder confirm that they demonstrate tighter intralimb and interlimb couplings and reduced flexibility in motor behavior compared with age-matched controls.66 There also is some evidence of changing levels of stiffness and exploration across the development of walking in toddlers with Down syndrome.67 When encountering increased variability in clinical populations, the practitioner thus should carefully evaluate whether it is part of the solution, indicated by underlying structure in the variability as in infants learning to sit,29 or part of the problem, indicated by a lack of underlying structure and poor consistency of performance, such as in individuals with an ataxic movement disorder.31 Further investigations of controlling and exploiting variability in clinical settings would be useful and might help pinpoint underlying problems of and possible solutions to coordination and motor control.

Conclusions

Variability is omnipresent in behavior and measurements of behavior. Variability can be noise, but more often than not it has inherent structure and meaning. Variability can be problematic when it interferes with accomplishing a task, but it also can be functional by providing flexibility in performance and strategies. Knowing that variability can be all these things provides new ideas and tools for how to study childhood complexity and variability and how to deal with the latter in clinical situations. Practitioners should look at the structure of variability in addition to its amount and carefully decide whether the focus should be on increasing or decreasing variability. Changes in variability as a result of learning, development, or aging indicate that variability can be used, manipulated, and exploited in order to improve performance, either by the performer or by a practitioner trying to influence performance. Practitioners should endeavor to build complexity in the tasks they give their patients, cleverly varying the practice space and encouraging multiple movement approaches. Furthermore, changes in variability as a result of injury or disease can provide clues as to what the underlying problems might be and can point to possible directions for intervention. Finally, with the development of new tools for analyzing the structure of variability in performance, new knowledge and insights about the underlying motor control system and how the latter might be influenced in clinical practice become available.

Footnotes

  • This article is based on a lecture presented at the International Conference for Paediatric Physical Therapy “Variability in Perspective”; November 15, 2008; Amsterdam, the Netherlands.

  • Received January 14, 2010.
  • Accepted June 1, 2010.
  • © 2010 American Physical Therapy Association

References

  1. ↵
    1. Gesell A
    . Reciprocal interweaving in neuromotor development. J Comp Neurol. 1939;70:161–180.
    OpenUrl
  2. ↵
    1. McGraw MB
    . The Neuromuscular Maturation of the Human Infant. New York, NY: Columbia University Press; 1945.
  3. ↵
    1. Shirley MM
    . The First Two Years: A Study of Twenty-five Babies, Vol 1: Postural and Locomotor Development. Minneapolis, MN: University of Minnesota Press; 1931.
  4. ↵
    1. Bayley N
    . Bayley Scales of Infant Development. New York, NY: The Psychological Corp; 1969.
  5. ↵
    1. Frankenburg WK,
    2. Dodds JB
    . The Denver developmental screening test. J Pediatr. 1967;71:181–191.
    OpenUrlCrossRefPubMedWeb of Science
  6. ↵
    1. Savelsbergh GJP,
    2. van der Maas HLJ,
    3. van Geert PCL
    1. Vereijken B,
    2. Adolph KE
    . Transitions in the development of locomotion. In: Savelsbergh GJP, van der Maas HLJ, van Geert PCL eds. Non-linear Developmental Processes. Amsterdam, the Netherlands: Elsevier; 1999:137–149.
  7. ↵
    1. Thelen E,
    2. Corbetta D,
    3. Kamm K,
    4. et al
    . The transition to reaching: mapping intention and intrinsic dynamics. Child Dev. 1993;64:1058–1098.
    OpenUrlCrossRefPubMedWeb of Science
  8. ↵
    1. Spencer JP,
    2. Vereijken B,
    3. Diedrich FJ,
    4. Thelen E
    . Posture and the emergence of manual skills. Dev Sci. 2000;3:216–233.
    OpenUrlCrossRef
  9. ↵
    1. Adolph KE,
    2. Vereijken B,
    3. Denny MA
    . Learning to crawl. Child Dev. 1998;69:1299–1312.
    OpenUrlCrossRefPubMedWeb of Science
  10. ↵
    1. Adolph KE,
    2. Vereijken B,
    3. Shrout PE
    . What changes in infant walking and why. Child Dev. 2003;74:475–497.
    OpenUrlCrossRefPubMedWeb of Science
  11. ↵
    1. Darwin C
    . On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life. London, United Kingdom: John Murray; 1859.
  12. ↵
    1. Hultsch DF,
    2. MacDonald SW,
    3. Dixon RA
    . Variability in reaction time performance of younger and older adults. J Gerontol B Psychol Sci Soc Sci. 2002;57:P101–P115.
    OpenUrlAbstract/FREE Full Text
  13. ↵
    1. Bernstein NA
    . The Co-ordination and Regulation of Movements. London, United Kingdom: Pergamon Press; 1967.
  14. ↵
    1. Forssberg H,
    2. Eliasson AC,
    3. Kinoshita H,
    4. et al
    . Development of human precision grip. I: basic coordination of force. Exp Brain Res. 1991;85:451–457.
    OpenUrlPubMedWeb of Science
  15. ↵
    1. Gill SV,
    2. Adolph KE,
    3. Vereijken B
    . Change in action: how infants learn to walk down slopes. Dev Sci. 2009;12:888–902.
    OpenUrlCrossRefPubMedWeb of Science
  16. ↵
    1. Whitall J,
    2. Getchell N
    . From walking to running: applying a dynamical systems approach to the development of locomotor skills. Child Dev. 1995;66:1541–1553.
    OpenUrlCrossRefPubMedWeb of Science
  17. ↵
    1. Marques-Bruna P,
    2. Grimshaw PN
    . Variability in development of overarm throwing: a longitudinal case study over the first 6 months of throwing. Percept Mot Skills. 1998;86(3 pt 2):1403–1418.
    OpenUrlPubMedWeb of Science
  18. ↵
    1. Arutyunyan GH,
    2. Gurfinkel VS,
    3. Mirskii ML
    . Organization of movements on execution by man of an exact postural task. Biofizika. 1969;14:1162–1167.
    OpenUrl
  19. ↵
    1. Vereijken B,
    2. Pedersen AV,
    3. Størksen JH
    . Early independent walking: a longitudinal study of load perturbation effects. Dev Psychobiol. 2009;51:374–383.
    OpenUrlCrossRefPubMedWeb of Science
  20. ↵
    1. Joh AS,
    2. Adolph KE
    . Learning from falling. Child Dev. 2006;77:89–102.
    OpenUrlCrossRefPubMedWeb of Science
  21. ↵
    1. Thelen E,
    2. Corbetta D,
    3. Spencer JP
    . Development of reaching during the first year: role of movement speed. J Exp Psychol Hum Percept Perform. 1996;22:1059–1076.
    OpenUrlCrossRefPubMedWeb of Science
  22. ↵
    1. Latash ML,
    2. Anson JG
    . Synergies in health and disease: relations to adaptive changes in motor coordination. Phys Ther. 2006;86:1151–1160.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    1. Scholz JP,
    2. Schöner G
    . The uncontrolled manifold concept: identifying control variables for a functional task. Exp Brain Res. 1999;126:289–306.
    OpenUrlCrossRefPubMedWeb of Science
  24. ↵
    1. Black DP,
    2. Smith BA,
    3. Wu J,
    4. Ulrich BD
    . Uncontrolled manifold analysis of segmental angle variability during walking: preadolescents with and without Down syndrome. Exp Brain Res. 2007;183:511–521.
    OpenUrlCrossRefPubMedWeb of Science
  25. ↵
    1. Reisman DS,
    2. Scholz JP
    . Workspace location influences joint coordination during reaching in post-stroke hemiparesis. Exp Brain Res. 2006;170:265–276.
    OpenUrlCrossRefPubMedWeb of Science
  26. ↵
    1. Shinohara M,
    2. Scholz JP,
    3. Zatsiorsky VM,
    4. Latash ML
    . Finger interaction during accurate multi-finger force production tasks in young and elderly persons. Exp Brain Res. 2004;156:282–292.
    OpenUrlCrossRefPubMedWeb of Science
  27. ↵
    1. Stergiou N
    . Innovative Analyses of Human Movement. Champaign, IL: Human Kinetics; 2004.
  28. ↵
    1. Pincus S
    . Approximate entropy (ApEn) as a complexity measure. Chaos. 1995;5;110–117.
    OpenUrlCrossRefPubMedWeb of Science
  29. ↵
    1. Harbourne RT,
    2. Stergiou N
    . Nonlinear analysis of the development of sitting postural control. Dev Psychobiol. 2003;42:368–377.
    OpenUrlCrossRefPubMedWeb of Science
  30. ↵
    1. Deffeyes JE,
    2. Harbourne RT,
    3. Kyvelidou A,
    4. et al
    . Nonlinear analysis of sitting postural sway indicates developmental delay in infants. Clin Biomech (Bristol, Avon). 2009;24:564–570.
    OpenUrlCrossRefPubMed
  31. ↵
    1. Harbourne RT,
    2. Stergiou N
    . Movement variability and the use of nonlinear tools: principles to guide physical therapist practice. Phys Ther. 2009;89:267–282.
    OpenUrlAbstract/FREE Full Text
  32. ↵
    1. Hausdorff JM
    . Gait dynamics, fractals and falls: finding meaning in the stride-to-stride fluctuations of human walking. Hum Mov Sci. 2007;26:555–589.
    OpenUrlCrossRefPubMedWeb of Science
  33. ↵
    1. Hausdorff JM,
    2. Peng CK,
    3. Ladin Z,
    4. et al
    . Is walking a random walk: evidence for long-range correlations in stride interval of human gait. J Appl Physiol. 1995;78:349–358.
    OpenUrlAbstract/FREE Full Text
  34. ↵
    1. Hausdorff JM,
    2. Mitchell SL,
    3. Firtion R,
    4. et al
    . Altered fractal dynamics of gait: reduced stride-interval correlations with aging and Huntington's disease. J Appl Physiol. 1997;82:262–269.
    OpenUrlAbstract/FREE Full Text
  35. ↵
    1. Gilden DL
    . Fluctuations in the time required for elementary decision. Psychol Sci. 1997;8:296–301.
    OpenUrlAbstract/FREE Full Text
  36. ↵
    1. Gilden DL
    . Cognitive emissions of 1/f noise. Psychol Rev. 2001;108:33–56.
    OpenUrlCrossRefPubMedWeb of Science
  37. ↵
    1. Ihlen EAF,
    2. Vereijken B
    . Interaction-dominant dynamics in human cognition: beyond 1/fα fluctuation. J Exp Psychol Gen. 2010;139(3):436–463.
    OpenUrlCrossRefPubMedWeb of Science
  38. ↵
    1. Samson MM,
    2. Meeuwsen IB,
    3. Crowe A,
    4. et al
    . Relationships between physical performance measures, age, height and body weight in healthy adults. Age Ageing. 2000;29:235–242.
    OpenUrlAbstract/FREE Full Text
  39. ↵
    1. Sheets-Johnstone M
    . The Primacy of Movement. Amsterdam, the Netherlands: John Benjamins Publishing Co; 1999.
  40. ↵
    1. Thelen E,
    2. Ulrich BD
    . Hidden skills: a dynamic systems analysis of treadmill stepping during the first year. Monogr Soc Res Child Dev. 1991;56:1–98.
    OpenUrlPubMed
  41. ↵
    1. Vereijken B,
    2. Thelen E
    . Training infant treadmill stepping: the role of individual pattern stability. Dev Psychobiol. 1997;30:89–102.
    OpenUrlCrossRefPubMedWeb of Science
  42. ↵
    1. Ledebt A,
    2. Bril B
    . Acquisition of upper body stability during walking in toddlers. Dev Psychobiol. 2000;36:311–324.
    OpenUrlCrossRefPubMedWeb of Science
  43. ↵
    1. Wade MG,
    2. Whiting HTA
    1. Newell KM
    . Constraints on the development of coordination. In: Wade MG, Whiting HTA eds. Motor Development in Children: Aspects of Coordination and Control. Dordrecht, the Netherlands: Nijhoff; 1986:341–360.
  44. ↵
    1. Vallis LA,
    2. McFadyen BJ
    . Children use different anticipatory control strategies than adults to circumvent an obstacle in the travel path. Exp Brain Res. 2005; 167:119–127.
    OpenUrlCrossRefPubMedWeb of Science
  45. ↵
    1. Bootsma RJ,
    2. Van Wieringen PC
    . Timing an attacking forehand drive in table tennis. J Exp Psychol Hum Percept Perform. 1990;16:21–29.
    OpenUrlCrossRef
  46. ↵
    1. Adolph KE,
    2. Robinson SR,
    3. Young JW,
    4. Gill-Alvarez F
    . What is the shape of developmental change? Psychol Rev. 2008;115:527–543.
    OpenUrlCrossRefPubMedWeb of Science
  47. ↵
    1. Van Orden GC,
    2. Holden JG,
    3. Turvey MT
    . Self-organization of cognitive performance. J Exp Psychol Gen. 2003;132:331–350.
    OpenUrlCrossRefPubMedWeb of Science
  48. ↵
    1. Edelman GM
    . Neural Darwinism: The Theory of Neuronal Group Selection. New York, NY: Basic Books; 1987.
  49. ↵
    1. Edelman GM
    . Bright Air, Brilliant Fire: On the Matter of the Mind. New York, NY: Basic Books; 1992.
  50. ↵
    1. Hebb DO
    . The Organization of Behavior. New York, NY: John Wiley & Sons Inc; 1949.
  51. ↵
    1. Sporns O,
    2. Edelman GM
    . Solving Bernstein's problem: a proposal for the development of coordinated movement by selection. Child Dev. 1993;64:960–981.
    OpenUrlCrossRefPubMedWeb of Science
  52. ↵
    1. Hadders-Algra M
    . The neuronal group selection theory: promising principles for understanding and treating developmental motor disorders. Dev Med Child Neurol. 2000;42:707–715.
    OpenUrlCrossRefPubMedWeb of Science
  53. ↵
    1. Hadders-Algra M
    . Reduced variability in motor behaviour: an indicator of impaired cerebral connectivity? Early Hum Dev. 2008;84:787–789.
    OpenUrlCrossRefPubMedWeb of Science
  54. ↵
    1. Rovee-Collier C,
    2. Lipsitt LP
    1. Muchisky M,
    2. Gershkoff-Stowe L,
    3. Cole E,
    4. Thelen E
    . The epigenetic landscape revisited: a dynamic interpretation. In: Rovee-Collier C, Lipsitt LP eds. Advances in Infancy Research, vol 10. Norwood, NJ: Ablex; 1996:121–159.
    OpenUrl
  55. ↵
    1. Thelen E,
    2. Smith LB
    . A Dynamic Systems Approach to the Development of Cognition and Action. Cambridge, MA: MIT Press; 1994.
  56. ↵
    1. Thelen E,
    2. Fisher DM,
    3. Ridley-Johnson R
    . The relationship between physical growth and a newborn reflex. Infant Behav Dev. 1984;7:479–493.
    OpenUrlCrossRefWeb of Science
  57. ↵
    1. Piaget J
    . The Construction of Reality in the Child. New York, NY: Basic Books; 1954.
  58. ↵
    1. Thelen E,
    2. Schöner G,
    3. Scheier C,
    4. Smith LB
    . The dynamics of embodiment: a field theory of infant perseverative reaching. Behav Brain Sci. 2001;24:1–34.
    OpenUrlCrossRefPubMedWeb of Science
  59. ↵
    1. Volman MJ,
    2. Geuze RH
    . Stability of rhythmic finger movement in children with a developmental coordination disorder. Motor Control. 1998;2:34–60.
    OpenUrlPubMed
  60. ↵
    1. Goldberger AL,
    2. Amaral LA,
    3. Hausdorff JM,
    4. et al
    . Fractal dynamics in physiology: alterations with disease and aging. Proc Natl Acad Sci U S A. 2002;99(suppl 1):2466–2472.
    OpenUrlAbstract/FREE Full Text
  61. ↵
    1. Vereijken B,
    2. van Emmerik REA,
    3. Whiting HTA,
    4. Newell KM
    . Free(z)ing degrees of freedom in skill acquisition. J Mot Behav. 1992;24:133–142.
    OpenUrlCrossRefWeb of Science
  62. ↵
    1. Vereijken B,
    2. van Emmerik REA,
    3. Bongaardt R,
    4. et al
    . Changing coordinative structures in skill acquisition. Hum Mov Sci. 1997;16:823–844.
    OpenUrlCrossRef
  63. ↵
    1. Paavolainen L,
    2. Häkkinen K,
    3. Hämäläinen I,
    4. et al
    . Explosive-strength training improves 5-km running time by improving running economy and muscle power. J Appl Physiol. 1999;86:1527–1533.
    OpenUrlAbstract/FREE Full Text
  64. ↵
    1. Haehl V,
    2. Vardaxis V,
    3. Ulrich BD
    . Learning to cruise: Bernstein's theory applied to skill acquisition during infancy. Hum Mov Sci. 2000;19:685–715.
    OpenUrlCrossRefWeb of Science
  65. ↵
    1. Vereijken B,
    2. Waardenburg M
    . Changing Patterns of Interlimb Coordination From Supported to Independent Walking. Poster presented at: 10th International Conference on Infant Studies; April 1996; Providence, Rhode Island.
  66. ↵
    1. Astill S,
    2. Utley A
    . Two-handed catching in children with developmental coordination disorder. Motor Control. 2006;10:109–124.
    OpenUrlPubMedWeb of Science
  67. ↵
    1. Black D,
    2. Chang CL,
    3. Kubo M,
    4. et al
    . Developmental trajectory of dynamic resource utilization during walking: toddlers with and without Down syndrome [erratum in Hum Mov Sci. 2009;28:551]. Hum Mov Sci. 2009;28:141–154.
    OpenUrlCrossRefPubMedWeb of Science
View Abstract
Back to top
Vol 96 Issue 12 Table of Contents
Physical Therapy: 96 (12)

Issue highlights

  • Musculoskeletal Impairments Are Often Unrecognized and Underappreciated Complications From Diabetes
  • Physical Therapist–Led Ambulatory Rehabilitation for Patients Receiving CentriMag Short-Term Ventricular Assist Device Support: Retrospective Case Series
  • Education Research in Physical Therapy: Visions of the Possible
  • Predictors of Reduced Frequency of Physical Activity 3 Months After Injury: Findings From the Prospective Outcomes of Injury Study
  • Use of Perturbation-Based Gait Training in a Virtual Environment to Address Mediolateral Instability in an Individual With Unilateral Transfemoral Amputation
  • Effect of Virtual Reality Training on Balance and Gait Ability in Patients With Stroke: Systematic Review and Meta-Analysis
  • Effects of Locomotor Exercise Intensity on Gait Performance in Individuals With Incomplete Spinal Cord Injury
  • Case Series of a Knowledge Translation Intervention to Increase Upper Limb Exercise in Stroke Rehabilitation
  • Effectiveness of Rehabilitation Interventions to Improve Gait Speed in Children With Cerebral Palsy: Systematic Review and Meta-analysis
  • Reliability and Validity of Force Platform Measures of Balance Impairment in Individuals With Parkinson Disease
  • Measurement Properties of Instruments for Measuring of Lymphedema: Systematic Review
  • myMoves Program: Feasibility and Acceptability Study of a Remotely Delivered Self-Management Program for Increasing Physical Activity Among Adults With Acquired Brain Injury Living in the Community
  • Application of Intervention Mapping to the Development of a Complex Physical Therapist Intervention
Email

Thank you for your interest in spreading the word on JCORE Reference.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
The Complexity of Childhood Development: Variability in Perspective
(Your Name) has sent you a message from JCORE Reference
(Your Name) thought you would like to see the JCORE Reference web site.
Print
The Complexity of Childhood Development: Variability in Perspective
Beatrix Vereijken
Physical Therapy Dec 2010, 90 (12) 1850-1859; DOI: 10.2522/ptj.20100019

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Save to my folders

Share
The Complexity of Childhood Development: Variability in Perspective
Beatrix Vereijken
Physical Therapy Dec 2010, 90 (12) 1850-1859; DOI: 10.2522/ptj.20100019
del.icio.us logo Digg logo Reddit logo Technorati logo Twitter logo CiteULike logo Connotea logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One
  • Article
    • Abstract
    • Characterizing Variability
    • Developmental Changes in Variability
    • The Role of Variability
    • Conclusions
    • Footnotes
    • References
  • Info & Metrics
  • PDF

Related Articles

Cited By...

More in this TOC Section

  • Perspective on Variability in the Development of Human Action
  • Opportunities for Early Intervention Based on Theory, Basic Neuroscience, and Clinical Science
  • A Comparison of Interventions for Children With Cerebral Palsy to Improve Sitting Postural Control: A Clinical Trial
Show more Pediatrics Special Issue

Subjects

Footer Menu 1

  • menu 1 item 1
  • menu 1 item 2
  • menu 1 item 3
  • menu 1 item 4

Footer Menu 2

  • menu 2 item 1
  • menu 2 item 2
  • menu 2 item 3
  • menu 2 item 4

Footer Menu 3

  • menu 3 item 1
  • menu 3 item 2
  • menu 3 item 3
  • menu 3 item 4

Footer Menu 4

  • menu 4 item 1
  • menu 4 item 2
  • menu 4 item 3
  • menu 4 item 4
footer second
footer first
Copyright © 2013 The HighWire JCore Reference Site | Print ISSN: 0123-4567 | Online ISSN: 1123-4567
advertisement bottom
Advertisement Top