Emerging neuron network technology of associative memories and pattern recognition in coupled oscillator system on the basis of transition metal oxide switching structures.

Project RSF 16-19-00135
A priority:
Neurotechnology and cognitive research
Specific objectives:
P16-3 Development of new assistive and replacement technologies for improving human cognitive abilities
Key issues:
P16-3-2 Development of artificial cognitive systems, including the development of new paradigms and theories of neurocomputers and biosimilar neural networks, large-scale simulators of neural networks, specialized architectures for neuromorphic computing.

The project seeks to implement artificial dynamic neural networks with functions of associative memory and pattern recognition using an array of coupled oscillators. As the object of research and development, we will use structures based on vanadium dioxide with the effects of metal-insulator phase transition and an electric switching, on the basis of which the relaxation oscillators (autogenerators) are designed, and the basis of physical modeling is derived from a model of the network of interacting neural oscillators.
The significance and urgency of the project task is associated primarily with the development of a new paradigm of computing and pattern recognition, and also it has an aspect related to the understanding of the physics of functioning of oxide micro and nano switching structures.  
At the moment, there are several approaches to the modeling of neural networks. First, this is a network of neurons in which the dynamics of each element is described by a system of differential equations, for example, the equation for the ion transport through the membrane as in the Hodgkin – Huxley model, a network of integrative-threshold neurons accumulating the input signals and generating a pulse (spike) when a threshold is reached, and a network of interacting neural oscillators, including phase oscillators.
Within the oscillator approach, the neuron oscillator networks are investigated by the methods of the bifurcation theory, which allows analytical and numerical description of the range of parameters for which there is a particular type of network dynamics. In a simpler approach, the activity of coupled oscillators is characterized by a phase or frequency difference. The main objective of this approach is to describe regions of the parameter space corresponding to different modes of synchronization (full or partial), and the pattern recognition is treated as entering of input parameters into the region of synchronization of interacting oscillators.
‘Artificial neurons' developed in the project, based on vanadium dioxide with the effect of an electric switching, along with relaxation oscillations, also exhibit a natural electrical noise that has sometimes a determinate character. This phenomenon (near room temperature), inherent in real biological objects, in conjunction with the speed and nanoscale possibility, make the switches on the basis of VO2 promising elements to create an artificial neural network of coupled oscillators. The result will be the development of technology of switching structures creation, both of planar and sandwich type, based on vanadium oxides, with the effect of electrical switching, with micron and nano-sized workspaces and electrodes.

One of the scientific problems are to be solved in the project is the development of new methods for entering information into the artificial neural network. It is planned to use the property of vanadium oxide structures to be reversibly modified under the influence of electron-beam irradiation (EBM). This will allow direct managing the dynamics of relaxation oscillations, through a change in the threshold characteristics of the switching element, and the electron lithography system will convert any visible image into a distribution of the exposure dosage. It is also planned to study photo-resistive converters connected in series with the switching elements for the direct impact of lighting from a recognizable image onto the dynamics of oscillatory neural networks.
Another scientific problem is that of the storing of the test object image in the oscillator parameters. Here we will explore oscillator circuits comprising a resistive memory element playing the role of function of system state (frequency, phase), memory followed by recognition of this state. One of the aspects of the problem will be the search for oxide structures possessing multiple stable resistive states, the so-called Multilevel ReRam.
In addition, experimental observations will be complemented by the results of numerical simulation of the dynamics of coupled oscillators, as well as by software and hardware techniques of signal processing.
A technique for creation of an array (with dimension of at least 4*4) of coupled oscillators will be delivered, along with resistive memory elements based on transition metal oxide switching structures. Influence of EBM and photomodification on the oscillation dynamics of the array of coupled oscillator is to be studied to realize the function of associative memory and pattern recognition.
Thus, a new line of research in the development of neural networks is opened based on fundamentally new oscillators not only from the viewpoint of their physical mechanism, but also organized on an entirely different principle of parametric effects. From a practical point of view, this work will contribute to the development of new devices of oxide bio-inspired electronics and information processing methods.
Neuroscience of the modern biology attracts special attention of specialists in mathematics and physics. The number of articles in this area compete with the articles in the areas of molecular genetics and ecotechnologies. One of such areas, namely the theory of neural networks, on the one hand, solves a fundamental problem of general principles of information processing of live organisms, on the other hand,  as a part of modern cybernetics it directs the main efforts at dramatic change in computational paradigm embracing quantum computers and neuroprocessors development. 
The current research in the field of neoroprocessors development are focused on  new computational architectures development based on dynamically adaptive cerebration with massive parallel logics. The backbone of these research is development of devices which are able “to learn” to respond on various external  impacts. 
 Currently there are several approaches to neural network modeling. First of all these are networks of neurons in which dynamics of each element is described by a system of differential equations, for example, the equation for the ion transport through the membrane as in the model of Hodgkin - Huxley, a network of integrative-fire neurons accumulating incoming signals and generating an impulse (spike)  when the threshold is reached; and networks of interacting neuron oscillators including phase oscillators [1].
The latest research showed that memristors harnessed by electrostatic feedback can behave as ideal artificial nanosynapses [2] and be considered as construction blocks for development of principally new computational systems. Hence a breakthrough moment in creation of neuron memristor network is the approach based on oscillator chain where dynamics of each element is characterized only by one variable – oscillation phase, and connection between the oscillators is described by a bifurcation theory.  The main problem of this approach is to describe the regions of parameters space corresponding to various modes of synchronization (full or partial), and image recognition is just matching input parameters with  the synchronization area. Oscillator approach is difficult for computation modeling but as to its physical mechanism it is similar to real neuron networks  and may be implemented experimentally by using elements with non-linear current-voltage characteristic of    switching type. As has been shown in recent works [3-5] the systems with synchronized oscillator modes have unique potential for implementing associated computational schemes and algorithms. 
Memristors development based only on traditional silicon technology (CMOS) narrows the opportunities and potential of such research due to duplication of physical principles of control, to fundamental (for example, quantum-mechanical) limitations of further increase of micro schemes integration degree or to demands for more complicated and expensive technical solutions. In spite of continuous efforts to create new computational systems with CMOS components there are some concerns as for their further scaling and performance enhancement. Alternative approaches are based on new physical mechanisms such as spintronics, superconducting electronics, single electronics. In particular, spin-torque oscillators, STO, connected by spin diffusion current present a special interest [6]. 
In spite of good scaling connected STO experience high shift currents (of mA order) and have low speed limited by angle spin precession. Besides, although connections through spin diffusion currents have low power, but are localized at diffusion length in order of microns, at room temperature. So researchers keep searching for alternative topologies of nanogenerators with promising scaling and more stable galvanic  separations.  
One of new directions, oxide electronics is based on the idea of using unique properties and physical phenomena in heavily correlated oxides of transition metals (OTM). Metal-insulator phase transition (MIT) inherent in a number of transition metals is one of such phenomena
Potential to control MIT in OTM especially in vanadium dioxide are in the spotlight of many researchers which is due to implementation of MIPT in various technical devices including memory elements [7]. Among many well-known methods of MIT parameters control such as application of electric field [8], charge injection through a dielectric [9], laser  emission effect [10] and so on, a special place is taken by electron-beam modification (EBM) [11, 12].Currently EBM control is not studied enough and acquires more significance due to EBM-tuning of MIT parameters in interactive mode and small-scale work areas of devices.  
 MIT has a “genetic” relation, at least in vanadium dioxide, to anther technical effect – electric switching which results from current instabilities in strong electric fields in phase transition conditions and is accompanied by appearance of VI areas with negative differential conductivity (NDC) This effect in VO2 is observed in monocrystalls, in thin film planar structures, in sandwich structures V-VO2-metall, and in various VO2- systems: in oxide,  in anadat-phosphorous glasses, in films of V2O5 gel, in ceramics of VOx-SnO2-Pd  composition. 
Vanadium dioxide-based structures with effect of electric switching used in this project are  perfect objects for coupled oscillators creation not only due to the fact that metal-isnsulator phase transition in VO2 is well studied and switching effect is observed at room temperatures (0-60 oC) where real bio objects inhabit. But also due to natural electric noise sometimes of determined character [13] which is present in real bio objects and also due to speed of operation and nanoscaling potential of VO2-based devices. 
Dealing with the problem of MIT control in vanadium dioxide our recent result [12] is worth noting  regarding EBM effect on switching parameters in thin films structures based on this metal. This will allow immediate control of relaxation oscillations dynamics through shift switch properties change while electron-lithography system will transform any visible image in exposure dose distribution. This will directly manage the dynamics of relaxation oscillations, through changes in the properties of the switching element, the electron lithography system will convert any image into a visible distribution of exposed dose. 
This opens a new direction of research in neuron networks based on revolutionary new oscillators that have not only different physical mechanism compared with STO but are organized according to quite different principle of parametric effect. 
In view of facts presented above it could be promising to research the potential of switching effect usage in VO-based structures to simulate neural network function on the basis of oscillator approach. It should be noted that implement these ideas other TMO could be used, for example, niobium oxide. However because of high transition temperature (1070 K)  niobium dioxide is high power consuming. Vanadium dioxide has only 340 K MIT temperature thus is seems much more attractive material.   
Thus judging by the increasing number of publications the significance of this project’s science problem is undoubtful and one of the candidates to develop new switching devices of neurotechnology are metals and semiconductors oxides, vanadium dioxide in particular. 


1.	Borisyuk G.N., Borisyuk R.M., Kazanovich Ya.B., Ivanitski G.R. Models of neuron activity dynamics during information processing  brain: overall results of a decade. Success of Physical sciences (SPS), volume 172,  № 10, 1189-1216 (2002). 
2.	Pickett M. D., Medeiros-Ribeiro G. and Williams R. S. A scalable neuristor built with Mott memristors. NATURE MATERIALS 12, 114-117 (2013).
3.	 Maffezzoni P., Daniel L., Shukla N., Datta S. and Raychowdhury A. Modeling and Simulation of Vanadium Dioxide Relaxation Oscillators. IEEE Trans. on Circ. and Syst.  62, №9, 2207-2214 (2015).
4.	Datta S., Shukla N., Cotter M., Parihar A., Raychowdhury A. Neuro Inspired Computing with Coupled Relaxation Oscillators. Design Automation Conference (DAC), 2014 51st ACM/EDAC/IEEE, Date 1-5 June 2014, pp.1-6.
5.	Nikonov D. E., Csaba G., Porod W., Shibata T., Voils D., Hammerstrom D., Young I. A. and Bourianoff G. I. Coupled-Oscillator Associative Memory Array Operation.  arXiv:1304.6125 [cond-mat.mes-hall], 32 c.
6.	Kaka S., Pufall M. R., Rippard W. H., Silva T. J., Russek S. E. and Katine J., Mutual phase-locking of microwave spin torque nano-oscillators. Nature 437, № 7057, 389– 92 (2005).
7.	S.-H. Bae , S. Lee , H. Koo , L. Lin , B. H. Jo , C. Park and Z. L. Wang. The Memristive Properties of a Single VO2 Nanowire with Switching Controlled by Self-Heating. Adv. Mater., 25, 36, p. 5098 (2013).
8.	M. A. Belyaev, V. V. Putrolaynen, A. A. Velichko, G. B. Stefanovich and A. L. Pergament, Field-effect modulation of resistance in VO2 thin film at lower temperature. Jpn. J. Appl. Sci., 53, 11, p. 111102 (2014).
9.	G. B. Stefanovich., A. L. Pergament and D. G. Stefanovich. Electrical switching and Mott transition in VO2. J. Phys.:Condens. Matter, 12, p. 8837 (2000).
10.	A. Cavalleri, Th. Dekorsy, H. H. W. Chong, J. C. Kieffer and R. W. Schoenlein. Evidence for a structurally-driven insulator-to-metal transition in VO2: A view from the ultrafast timescale. Phys. Rev. B, 70, p. 161102 (2004).
11.	A. V. Ilinskii, V. Yu. Davydov, R. A. Kastro, O. E. Kvashenkina, M. E. Pashkevich and E. B. Shadrin. ElectronBeam Modification of the Parameters of the Insulator–Metal Phase Transition in Vanadium Dioxide Films. Technical Phys. Lett., 39, 8 (2013).
12.	M. A. Belyaev, A. A. Velichko, S. D. Khanin, G. B. Stefanovich, V. A. Gurtov and A. L. Pergament. Electron-beam modification and electrical property recovery dynamics of vanadium dioxide films in semiconducting and metallic phases. // Japanese Journal of Applied Physics 54, 051102 (2015).
13.	Velichko A.A., Stefanovich G.B., Pergament A.L., Boriskov P.P. Determined noise in vanadium dioxide- based structures. Technical Phys. Lett., 29,10, p. 82-87 (2003)

137 total views, 1 views today