In 2018, we conducted theoretical and experimental research activities on the methods development for oscillatory neural networks’ operations (networks training), for such applications as pattern storing and pattern recognition, information coding. The synchronization effects in coupled oscillators’ arrays were investigated in details, using a special family of synchronization estimation metrics; and the effects of thermal coupling delay in 3D integration of oscillators were demonstrated. The results were published in the ranked journals, including journals with open access, a number of publications are in review process; intellectual property rights’ applications were submitted and intellectual property rights were obtained. VO2 switches (oscillators) are model objects, which manufacturing technology is widely used all over the world. The described effects are of a general nature and can be used to create networks based on a wide variety of physical oscillators – electric, magnetic, and optical, which industrial manufacturing technology have been developed. In addition, the results form a new direction in technology development for the oscillatory neural networks’ operations on high-performance computing platforms: video cards and programmable logic integrated circuits.
- A technology has been developed for creating a coupled oscillators’ array based on switching structures of transition metal oxides. The main elements of the oscillator network are VO2 switches, which constitute two-electrode planar structures with a functional layer of vanadium dioxide and two metal contacts. The connection between the oscillators is carried out by heat flows, propagating through the substrate and resulting from the Joule heating when the switches are turned on. Synthetic corundum (Al2O3) is used as a substrate; however, any other dielectric material (quartz, sitall, etc.) can be used. The substrates underwent a standard cleaning procedure from organic impurities using acetone, methanol and isopropanol. A film of amorphous vanadium oxide, which is close in stoichiometry to VO2, was deposited onto the substrate using a magnetron sputtering of a metal target in an oxygen-argon atmosphere at room temperature. The VO2 film thickness varied from 100 to 200 nm, depending on the requirements for the switch parameters. At the second stage of oscillators’ array fabrication, the square-shaped vanadium dioxide regions were formed using optical lithography methods. After the production of a lithographic mask, an amorphous oxide was etched in a 3N solution of nitric acid for 2 minutes. Then, a photoresist mask was removed using acetone, and the substrate was annealed again at 480 °C for 30-60 minutes (depending on the required switching parameters) at a pressure of 10E-2 mm Hg. and flow rate of 10 cm3/min. The final stage of the coupled oscillators’ array formation was the deposition of gold contacts.
- A method for considering the interaction time delay of the thermally coupled oscillators has been proposed. The heat signal can propagate with a significant delay relative to the magnitude of the oscillation period, and the standard algorithm for determining the synchronization requires clarification. The modelling of the two oscillators operation at 3D integration was performed using the COMSOL computing platform. In the case of the presence of the time delay in the interaction of oscillators, there are relative temporal displacements of the current pulses on the oscillograms of currents. To determine the basic synchronization parameters of high order (SHR and ) in the presence of interaction time shift , it is required to determine the set of time shifts of current peaks 1, 2…i…p, choose from this set the most common value i within the duration of the processed oscillogram Tall, shift the oscillograms relative to each other by this value, and then apply standard calculation procedures. Analysing η() graph, the curve has a number of periodic minima and, in general, the efficiency of η decreases with increasing of . Designing 3D ONS, the distance l between the oscillators can be varied, and because not only the value of τ changes, but also the amount of thermal interaction changes, we observe a rather complex dependence η(l) with present minima. Apparently, the appearance of minima and maxima of the synchronization parameters, when the time delay of oscillator interaction in the network varies, is a general effect.
- The modelling of the sandwich switch was performed. It was demonstrated that the modelled I-V characteristics had a region with negative differential resistance. The resistance of the high-resistance branch was Roff ~ 15 kΩ, and the low-resistance brach was Ron ~ 1 kΩ, the turn-on voltage was Vth ~ 1.55 V, and the switch-off voltage was Vth ~ 0.88 V, which corresponded to the order of magnitudes observed in the experiment with anode films. The temperature distribution at the turning-on moment of switch was calculated. The maximum temperature was detected at the centre of the channel and reached a value of ~ 370 K, corresponding to the phase transition temperature in vanadium dioxide. With a channel diameter of 200 nm and a film thickness of 100 nm, the effective thermal interaction radius was RT ~ 4.5 µm, and the interaction region was hemispherical.
- The influence of a bipolar memory element on oscillatory circuits was investigated. We modified the model circuit of the memristor by adding two diodes to it, which allowed us to simulate different rates of generation and recombination of vacancies, which concentration determined the resistances of the memristor RM. In addition, the circuit had the ability to supply a negative voltage to the VM memristor, which transferred it into a closed state. The circuit of two neuron-oscillators, connected by a memristor RM and a serial capacitance Cc, was investigated. Timing diagrams of the memristor resistance RM(t) and the output voltage Vo (t) demonstrated the bursting activity of the output neuron.
- The oscillations dynamics of an oscillators array and its application to the pattern recognition and logic problems was studied. A new direction was highlighted in this area, and the methods we propose can be successfully used in coding and recognition tasks along with already known techniques developed for neural networks. A special feature of the method is the use of the family of metrics, currently consisting of two parameters – this is the high order synchronization value SHR and the synchronization efficiency value (in the future we plan to increase their number). Another feature is that these parameters are integral over time values, which are not measured after a single impulse arrival, but the state is determined (set) after a significant number of oscillations, so, it is a more complex system where it is necessary to consider the dynamics of all associated oscillators at the same time. Two approaches to solving the recognition problem, “The Integral Approach” and “The Differential Approach”, were highlighted. To implement the former, we define a vector image, memorised or recognized as parameters of individual oscillators in the system, and study the complex synchronization of the entire system, including the input neurons. Hence, the concept of the input and output layers is absent, as they can be combined. In this case, the high order synchronization parameter has a complicated form of the type SHR=k1:k2:k3:..kN, consisting of the numbers of harmonics of several (N) oscillators. We presented this idea in a paper published in the journal “Electronics”. In the latter approach, called “The Differential Approach”, we separate the input and output layers of neurons, and synchronization is measured only between two oscillators — the output oscillator and the master (reference) oscillator, which has a constant frequency. This approach is similar to the methods applied to standard neural networks. The difference is that a single neuron at the output can have multiple states (a multilevel neuron), allowing the number of neurons to be reduced to solve certain tasks. The influence of noise on the network was studied, and the effects of stochastic resonance were detected.
- Using numerical simulation of two- and three-oscillator circuits with thermally coupled VO2 switches, a new method of pattern (vector) storing and recognition in an impulse oscillatory neural network (ONN) based on a high-order synchronization effect was demonstrated. The method allows the storing of many images, which amount is determined by the number of synchronous states Ns. Each state of the system is characterized by a synchronization order, defined as the ratio of the harmonic numbers at the total synchronization frequency. The model achieved Ns values of several orders of magnitude, Ns~650 for three oscillatory circuits and Ns~260 for two oscillatory circuits. A number of regularities were discovered, in particular, the presence of an optimal coupling force of oscillators was found, at which Ns reached a maximum. In addition, a general tendency towards a decrease in the information capacity with an increase in the coupling force and the amplitude of the internal noise of the switches was demonstrated. An algorithm for vectors storing and test vector recognition was proposed, and it was demonstrated that to reduce recognition uncertainty, it is necessary to use the number of coordinates in each vector by one less than the number of oscillators.
- Applying “The Differential Approach”, a new method of information coding using an impulse oscillatory neural network based on the high order synchronization effect was presented. In the proposed neural network scheme, the data is fed to the input layer in the form of supply current levels of the oscillators and converted into a set of non-repeating synchronous states of the output oscillator. Using the example of modelling a thermally coupled VO2-oscillator circuit, the network setup is demonstrated through the selection of coupling forces, power supply levels and the synchronization efficiency parameter.
- A problem of pattern recognition, representing the figures in a 3×3 matrix, was posed and solved. The input was a set of figures in the form of a 3×3 matrix, having 512 different combinations, divided into 102 classes Cm on the basis of symmetry. The output consisted of a single oscillator. The network was trained using the model annealing method, with the values of Un and ηth being recorded, while the currents (ION, IOFF, I0, I10) and the coupling strengths between oscillators (sr, so, sm) varied randomly in certain ranges. The problem was divided into the following subproblems. Problem I: It is required to find a solution when the network recognizes one class (P = 1, where P is the number of recognizable classes) out of 102 possible Cm, with a certain value of m. Problem II: The network recognizes P> 1 classes, with assigning an original SHR synchronization value to each class. Problem III: The network recognizes all classes P = 102. Results indicated that after training, the probability of finding any solution with P = 1 (Problem I) is ~ 10%, and this is the highest probability compared to other P>1. The highest probability (~ 4%) of the solution for the set of C1 and C102 appears when all cells of the input pattern are either empty or painted over. Solutions for other m appear much less frequently, with a probability two orders of magnitude lower (~ 0.03%). However, this histogram shows that it is possible to train the network in solving Problem I, with a certain, given in advance, value m. After all the stages of training, the maximum value of P reached P = 14 that currently constitutes the best result for solving Problem II. Clearly, it is not the limit, and the configuration and the training algorithm can be developed further; however, the purpose of this task was to show the possibility of implementing a multi-level neural network and its application for pattern recognition. Currently, Problem III has no solution that represents the goal for the future research. There may be number of solutions Np of the same problem, and the probability of finding a solution can be expressed in percent; this is another argument in favour of the validity of the solution search algorithm we used by random searching (as a type of method of model annealing). In addition, we studied the noise influence on solutions search, and there is an optimal amount of noise when both NP and P reach a maximum. Therefore, there is the presence of an effect similar to the effect of stochastic resonance, which should be studied in the future. Varying the value of synchronization efficiency, which is a parameter of the processing algorithm rather than a network parameter, also showed the presence of a maximum.
- An experiment was conducted on the additive effect of a harmonic signal and noise on a VO2 oscillator with a planar switch. In the self-oscillatory mode, we demonstrated the effect of stochastic synchronization with the presence of a resonance of signal-to-noise ratio, estimated by the originally developed spectral method.
- It was demonstrated that photo-resistive transformation of oscillator parameters by the elements of their external circuit (photodiodes) is a convenient way of translating visual images into ONN. Since the simplest option for adjusting the oscillation frequency of oscillators is a variation of their supply currents, photodiodes comprising load elements in the EMF circuit effectively solve this problem, unlike the complex and ambiguous electron beam modification of VO2 switches.
3 total views, 1 views today