Reference for citation: Korobeynikov A. G. Application of Big Data methods for comparing data of geomagnetic observatories in the INTERMAGNET network. Journal of Instrument Engineering. 2023. Vol. 66, N 12. P. 993—1001 (in Russian). DOI: 10.17586/0021-3454-2023-66-12-993-1001.

Abstract. Big data processing methods are used to solve various problems, for example, collecting, storing, analyzing, visualizing and interpreting large amounts of information received from various sources: the Internet, mobile applications and social networks. The use of special technologies and tools, such as MapReduce, Hadoop, Spark, speeds up the process due to parallel and distributed data processing. A comparison of data from five geomagnetic observatories included in the international INTERMAGNET network is carried out using visualization, which is one of the components of Big Data technology. In each observatory of the INTERMAGNET network, information about the current state of the Earth's magnetic field is collected using specially certified magnetometric equipment. Quite often the analysis of this information obtained over a long period is of scientific and practical interest. In this case, the information is big data, that is, data that does not fit into the RAM of the computer being used. Graphs of initial observation data for the period from January 1, 2018 to July 31, 2023 are presented. The MatLab system with Big Data methods implemented in it, is used as a toolkit.

Keywords: Big Data, INTERMAGNET, MatLab, datastore, Earth's magnetic field, geomagnetic observatory

Acknowledgement: This work was supported by the Russian Science Foundation, grant No. 23-27-00011.

References:

1. Korobeynikov A.G. Journal of Instrument Engineering, 2023, no. 7(66), pp. 533–538, DOI: 10.17586/0021-3454-2023-66-7-533-538. (in Russ.)
2. Makshanov A.V., Zhuravlev A.E., Tyndykar L.N. Bol'shiye dannyye. Big data (Big Data. Big Data), St. Petersburg, 2022, 188 р., ISBN 978-5-8114-9834-5. (in Russ.)
3. Korobeynikov A.G., Grishentsev A.Y., Velichko E.N., Aleksanin S.A., Fedosovskii M.E., Bondarenko I.B., Korikov C.C. Optical Memory & Neural Networks (Information Optics), 2016, no. 3(25), pp. 184–191.
4. Dyakonov V.P. MATLAB i SIMULINK dlya radioinzhenerov (MATLAB and SIMULINK for Radio Engineers), Moscow, 2016, 976 р. (in Russ.)
5. Novgorodtsev A.B. Raschet elektricheskikh tsepey v MATLAB (Calculation of Electrical Circuits in MATLAB), St. Petersburg, 2004, 250 р. (in Russ.)
6. Matyushkin I.V. Modelirovaniye i vizualizatsiya sredstvami MATLAB fiziki nanostruktur (Modeling and Visualization by Means of MATLAB of the Physics of Nanostructures), Moscow, 2011, 168р. (in Russ.)
7. Korobeynikov A.G., Fedosovsky M.E., Zharinov I.O., Shukalov A.V., Gurjanov A.V. International Journal of Applied Engineering Research, 2017, no. 6(12), pp. 1114–1122.
8. Gaiduk A.R., Belyaev V.E., Pyavchenko T.A. Teoriya avtomaticheskogo upravleniya v primerakh i zadachakh s resheniyami v MATLAB (Theory of Automatic Control in Examples and Problems with Solutions in MATLAB), St. Petersburg, 2016, 464 р. (in Russ.)
9. Porshnev S.V. Komp'yuternoye modelirovaniye fizicheskikh protsessov v pakete MATLAB (Computer Simulation of Physical Processes in the MATLAB Package), St. Petersburg, 2011, 736 р. (in Russ.)
10. Frisk V.V., Ganin V.I., Stepanova A.G. Komp'yuternyy analiz i modelirovaniye elektricheskikh tsepey postoyannogo toka v srede MATLAB (Computer Analysis and Modeling of DC Electrical Circuits in the MATLAB Environment), Moscow, 2021, 32 р. (in Russ.)
11. Korobeynikov A.G. Software & Systems, 2022, no. 3(35), pp. 452–457, DOI: 10.15827/0236-235X.139.452-457. (in Russ.)
12. Korobeynikov A.G. International Journal of Humanities and Natural Sciences, 2018, no. 8, pp. 91–98. (in Russ.)