|
Abstract.
We consider an entropy reduction method based on randomized (0,1)-projector matrices. The concept of a data matrix compactness indicator is introduced. We formulate an entropy reduction algorithm as a problem of conditional maximization of the entropy functional defined on the probability density functions of the projector matrices. Conditions for the existence and uniqueness of a positive solution are obtained.
Keywords:
random projection, data matrix compression and expansion, (0,1)-projector matrices, compactness indicator.
DOI 10.14357/20718632260108
EDN EHHOWR
PP. 92-99.
References
1. Carreira-Perpinan M.A. A review of dimension reduction techniques. Technical report CS-96-09, Department of Computer Science, University of Sheffield, 1997. 2. Imola K. A survey of dimension reduction techniques. Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, 2002. 3. Cunningham P. Dimension Reduction. Technical Report UCD-CSI-2007-7, University College Dublin, 2007. 4. Ajvazyan S.A., Buhshtaber V.M., Enyukov I.S., Meshalkin L.D. Prikladnaya statistika. Klassifikaciya i snizhenie razmernosti. M., Finansy i statistika, 1989. 5. Friedman J., Hastie T., Tibshirani R. Element of Statistical Learning: Prediction, Inference and Data Mining. Springer, 2001. 6. Bishop C. Pattern Recognition and Machine Learning, ser. Information Science and Statistics, Springer, 2006. 7. Comon P., Jutten C. Handbook of Blind Source Separation. Independent Component Analysis and Applications. Academic Press, Oxford UK, 2010. 8. Bruckstein A.M., Donoho D.L., Elad M. From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images. SIAM Rev. 2009, v.51, No.1, p.34-81. 9. Pirson K. On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 1901, v.2, p. 559-572. 10. Kendal M.Dzh., Styuart A. Statisticheskie vyvody i svyazi./ Per. s angl. M.. Nauka, 1973. 11. Jolliffe I.T. Principal Component Analysis. N.Y. Springer Verlag, 2002. 12. Polyak B.T., Hlebnikov M.V. Metod glavnyh komponent: robastnye versii. Avtomatika i Telemekhanika, 2017, №3, s.130-148. 13. Deerwester S.C., Dumias S.T., Landaurer T.K., Furnas G.W., Harshman R.A. Indexing by latent semantic analysis. Journal of American Society of Information Sciences, 1990, v.41(6), p.391-407. 14. Fisher R.A. The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics. 1936, v.7, p.179-188. 15. McLachlan G.J. Discriminant Analysis and Statistical Pattern Recognition. Wiley Interscience, 2004. 16. Johnson W.B., Lindenstrauss J. Extension Of Lipshitz mapping into Hilbert space. Conference in modern analysis and probability. American Mathematical Society, 1984, v.26, p.189-206. 17. Achlioptas D. Database-friendly random projections. Proceedings of the twentieth ACM Symposium on Principles of database systems, p.274-281. 18. Bingham E., Mannila H. Random projection in dimensionality reduction: applications to image and text data. Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p. 245–250. 19. Vempala S.S. The random projection method. AMS, 2005, v.65. 20. Ganin I.P., Kosichenko E.A., Kaplan A.YA. Osobennosti EEG-reakcij na emocional'no znachimye stimuly v teznologii interfejsa mozg-kompyuter na volne R300. Zhurnal vysshej nervnoj deyatel'nosti im. I.P.Pirogova, 2017, t.67, №4, s.453-463. 21. Huber F., Zorner T.O. Threshold cointegrationin international exchange rates: A Bayesian approach. International Journal of Forecasting, 2019, v.35, p.458-473. 22. Kosinskia M., Stillwella D., Groepelb T. Private traits and attributtes are predictable from digital records of human behaviour. Proceedings of the National Agademy of Sciences of the USA, 2013, v.110(15), p 5802-5805. 23. Blum A.,Langly P. Selection of relevant feature and examples in mashine learning. Artificial Intelligence, 1997, v.97(1-2), p.245-271. 24. Cover T.M., Thomas J.A. Elements of Information Theory. NY, John Wiley and Sons, 1991. 25. Peng H.C., Long F., Ding C. Feature selection based on mutual information: criteria of max-dependency, maxrelevance, and min-redudancy. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2005, v.27(8), p. 1226-1238. 26. Zhang Y., Li S., Wang T., Zhang Z. Divergence-based feature selection for separate classes. Neurocomputing, 2013, v.101, p.32-42. 27. Darhovskij B.S., Kaplan A.YA., SHishkin S.L. O podhode k ocenke slozhnosti krivyh (na primere elektroencefalogrammy cheloveka). Avtomatika i Telemekhanika, 2002, № 3, s. 134-140. 28. Darhovskij B.S., Piryatinskaya A. Novyj podhod k probleme segmentacii vremennyh ryadov proizvol'noj struktury. Trudy MIAN im. V.A.Steklova, 2014, t.287, s.61-74. 29. Darhovsky B., Piryatinska A. Quickest detection of changes in the generating mechanism of a time series via the ε-complexity of continuous functions. Sequantial Analysis, 2014, v.33, p. 231-250. 30. Efron B. Bootstrap Methods: Another Look at the Jackknife. Annals of Statistics, 1979, v.7, No.1, p.1-26. 31. Bach F.R. Bolasso: model consistent Lasso estimation through the bootstrap. Proceedings of the 25-th International Conference on Machine Learning. 2008, p.33-40. 32. Yu. S. Popkov. Asymptotic efficiency of maximum entropy estimates // Dokl. Math., 102:1 (2020), 350–352. 33. Popkov Y.S., Popkov A.Y., Dubnov Y.A. Entropy Randomization in Machine Learning. CRC Press, 2023, p.389. 34. Ioffe A.D., Tihomirov V.M. Teoriya ekstremalnyh zadach. M., Nauka, 1974. 35. Darkhovsky, B.S., Popkov, Y.S., Popkov, A.Y., Aliev, A.S. A Method of Generating Random Vectors with a Given Probability Density Function, 2018, Automation and Remote Control 79(9), pp. 1569-1581. 36. Magnus YA.R., Nejdekker H. Matrichnoe differencial'noe ischislenie s prilozheniemi k statistike i ekonometrike. M., Fihmatlit, 2002, 495s.
|
