Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization

Nonnegative matrix factorization (NMF) is a powerful technique for dimension reduction, extracting latent factors and learning part-based representation. For large datasets, NMF performance depends on some major issues such as fast algorithms, fully parallel distributed feasibility and limited internal memory. This research designs a fast fully parallel and distributed algorithm using limited internal memory to reach high NMF performance for large datasets. Specially, we propose a flexible accelerated algorithm for NMF with all its (Formula presented.)(Formula presented.) regularized variants based on full decomposition, which is a combination of exact line search, greedy coordinate descent, and accelerated search. The proposed algorithm takes advantages of these algorithms to converges linearly at an over-bounded rate (Formula presented.) in optimizing each factor matrix when fixing the other factor one in the sub-space of passive variables, where r is the number of latent components, and (Formula presented.) and L are bounded as (Formula presented.). In addition, the algorithm can exploit the data sparseness to run on large datasets with limited internal memory of machines, which is is advanced compared to fast block coordinate descent methods and accelerated methods. Our experimental results are highly competitive with seven state-of-the-art methods about three significant aspects of convergence, optimality and average of the iteration numbers.

Title:	Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization
Authors:	Nguyen, D.K. Ho, T.B.
Keywords:	Accelerated anti-lopsided algorithm Cooridinate descent algorithm Non-negative matrix factorization Parallel and distributed algorithm
Issue Date:	2016
Publisher:	Springer New York LLC
Abstract:	Nonnegative matrix factorization (NMF) is a powerful technique for dimension reduction, extracting latent factors and learning part-based representation. For large datasets, NMF performance depends on some major issues such as fast algorithms, fully parallel distributed feasibility and limited internal memory. This research designs a fast fully parallel and distributed algorithm using limited internal memory to reach high NMF performance for large datasets. Specially, we propose a flexible accelerated algorithm for NMF with all its (Formula presented.)(Formula presented.) regularized variants based on full decomposition, which is a combination of exact line search, greedy coordinate descent, and accelerated search. The proposed algorithm takes advantages of these algorithms to converges linearly at an over-bounded rate (Formula presented.) in optimizing each factor matrix when fixing the other factor one in the sub-space of passive variables, where r is the number of latent components, and (Formula presented.) and L are bounded as (Formula presented.). In addition, the algorithm can exploit the data sparseness to run on large datasets with limited internal memory of machines, which is is advanced compared to fast block coordinate descent methods and accelerated methods. Our experimental results are highly competitive with seven state-of-the-art methods about three significant aspects of convergence, optimality and average of the iteration numbers.
Description:	Journal of Global Optimization 4 October 2016, Pages 1-22
URI:	http://link.springer.com/article/10.1007%2Fs10898-016-0471-z http://repository.vnu.edu.vn/handle/VNU_123/32325
ISSN:	09255001
Appears in Collections:	Bài báo của ĐHQGHN trong Scopus