This dissertation advances the state of the art for scalable high-performance graph analytics and data mining using the language of linear algebra. Many graph computations suffer poor scalability due to their irregular nature and low operational intensity. A small but powerful set of linear algebra primitives that specifically target graph and data mining applications can expose sufficient coarse-grained parallelism to scale to thousands of processors.

In this dissertation we advance existing distributed memory approaches in two important ways. First, we observe that data scientists and domain experts know their analysis and mining problems well, but suffer from little HPC experience. We describe a system that presents the user with a clean API in a high-level language that scales from a laptop to a supercomputer with thousands of cores. We utilize a Domain-Specific Embedded Language with Selective Just-In-Time Specialization to ensure a negligible performance impact over the original distributed memory low-level code. The high-level language enables ease of use, rapid prototyping, and additional features such as on-the-fly filtering, runtime-defined objects, and exposure to a large set of third-party visualization packages.

The second important advance is a new sparse matrix data structure and set of algorithms. We note that shared memory machines are dominant both in stand-alone form and as nodes in distributed memory clusters. This thesis offers the design of a new sparse-matrix data structure and set of parallel algorithms, a reusable implementation in shared memory, and a performance evaluation that shows significant speed and memory usage improvements over competing packages. Our method also offers features such as in-memory compression, a low-cost transpose, and chained primitives that do not materialize the entire intermediate result at any one time. We focus on a scalable, generalized, sparse matrix-matrix multiplication algorithm. This primitive is used extensively in many graph algorithms such as betweenness centrality, graph clustering, graph contraction, and subgraph extraction.

Solving Laplacian linear systems is an important task in a variety of practical and theoretical applications. Laplacians of structured graphs, such as two and three dimensional meshes, have long been important in finite element analysis and image processing. More recently, solving linear systems on the Laplacians of large graphs without mesh-like structure has emerged as an important computational task in network analysis. A number of theoretical solvers with good asymptotic complexity have been proposed over the past couple decades, but these ideas have not made their way into practical solvers. Nor is it clear that a class of challenging problems exist which would benefit from asymptotically fast solvers. Yet it seems that one of the following should be true: either existing solvers have tighter Big-O bounds than currently believed, or there are some problems where recent asymptotically fast (but theoretical) algorithms should be useful.

This work considers the latter possibility; we aim to bridge the gap between theoretical and practical Laplacian algorithms by experimenting with Laplacian solvers and by searching for difficult test problems. We examine the performance of existing algorithms for solving Laplacian linear systems and identify the strengths and weaknesses of different methods on different test problems. We perform an extensive evaluation of the KOSZ solver, one of the recently proposed Õ(m) Laplacian algorithms. We test various extensions of KOSZ which we propose to try and improve its performance in practice. We introduce heavy path graphs, a novel class of graphs for experimenting with Laplacian solvers.

To challenge existing solver implementations, we propose the use of genetic algorithms to create difficult test graphs for existing solvers. At the same time, these algorithms could be used to find graphs with good performance for recently proposed solvers. Searching for graphs which satisfy both objectives could be instrumental towards bridging the theory-practice gap of Laplacian solvers. We demonstrate the successful evolution of graphs which are difficult for conjugate gradient with diagonal scaling, while relatively simple for KOSZ. Such graph evolution techniques could be useful for finding graphs with a variety of combinatorial properties.

Scalable algorithm design has become central in the era of large-scale data analysis. The vast amounts of data pouring in from a diverse set of application domains, such as bioinformatics, recommender systems, sensor systems, and social networks, cannot be analyzed efficiently using many data mining and statistical tools that were designed for a small scale setting. It is an ongoing challenge to the data mining, machine learning, and statistics communities to design new methods for efficient data analysis. Confounding this challenge is the noisy and incomplete nature of real-world data sets. Research scientists as well as practitioners in industry need to find meaningful patterns in data with missing value rates often as high as 99%, in addition to errors in the data that can obstruct accurate analyses.

My contribution to this line of research is the design of new algorithms for scalable clustering, data reduction, and similarity evaluation by exploiting inherent clustering structure in the input data to overcome the challenges of significant amounts of missing entries. I demonstrate that, by focusing on underlying clustering properties of the data, we can improve the efficiency of several data analysis methods on sparse, discrete-valued data sets. I highlight new methods that I have developed with my collaborators for three diverse knowledge discovery tasks: (1) clustering genetic markers into linkage groups, (2) reducing large-scale genetic data to a much smaller, more accurate representative data set, and (3) computing similarity between users in recommender systems. In each case, I point out how the underlying clustering structure can be used to design more efficient algorithms, even when high missing value rates are present.