Research Projects
Brain-Inspired Hyper-Dimensional Computing:
To achieve real-time performance with high energy-efficiency, we rethink not only how we accelerate machine learning algorithms in hardware, but also we redesign the algorithms themselves using strategies that more closely model the ultimate efficient learning machine: the human brain. My Ph.D. research developed brain-inspired HyperDimensional (HD) computing as an interdisciplinary research area emerged from theoretical neuroscience. HD computing is motivated by the understanding that the human brain operates on high-dimensional representations of data originated from the large size of brain circuits. It thereby models the human memory using points of a high-dimensional space. HD computing mimics several desirable properties of the human brain, including: robustness to noise and hardware failure and single-pass learning where training happens in one-shot without storing the training data points or using complex gradient-based algorithms. These features make HD computing a promising solution for: (1) today's embedded devices with limited storage, battery, and resources, as well as (2) future computing systems in deep nanoscaled technology, which will have high noise and variability. I exploited the mathematics and the key principles of brain functionalities to create cognitive platforms.
Our platform includes: (1) novel HD algorithms supporting classification, clustering, regression, and reinforcement learning which represent the most popular categories of algorithms used regularly by professional data scientist [D&T'17, DATE'19, DAC'19], (2) novel HD hardware accelerators capable of up to three orders of magnitude improvement in energy efficiency relative to GPU implementations [HPCA'17, FCCM'19, FPGA'19], and (3) a software infrastructure that makes it easy for users to integrate HD computing as a part of any system and enable secure distributed learning on encrypted information [CLOUD'19]. Our research opened a new direction in the brain-inspired learning method that involves many different schools, government agencies, and companies. In addition, DARPA recently opened a new program influenced by my Ph.D. research (Link).
To achieve real-time performance with high energy-efficiency, we rethink not only how we accelerate machine learning algorithms in hardware, but also we redesign the algorithms themselves using strategies that more closely model the ultimate efficient learning machine: the human brain. My Ph.D. research developed brain-inspired HyperDimensional (HD) computing as an interdisciplinary research area emerged from theoretical neuroscience. HD computing is motivated by the understanding that the human brain operates on high-dimensional representations of data originated from the large size of brain circuits. It thereby models the human memory using points of a high-dimensional space. HD computing mimics several desirable properties of the human brain, including: robustness to noise and hardware failure and single-pass learning where training happens in one-shot without storing the training data points or using complex gradient-based algorithms. These features make HD computing a promising solution for: (1) today's embedded devices with limited storage, battery, and resources, as well as (2) future computing systems in deep nanoscaled technology, which will have high noise and variability. I exploited the mathematics and the key principles of brain functionalities to create cognitive platforms.
Our platform includes: (1) novel HD algorithms supporting classification, clustering, regression, and reinforcement learning which represent the most popular categories of algorithms used regularly by professional data scientist [D&T'17, DATE'19, DAC'19], (2) novel HD hardware accelerators capable of up to three orders of magnitude improvement in energy efficiency relative to GPU implementations [HPCA'17, FCCM'19, FPGA'19], and (3) a software infrastructure that makes it easy for users to integrate HD computing as a part of any system and enable secure distributed learning on encrypted information [CLOUD'19]. Our research opened a new direction in the brain-inspired learning method that involves many different schools, government agencies, and companies. In addition, DARPA recently opened a new program influenced by my Ph.D. research (Link).
Deep Learning and Big Data Processing Acceleration:
Running data/memory-intensive workloads on traditional cores results in high energy consumption and slow processing speeds, primarily due to a large amount of data movement between memory and processing units. I have designed a digital-based Processing in-memory (PIM) platform capable of accelerating fundamental big data applications in real-time with orders of magnitude higher energy efficiency [ISCA'19, HPCA'2020, DAC'17]. My design accelerates entire applications directly in storage-class memory without using extra processing cores. My platform opened a new direction towards making the PIM technology practical. In contrast to prior method that enable PIM functionality in analog domain, we design the first digital-based PIM architecture that (i) works on digital data; thus, it eliminates ADC/DAC blocks that dominate the area. (ii) it addresses internal data movement issue by enabling in-place computation where the big data is stored, (iii) it natively supports floating-point precision that is essential for many scientific applications, (iv) it is compatible with any bipolar memory technology, including Intel 3D XPoint. My proposed platform can also accelerate a wide range of big data applications including machine learning [ISCA'19, HPCA'20, TC'19], query processing [TCAD'18], graph processing [ISLPED'18], and bioinformatics [ISLPED'19]. One particularly successful application of my design is FloatPIM architecture [ISCA'19], which significantly accelerates state-of-the-art Convolutional Neural Networks (CNNs).
Running data/memory-intensive workloads on traditional cores results in high energy consumption and slow processing speeds, primarily due to a large amount of data movement between memory and processing units. I have designed a digital-based Processing in-memory (PIM) platform capable of accelerating fundamental big data applications in real-time with orders of magnitude higher energy efficiency [ISCA'19, HPCA'2020, DAC'17]. My design accelerates entire applications directly in storage-class memory without using extra processing cores. My platform opened a new direction towards making the PIM technology practical. In contrast to prior method that enable PIM functionality in analog domain, we design the first digital-based PIM architecture that (i) works on digital data; thus, it eliminates ADC/DAC blocks that dominate the area. (ii) it addresses internal data movement issue by enabling in-place computation where the big data is stored, (iii) it natively supports floating-point precision that is essential for many scientific applications, (iv) it is compatible with any bipolar memory technology, including Intel 3D XPoint. My proposed platform can also accelerate a wide range of big data applications including machine learning [ISCA'19, HPCA'20, TC'19], query processing [TCAD'18], graph processing [ISLPED'18], and bioinformatics [ISLPED'19]. One particularly successful application of my design is FloatPIM architecture [ISCA'19], which significantly accelerates state-of-the-art Convolutional Neural Networks (CNNs).
Online Memorization for Approximation:
Today’s computing systems are designed to deliver only exact solutions. However, many applications, i.e., machine learning applications, are statistical in nature and do not require exact answers. While prior research has explored approximate computing, most solutions to date are (1) isolated to only a few of the components in the system stack, and (2) do not learn to enable an intelligent approximation. The real challenge arises when developers want to employ approximation across multiple layers of the computing stack simultaneously. I proposed a novel architecture with software and hardware support for the acceleration of learning and multimedia applications on today’s computing systems [DATE'16]. Hardware components are enhanced with the ability to adapt self-learning approximation at a quantifiable and controllable cost in terms of accuracy. Software services complement hardware to ensure the user’s perception is not compromised while maximizing the energy savings due to approximations. I have enhanced the computing units (CPU, GPU, and DSPs) with a small associative memory placed close to each streaming core [ISLPED'16, TETC'16]. This associative memory with the capability of self-learning is placed beside each computing unit to remember frequent patterns and reduce redundant computations [ISLPED'16]. The main idea of the approximation is to return pre-computed results from the associative memory, not only for perfect matches of operands but also for inexact matches [ISLPED'16].
Today’s computing systems are designed to deliver only exact solutions. However, many applications, i.e., machine learning applications, are statistical in nature and do not require exact answers. While prior research has explored approximate computing, most solutions to date are (1) isolated to only a few of the components in the system stack, and (2) do not learn to enable an intelligent approximation. The real challenge arises when developers want to employ approximation across multiple layers of the computing stack simultaneously. I proposed a novel architecture with software and hardware support for the acceleration of learning and multimedia applications on today’s computing systems [DATE'16]. Hardware components are enhanced with the ability to adapt self-learning approximation at a quantifiable and controllable cost in terms of accuracy. Software services complement hardware to ensure the user’s perception is not compromised while maximizing the energy savings due to approximations. I have enhanced the computing units (CPU, GPU, and DSPs) with a small associative memory placed close to each streaming core [ISLPED'16, TETC'16]. This associative memory with the capability of self-learning is placed beside each computing unit to remember frequent patterns and reduce redundant computations [ISLPED'16]. The main idea of the approximation is to return pre-computed results from the associative memory, not only for perfect matches of operands but also for inexact matches [ISLPED'16].
Mentorship
|
During my PhD, I have closely mentored more than 22 undergraduate and 20 graduate students, including 8 PhD students. I was honored to mentor about 14 female students and have my small contribution to increase diversity in engineering and computing fields. I have published more than 27 papers in the top venue conferences/journals with undergraduates where more than 18 are with minorities. I have also volunteered to participated in ERSP (2018, 2019) and ENLACE (2017, 2018) programs which aim to support undergraduate and the research of Latino students at UC San Diego. More information about my Mentorship and contribution to Diversity.
|
|
What's New?
New! May. 2020. Will joining as an Assistant Professor to the Department of Computer Science at UC Irvine. New! May. 2020. Received my PhD from UC San Diego! :) New! Nov. 2019. Our CryptoPIM: best paper candidate at DAC 2020! New! Nov. 2019. Our GenieHD: best paper candidate at DATE 2020! New! Nov. 2019. paper accepted in HPCA 2020 (acceptance rate 19.3%)! New! Nov. 2019. two papers accepted in DAC 2020 (collaboration with Intel and Prof. Nikil Dutt @ UC Irvine)! Nov. 2019. two papers accepted in DATE 2020! Sept. 2019. Best Presentation Award @ SRC TECHCON 2019 (Photo)! Sept. 2019. Accepted paper in ICCD 2019! Sept. 2019. Accepted paper in BioCAS! collaboration with, prof. Giovanni De Micheli July. 2019. Session Chair in ISLPED'19! June. 2019. Two accepted papers in ICCAD'19! June. 2019. Best Poster Award, DAC PhD Forum, 2019! June. 2019. Best paper candidate at DAC 2019! May. 2019. Honored to receive Gordon Leadership award (Video, photos)! May. 2019. the Bernard and Sophia Gordon Engineering Leadership award, the most prestigious leadership award in the school of engineering at UC San Diego (check here)! May. 2019. Two accepted papers in ISLPED'19! Mar. 2019. Accepted papers in ISCA'19 (acceptance rate 16.9%)! Feb. 2019. Accepted papers in FCCM'19 and BHI'19 conferences! Feb. 2019. Outstanding Graduate Student Award in the school of engineering! Feb. 2019. Four papers accepted in DAC'19! Jan. 2019. Accepted paper in CLOUD (acceptance rate 14.3%)! Dec. 2018. Accepted papers in IEEE TC, ACM JETC, and IEEE D&T journals!! Dec. 2018. Accepted for DATE 2019 PhD Forum! Nov. 2018. Accepted paper in FPGA'19! Oct. 2018. Three accepted papers in DATE'19! Sept. 2018. Three accepted papers in ASPDAC'19! June 2018. Accepted papers in ICCAD'18! June 2018. Doctoral Award for Excellence in Research (link)! April 2018. Best poster award ResearchExpo'18! Presenter Daniel Peroni. April 2018. Invitation talk at UC Berkeley, Thanks Prof. Rabeay for hosting. March 2018. Best paper candidate ISQED'18! Thanks Joonseop for the great presentation! Jan 2018. Program committee member in 14th IEEE PRIM'18 and 31st IEEE SOCC'18 Nov-Oct 2017. Presentation at CNS'17 (La Jolla), ICRC (Washington Dc), ICCAD (Irvine) Sep 2017. IEEE invitation talk at San Diego State University, CA. Sep 2017. Teaching two undergraduate courses at UC San Diego, check here! June 2017. My mentee Max Masich defensed his master thesis! June 2017. My mentee Deqian Kong selected as best undergrad researcher at CSE [Picture]! May 2017. Member of program committee in IEEE SOC'17 and IEEE PRIM'17! Aug 2016. ICCD'16 paper selected as top ranked paper for publishing in IEEE TETC! June 2016. Being nominated for the Outstanding Graduate Student Award at UCSD! May 2016. Defensed my proposal!! (PhD Candidate) May 2015. Research exam passed! |
Collaborators:
|
