EIC LabSKKUSKKU

Members only

Please connect to SKKU VPN to access.

Open GPU Monitor

Publications

Publications

* Equal contribution   ⁺ Corresponding author

Preprints

  • , , . arXiv preprint arXiv:2605.09982, 2026.arXiv

    PaperCode

    arXiv preprint arXiv:2605.09982, 2026 · arXiv

    ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruning

    , ,

    Abstract

    Recent advancements in Vision-Language Models (VLMs) enable large language models (LLMs) to process high-resolution images, significantly improving real-world multimodal understanding. However, this capability introduces a large number of vision tokens, resulting in substantial computational overhead. To mitigate this issue, various vision token pruning methods have been proposed. Nevertheless, existing approaches predominantly rely on learned semantic features within the model to capture visual redundancy. Moreover, they lack adaptive mechanisms to adjust pruning strategies according to the complexity of the input image. In this paper, we propose ERASE, a two-stage vision token pruning framework that identifies and retains salient tokens through pruning strategies adaptive to image complexity. Experiment results demonstrate that ERASE significantly reduces vision tokens while preserving accuracy. For Qwen2.5-VL-7B, at a token pruning ratio of 85%, ERASE retains 89.46% of the original model accuracy, whereas the best prior method retains only 78.1%.

2026

Conference

  • Jungmin Lee, Gwangeun Byeon, , Seokin Hong⁺. Conference on Neural Information Processing Systems (NeurIPS), 2026.NeurIPS

    Paper

    Conference on Neural Information Processing Systems (NeurIPS), 2026 · NeurIPS

    Token Filtering: Online Attention Pruning via KV Similarity for Efficient LLM Inference

    Jungmin Lee, Gwangeun Byeon, , Seokin Hong⁺

    Abstract

    Pruning has emerged as a promising direction for accelerating large language model (LLM) inference. However, many existing methods rely on offline calibration data, making them sensitive to distribution shifts between calibration data and inference inputs. In this paper, we introduce Token Filtering, a lightweight online pruning method that selectively skips attention computation for redundant tokens during inference, without requiring any calibration data or fine-tuning. Token Filtering identifies redundant tokens based on joint key–value (KV) similarity and bypasses their attention computation. This approach reduces both compute and KV cache size while preserving essential contextual information. To preserve accuracy while meeting the target pruning ratio, we restrict pruning to later layers, which are typically less sensitive to pruning, with a layer-wise threshold that adaptively tracks the target pruning ratio. Extensive experiments on LLaMA3.1-8B, LLaMA3.1-8B-Instruct, and Qwen3-8B demonstrate that Token Filtering consistently achieves better accuracy–efficiency trade-offs than prior methods. In long-context generation, Token Filtering achieves up to 1.5X higher accuracy than the best-performing pruning baseline while reducing latency by up to 44% compared to the dense baseline.

  • , , , . Conference on Neural Information Processing Systems (NeurIPS), 2026.NeurIPS

    PaperCode

    Conference on Neural Information Processing Systems (NeurIPS), 2026 · NeurIPS

    AnalogToBi: Device-Level Analog Circuit Topology Generation via Bipartite Graph and Grammar Guided Decoding

    , , ,

    Abstract

    Analog circuit design remains highly dependent on expert knowledge due to the complexity of device-level interactions and topology design. Recent transformer-based approaches for device-level topology generation have shown promise, yet they suffer from low electrical validity without human-in-the-loop (HITL) training and severe memorization caused by sequence-based circuit representations. In this work, we propose AnalogToBi, a framework for device-level analog circuit topology generation. AnalogToBi introduces circuit-type conditioning for categorizing heterogeneous multi-type topology datasets, device renaming augmentation to mitigate memorization, a bipartite graph representation for improved structural generalization, and grammar-guided decoding to enforce structural validity during bipartite graph generation. Experimental results demonstrate that AnalogToBi achieves high validity and novelty without HITL training while effectively avoiding memorization of training topologies. Our code is available at https://github.com/Seungmin0825/AnalogToBi.

  • , , . International Conference on Machine Learning (ICML), 2026.ICML

    Paper

    International Conference on Machine Learning (ICML), 2026 · ICML

    QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs

    , ,

    Abstract

    Large language models (LLMs) demand substantial computational and memory resources, posing challenges for efficient deployment. Two complementary approaches have emerged to address these issues: token-adaptive layer execution, which reduces floating-point operations (FLOPs) by selectively bypassing layers, and quantization, which lowers memory footprint by reducing weight precision. However, naively integrating these techniques leads to additional accuracy degradation due to reduced redundancy in token-adaptive models. We propose QTALE (Quantization-Robust Token-Adaptive Layer Execution for LLMs), a novel framework that enables seamless integration of token-adaptive execution with quantization while preserving accuracy. Conventional token-adaptive methods reduce redundancy in two ways: (1) by limiting the diversity of training paths explored during fine-tuning, and (2) by lowering the number of parameters actively involved in inference. To overcome these limitations, QTALE introduces two key components: (1) a training strategy that ensures diverse execution paths are actively explored during fine-tuning, and (2) a post-training mechanism that allows flexible adjustment of the execution ratio at inference to reintroduce redundancy when needed. Experimental results show that QTALE enables seamless integration of token-adaptive layer execution with quantization, while keeping the accuracy gap to quantization-only models below 0.5% on CommonsenseQA benchmarks. By combining token-adaptive execution for FLOPs reduction and quantization for memory savings, QTALE provides an effective solution for efficient LLM deployment.

  • Dongwon Jo*, Jiwon Song*, , Jae-Joon Kim⁺. “FastKV: KV Cache Compression for Fast Long-Context Processing with Token-Selective Propagation.” Findings of the Association for Computational Linguistics (ACL Findings), 2026.ACL Findings

    Paper

  • Hyesung Jeon*, Seojune Lee*, Beomseok Kang, , Jae-Joon Kim⁺. “QWHA: Quantization-Aware Walsh-Hadamard Adaptation for Parameter-Efficient Fine-Tuning on Large Language Models.” International Conference on Learning Representations (ICLR), 2026.ICLR

    Paper

  • Munhyeon Kim, Sukhyun Choi, , Jae-Joon Kim⁺. “3D Integration of Hybrid IGZO/Si and IGZO eDRAMs for High-Density/High-Performance On-Chip Memory.” Design, Automation and Test in Europe (DATE), 2026.DATE

    Paper

Journal

  • Jiyoon Kim, Kang Eun Jeon, , Jong Hwan Ko⁺. “RUnQuant: High-Resolution Weight Quantization via Unanchored Weight Decomposition in Column-wise Granularity for CIM Accelerators.” Journal of Systems Architecture (JSA), 2026.JSA

    Paper

2025

Conference

  • Jiwon Song, Dongwon Jo, , Jae-Joon Kim⁺. “Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning.” Conference on Neural Information Processing Systems (NeurIPS), 2025.NeurIPS

    Paper

  • Hyesung Jeon, , Jae-Joon Kim⁺. “L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models.” Annual Meeting of the Association for Computational Linguistics (ACL), 2025.ACL

    Paper

  • Jiyoon Kim, Kang Eun Jeon, , Jong Hwan Ko⁺. “Column-wise Quantization of Weights and Partial Sums for Accurate and Efficient Compute-In-Memory Accelerators.” Design, Automation and Test in Europe Conference (DATE), 2025.DATE

    Paper

  • Munhyeon Kim, , Jae-Joon Kim⁺. “Compute-in-Memory Array Design Using Stacked Hybrid IGZO/Si eDRAM Cells.” Design, Automation and Test in Europe Conference (DATE) — Nominated for the Best Paper Award, 2025.DATE

    Paper

2024

Conference

  • Dongwon Jo, Taesu Kim, , Jae-Joon Kim⁺. “Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Models.” Conference on Neural Information Processing Systems (NeurIPS), 2024.NeurIPS

    Paper

  • Jiwon Song*, Kyungseok Oh*, Taesu Kim, Hyungjun Kim, , Jae-Joon Kim⁺. “SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks.” International Conference on Machine Learning (ICML), 2024.ICML

    Paper

  • Jaeyong Jang*, , Juheun Lee, Jae-Joon Kim⁺. “FIGNA: Integer Unit-based Accelerator Design for FP-INT GEMM Preserving Numerical Accuracy.” IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2024.HPCA

    Paper

2023

Conference

  • , Dongwon Jo, Hyesung Jeon, Taesu Kim, Daehyun Ahn, Hyungjun Kim, Jae-Joon Kim⁺. “Leveraging Early-Stage Robustness in Diffusion Models for Efficient and High-Quality Image Synthesis.” Conference on Neural Information Processing Systems (NeurIPS), 2023.NeurIPS

    Paper

  • , Jaeyong Jang, Jehun Lee, Jihoon Park, Jeonghoon Kim, Byeongwook Kim, Baeseong Park, Se Jung Kwon, Dongsoo Lee, Jae-Joon Kim⁺. “Winning Both the Accuracy of Floating Point Activation and the Simplicity of Integer Arithmetic.” International Conference on Learning Representations (ICLR), 2023.ICLR

    Paper

Workshop

  • Jiwoong Choi, Minkyu Kim, Daehyun Ahn, Taesu Kim, , Dongwon Jo, Hyesung Jeon, Jae-Joon Kim, Hyungjun Kim⁺. “Squeezing Large-Scaling Diffusion Models for Mobile.” ICML Workshop on Challenges of Deploying Generative AI, 2023.ICML Workshop

    Paper

2018~2022

Conference

  • Hyunmyung Oh, Hyungjun Kim, Daehyun Ahn, Jihoon Park, , Inhwan Lee, Jae-Joon Kim⁺. “Energy-Efficient Charge-Sharing-Based 8T2C SRAM In-Memory Accelerator for Binary Neural Networks in 28 nm CMOS.” IEEE Asian Solid-State Circuits Conference (A-SSCC), 2021.A-SSCC

    Paper

  • Hyunmyung Oh, Hyungjun Kim, Nameun Kang, , Jihoon Park, Jae-Joon Kim⁺. “Single RRAM Cell-Based In-Memory Accelerator Architecture for Binary Neural Networks.” IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2021.AICAS

    Paper

  • , Hyungjun Kim, Jihoon Park, Hyunmyung Oh, Jae-Joon Kim⁺. “Mapping Binary ResNets on Computing-In-Memory Hardware with Low-Bit ADCs.” Design, Automation and Test in Europe Conference and Exhibition (DATE), 2021.DATE

    Paper

  • Naebeom Park, , Daehyun Ahn, Taesu Kim, Jae-Joon Kim⁺. “Time-Step Interleaved Weight Reuse for LSTM Neural Network Computing.” IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), 2020.ISLPED

    Paper

  • Hyungjun Kim, , Sungju Ryu, Jae-Joon Kim⁺. “Algorithm-Hardware Co-Design for In-Memory Neural Network Computing with Minimal Peripheral Circuit Overhead.” ACM/IEEE Design Automation Conference (DAC), 2020.DAC

    Paper

  • Sungju Ryu, Hyungjun Kim, Wooseok Yi, Jongeun Koo, Eunhwan Kim, , Taesu Kim, Jae-Joon Kim⁺. “A 44.1 TOPS/W Precision-Scalable Hardware Accelerator for Quantized Neural Networks in 28 nm CMOS.” IEEE Custom Integrated Circuits Conference (CICC), 2020.CICC

    Paper

  • Jinseok Kim, Jongeun Koo, Taesu Kim, , Hyungjun Kim, Seunghyun Yoo, Jae-Joon Kim⁺. “Area-Efficient and Variation-Tolerant In-Memory BNN Computing Using 6T SRAM Array.” Symposium on VLSI Circuits, 2019.VLSI Circuits

    Paper

  • Hyungjun Kim, , Sungju Ryu, Jae-Joon Kim⁺. “BitSplit-Net: Multi-Bit Deep Neural Network with Bitwise Activation Function.” arXiv preprint arXiv:1903.09807, 2019.arXiv

    Paper

  • Wooseok Yi, , Jae-Joon Kim⁺. “Effect of Device Variation on Mapping Binary Neural Network to Memristor Crossbar Array.” Design, Automation and Test in Europe (DATE), 2019.DATE

    Paper

  • Hyungjun Kim, , Jae-Joon Kim⁺. “In-Memory Batch-Normalization for Resistive Memory-Based Binary Neural Network Hardware.” Asia and South Pacific Design Automation Conference (ASP-DAC), 2019.ASP-DAC

    Paper

  • , Hyungjun Kim, Jae-Joon Kim⁺. “Neural Network-Hardware Co-Design for Scalable RRAM-Based BNN Accelerators.” arXiv preprint arXiv:1811.02187, 2018.arXiv

    Paper

  • Jinseok Kim, , Sungho Kim, Jae-Joon Kim⁺. “Compact Convolution Mapping on Neuromorphic Hardware Using Axonal Delay.” International Symposium on Low Power Electronics and Design (ISLPED), 2018.ISLPED

    Paper

  • , Hyungjun Kim, Daehyun Ahn, Jae-Joon Kim⁺. “Input-Splitting of Large Neural Networks for Power-Efficient Accelerator with Resistive Crossbar Memory Array.” International Symposium on Low Power Electronics and Design (ISLPED), 2018.ISLPED

    Paper

Journal

  • , Hyungjun Kim, Jae-Joon Kim⁺. “Extreme Partial-Sum Quantization for Analog Computing-In-Memory Neural Network Accelerators.” ACM Journal on Emerging Technologies in Computing Systems (JETC), vol. 18, no. 4, pp. 1–19, 2022.JETC

    Paper

  • Hyunmyung Oh, Hyungjun Kim, Daehyun Ahn, Jihoon Park, , Inhwan Lee, Jae-Joon Kim⁺. “Energy-Efficient In-Memory Binary Neural Network Accelerator Design Based on 8T2C SRAM Cell.” IEEE Solid-State Circuits Letters (SSC-L), vol. 5, pp. 70–73, 2022.SSC-L

    Paper

  • Sungju Ryu, Hyungjun Kim, Wooseok Yi, Eunhwan Kim, , Taesu Kim, Jae-Joon Kim⁺. “BitBlade: Energy-Efficient Variable Bit-Precision Hardware Accelerator for Quantized Neural Networks.” IEEE Journal of Solid-State Circuits (JSSC), vol. 47, no. 6, pp. 1924–1935, 2022.JSSC

    Paper

  • Daehyun Ahn, Hyunmyung Oh, Hyungjun Kim, , Jae-Joon Kim⁺. “Maximizing Parallel Activation of Word-Lines in MRAM-Based Binary Neural Network Accelerators.” IEEE Access, vol. 9, pp. 141961–141969, 2021.IEEE Access

    Paper

  • Wangxin He, Shihui Yin, , Xiaoyu Sun, Jae-Joon Kim, Shimeng Yu, Jae-sun Seo⁺. “2-Bit-per-Cell RRAM-Based In-Memory Computing for Area-/Energy-Efficient Deep Learning.” IEEE Solid-State Circuits Letters (SSC-L), vol. 3, pp. 194–197, 2020.SSC-L

    Paper

  • Shihui Yin, Yandong Luo, , Wangxin He, Xu Han, Xiaoyu Sun, Hugh Barnaby, Jae-Joon Kim, Shimeng Yu, Jae-sun Seo⁺. “Monolithically Integrated RRAM- and CMOS-Based In-Memory Computing Optimizations for Efficient Deep Learning.” IEEE Micro, vol. 39, no. 6, pp. 54–63, 2019.IEEE Micro

    Paper

Yulhwa Kim

Assistant Professor

yulhwakim_at_skku_dot_eduGoogle Scholar

Interests

Efficient AI Algorithms, Computer Architecture, Hardware-Software Co-Design, and AI-driven Hardware Design and Automation

Education

  • Ph.D. in Convergence IT Eng.POSTECH

  • B.S. in Creative IT Eng.POSTECH

Professional experiences

  • Assistant ProfessorSungkyunkwan University

    Mar. 2024 – Present

  • Postdoctoral ResearcherSeoul National University

    Aug. 2022 – Feb. 2024

  • Research InternQualcomm

    Jun. 2020 – Sep. 2020

  • Research InternIBM T.J. Watson Research Center

    Jun. 2019 – Aug. 2019

Office
Room #400306, Semiconductor building

Seungmin Kim

M.S. Student

Department of Semiconductor and Display Engineering

Current research

Analog circuit topology generation, device parameter optimization

Interests

AI for EDA(Electronic Design Automation)

Publications1

Mingun Kim

M.S. Student

Department of Electrical and Computer Engineering

Current research

Digital design automation, Agentic RTL verification

Interests

AI for EDA(Electronic Design Automation), AI Hardware Systems

Publications1

Yuna Lee

M.S. Student

Department of Electrical and Computer Engineering

Current research

Efficient video/image understanding in MLLMs

Interests

Efficient AI, Multimodal LLM

Publications2

Kyoungho Min

M.S.–Ph.D. Student

Department of Electrical and Computer Engineering

Current research

Flow model initial noise modification for enhanced image quality

Interests

Efficient AI, Multimodal, Image Generation (T2I)

Publications1

Kanghyun Noh

M.S.–Ph.D. Student

Department of Semiconductor Convergence Engineering

Current research

Efficient AI Agents

Interests

AI Algorithms, Efficient AI models

Publications1

Jinheon Choi

M.S.–Ph.D. Student

Department of Electrical and Computer Engineering

Current research

Efficient LLM Compression (KV Cache, Quantization)

Interests

Efficient AI, Multimodal, AI Agent Systems

Publications1