Bayesian Perspective on Memorization and Reconstruction
Haim Kaplan, Yishay Mansour, Kobbi Nissim, Uri Stemmer
2025-05-29Memorization
Abstract
We introduce a new Bayesian perspective on the concept of data reconstruction, and leverage this viewpoint to propose a new security definition that, in certain settings, provably prevents reconstruction attacks. We use our paradigm to shed new light on one of the most notorious attacks in the privacy and memorization literature - fingerprinting code attacks (FPC). We argue that these attacks are really a form of membership inference attacks, rather than reconstruction attacks. Furthermore, we show that if the goal is solely to prevent reconstruction (but not membership inference), then in some cases the impossibility results derived from FPC no longer apply.
Related Papers
What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests2025-07-15Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination2025-07-14Entropy-Memorization Law: Evaluating Memorization Difficulty of Data in LLMs2025-07-08MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI2025-06-30Listener-Rewarded Thinking in VLMs for Image Preferences2025-06-28Where to find Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Test2025-06-26Counterfactual Influence as a Distributional Quantity2025-06-25Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA2025-06-25