MMKE-Bench

We introduce MMKE-Bench, a benchmark designed to evaluate the ability of LMMs to edit visual knowledge in real-world scenarios. MMKE-Bench incorporates three editing tasks: visual entity editing, visual semantic editing, and user-specific editing. Additionally, it uses free-form natural language to represent and edit knowledge, offering more flexibility.The benchmark consists of 2,940 pieces of knowledge and 8,363 images across 33 broad categories, with evaluation questions automatically generated and human-verified.