Xiaoguang Chang, Teng Wang, Changyin Sun, Wenzhe Cai
Scene graph generation is a sophisticated task because there is no specific recognition pattern (e.g., "looking at" and "near" have no conspicuous difference concerning vision, whereas "near" could occur between entities with different morphology). Thus some scene graph generation methods are trapped into most frequent relation predictions caused by capricious visual features and trivial dataset annotations. Therefore, recent works emphasized the "unbiased" approaches to balance predictions for a more informative scene graph. However, human's quick and accurate judgments over relations between numerous objects should be attributed to "bias" (i.e., experience and linguistic knowledge) rather than pure vision. To enhance the model capability, inspired by the "cognitive bias" mechanism, we propose a novel 3-paradigms framework that simulates how humans incorporate the label linguistic features as guidance of vision-based representations to better mine hidden relation patterns and alleviate noisy visual propagation. Our framework is model-agnostic to any scene graph model. Comprehensive experiments prove our framework outperforms baseline modules in several metrics with minimum parameters increment and achieves new SOTA performance on Visual Genome dataset.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Scene Parsing | Visual Genome | mean Recall @100 | 17.24 | C-bias |
| Scene Parsing | Visual Genome | mean Recall @20 | 11.63 | C-bias |
| 2D Semantic Segmentation | Visual Genome | mean Recall @100 | 17.24 | C-bias |
| 2D Semantic Segmentation | Visual Genome | mean Recall @20 | 11.63 | C-bias |
| Scene Graph Generation | Visual Genome | mean Recall @100 | 17.24 | C-bias |
| Scene Graph Generation | Visual Genome | mean Recall @20 | 11.63 | C-bias |