APIBench

Introduced 2021-12-23

APIBench is a benchmark dataset designed for evaluating the performance of API recommendation approaches. It was introduced in the paper titled "Revisiting, Benchmarking, and Exploring API Recommendation: How Far Are We?"¹. Let's delve into the details:

  1. Purpose and Context:

    • APIBench serves as a standardized evaluation platform for assessing the effectiveness of various API recommendation techniques.
    • Researchers and practitioners can use it to compare and analyze different approaches in the field of API recommendation.

  2. Components:

    • APIBench comprises two sub-datasets:
      • APIBench-Q: This dataset focuses on query-based API recommendation. It involves providing relevant APIs to developers based on natural language queries that describe programming requirements.
      • APIBench-C: This dataset deals with code-based API recommendation. It aims to predict the next API to use based on the code context.

  3. Contents:

    • APIBench-Q:

      • Contains both Java and Python versions.
      • Includes original queries along with corresponding APIs and API classes.
      • Also provides reformulated queries derived from the original ones using various techniques (such as RACK, NLP2API, SEQUER, Google Prediction Service, and NLPAUG).
      • The dataset size:
        • Original Queries:
          • Python: 4,309
          • Java: 6,563
        • Expanded Queries (after reformulation):
          • Python: 173,517
          • Java: 400,126

    • APIBench-C:

      • Similar to APIBench-Q, it has both Java and Python versions.
      • Contains metadata files for code in different domains.
      • Metadata includes information about function lengths (long, normal, and short keywords).

  4. Access:

    • Since GitHub does not host large datasets, you can download the APIBench dataset from Zenodo².

In summary, APIBench provides a valuable resource for evaluating and advancing API recommendation techniques, fostering research and development in this area. 🚀🔍

Source: Conversation with Bing, 3/20/2024 (1) GitHub - JohnnyPeng18/APIBench: APIBench is a benchmark for evaluating .... https://github.com/JohnnyPeng18/APIBench. (2) APIBench: A Benchmark Dataset for Evaluating API ... - Zenodo. https://zenodo.org/records/5797297. (3) api-benchmark - npm. https://www.npmjs.com/package/api-benchmark. (4) The world's #1 API & code benchmark platform | CyBench. https://cybench.io/. (5) undefined. https://github.com/masud-technope/RACK-Replication-Package. (6) undefined. https://github.com/masud-technope/NLP2API-Replication-Package. (7) undefined. https://github.com/kbcao/sequer. (8) undefined. http://suggestqueries.google.com/complete/search?. (9) undefined. https://github.com/makcedward/nlpaug.