Request for Proposal

Request for Proposal

View all RFPs

SYCL 2020 CTS

Scope

The execution shall be divided into two phases, each limited to a fixed number of months. Work for each phase must be delivered in the order specified.

3.1 Phase 1: Creation of a Development Plan Time budget: 1 month.

The contractor shall devise a plan for how to approach development work on the SYCL CTS over the contract period. In particular, this should cover the following aspects:

  1. The contractor shall develop a methodology for assessing the current state of CTS coverage reading a particular SYCL feature or concept, possibly through automated or manual means, or a combination thereof.
  2. The contractor shall develop a system for structuring test code and conformance report generation
    taking account off:
    1. Optional features (e.g. USM, kernel bundles)
    2. Device capabilities/aspects (e.g. FP16, images)
    3. Deprecated functionality
    4. Functionality involving multiple devices (e.g. moving buffers between devices)
    5. Reduced feature set
    6. Backend interop
  3. The contractor shall select a representative set of devices to be tested on, in accordance with the working group. In particular, at least one CPU and GPU device should be selected, as well as one OpenCL-backend and one non-OpenCL-backend device should be selected.
  4. The contractor shall devise a system of tracking development progress and planning. In particular this process needs to be conducted in a way that allows coordination with other contributors such as Khronos Group members as well as third party open-source contributors. The working group suggests the usage of GitHub issues, pull requests and project boards for this purpose.

3.2 Phase 2: Continuous CTS Development Time budget: 5 months.

The contractor shall perform continuous work on the SYCL CTS by adding new test cases, improving existing ones as well as extending the underlying testing infrastructure as needed.
For each SYCL feature or concept that is to be tested, the following procedure should be adhered to:

  1. Assessment of current CTS coverage of the feature or concept.
  2. Creation of a test plan in accordance with the SYCL working group. Examples of test plans can be
    found in the public Khronos SYCL CTS GitHub repository [3].
  3. Implementation of said test plan.

The SYCL working group favors a “breadth-first” approach regarding coverage of SYCL features or concepts. In particular this includes coverage of important headline features added in SYCL 2020, including, but not limited to:

  • Backends & interoperability
  • Unified Shared Memory
  • Reductions
  • Group algorithms

In addition, development resources should be spent on updating existing 1.2.1 tests for SYCL 2020:

  • Focus on removed APIs, and Identify new, and changed cases within existing tests
  • Revisit generated test cases (sycl::vec, math-builtins)

FAQ:

  • Should there be PRs and a review period?
    • YES: We simply need to put more WG resources into weekly PR reviews.
  • Should contractors be in charge of reviewing external contributions?
    • NO: We’ll do that.
  • Test plans: Do we use them? Should contractors create them? Do we provide them (like Vulkan SC WG
    does)?
    • Contractors should create them before tackling new tests.

Estimated Amount of Work: View PDF

Responses Deadline: 5PM PT on Monday July 25, 2022

Contact Email: .(JavaScript must be enabled to view this email address)

RFP Document: This RFP document contains a detailed outline for this project and is not under NDA:

Download PDF