Testing

This section provides help for creating your API Test strategy and plans.

Shift left

Test early in the API lifecycle and continue testing at every stage. This 'shift left' approach challenges you to develop T-shaped skills. For testers, this may involve writing code for automated tests. For developers, it means recognising that testing is a shared responsibility.

Automate as much as you can

Test automation is crucial for shifting left and you SHOULD execute tests as part of your CI/CD pipelines. This practice helps identify and fix issues as they are introduced.

Validate the OpenAPI specification

Check the OpenAPI specification for structure and format to ensure the contract is correct. Use the out-of-the-box rules provided by linting tools like Spectral or Postman Linter to verify proper JSON and YAML formatting.

It is RECOMMENDED you write a custom ruleset to enforce the rules described in this document.

You SHOULD Inspect the API contract for the following, adding these checks to your CI/CD pipeline.

• Endpoints are named correctly.

• Resources and their types are accurate.

• Relationships between resources are properly defined.

• No functionality is missing or duplicated.

Unit tests

Write unit tests as you build your API to catch issues early. Start with minimal tests and expand as necessary -- do not overcomplicate matters.

You SHOULD add these tests to your CI/CD pipeline and publish code coverage metrics in a Test Summary report.

Functional tests

Functional tests SHOULD verify that:

• The correct HTTP headers and HTTP status codes are returned.

• The media type and its payload are correct, with appropriate field names, types and values.

• Endpoints behave according to the specified business logic and requirements.

Test for failure

Functional testing SHOULD include negative tests and check for valid error responses. These SHOULD cover scenarios for handling transient faults.

Behaviour driven development (bdd)

Automate validation by incorporating BDD executable scenarios into your functional tests. For example:

Security tests

Plan security testing with the API platform and cyber security teams. While many tests are executed against the proxy, you SHOULD also test the backend API for common vulnerabilities.

Refer to the Security section when planning your tests. Examples of common security tests include:

• Input validation tests to prevent SQL injection attacks.

• Verifying that only authorised users can access patient data.

• Data encryption checks to confirm that data is transmitted over HTTPS.

• Rate limiting tests to detect and mitigate potential brute-force attacks.

Performance tests

You MUST check the maximum response time.

You SHOULD consider whether to conduct, baseline, load, stress, soak and scalability tests.

Usability tests

You SHOULD test the entire API consumer journey including documentation, authentication and code examples, to ensure usability for users without prior knowledge of our systems.

Keep tests organised

Tests MUST be organised and named according to our coding standards.

Test summary report

You SHOULD write a Test Summary report for every major, minor or patch version of your API. The content MAY vary depending on the version but you SHOULD consider including each of the following headings.

Test environment

You SHOULD deploy APIs to a sandbox environment to allow client testing. Contact the platform team for further assistance.