Connecting to the DEFRA Digital Waste Tracking API
As part of building the technical infrastructure for the Digital Waste Tracking project, I have been working through the process of connecting the system to DEFRA's external API.
This has involved more than simply sending a request to an endpoint. Before a movement can be submitted, several layers of authentication, API requirements and test-environment configuration need to be understood.
1. Starting with authentication
The first technical issue I encountered was an authentication error:
400 invalid_client
At this stage, I had the DEFRA Client ID, Client Secret, OAuth URL and API base URL, but the authentication request was not succeeding.
I initially considered several possibilities, including whether the credentials had been activated correctly, whether I was using the correct environment, or whether there was an issue with the authentication flow.
I contacted DEFRA to verify the credentials.
They confirmed that the credentials were correct and clarified that the authentication process needed to be performed through the documented POST request rather than treating the OAuth endpoint as something that could simply be accessed through a browser.
I then tested the authentication flow separately.
The result was successful: the system was able to obtain an access token.
This established the first important milestone:
The software could authenticate with DEFRA's OAuth service.
2. Separating authentication from API requests
The next step was understanding that authentication and API integration are two separate stages.
The process is essentially:
Client ID + Client Secret
↓
DEFRA OAuth service
↓
Access token
↓
Bearer token authentication
↓
Receipt API request
Obtaining an access token does not mean that a waste movement has been successfully submitted.
It only confirms that the software has successfully authenticated and can now use the resulting token when communicating with the API.
To keep these stages separate, I created a standalone token test before connecting authentication directly to the movement submission logic.
3. Building the token-management layer
The authentication layer now handles several parts of the OAuth process.
Credentials are loaded through environment variables rather than being placed directly into the source code.
The system then:
constructs the required authentication request;
sends the POST request to the OAuth token endpoint;
receives and parses the access token;
handles HTTP and network errors;
caches the token;
checks when the token expires; and
refreshes the token before expiry.
A small safety buffer is also used before expiration so that the application does not wait until the exact expiry point before attempting to refresh the token.
This means the authentication component can operate independently from the waste-movement logic.
4. The apiCode question
Once authentication was working, the next question was different:
What is the apiCode required by the Receipt API?
The API request examples contained an apiCode field, but this was separate from the OAuth credentials.
This initially raised an important question about whether a real receiving organisation or waste site would need to be onboarded before a test movement could be submitted.
Further investigation of DEFRA's developer documentation resolved this.
DEFRA provides dedicated API codes for testing, including dummy codes that software developers can use when testing against the test environment.
This established an important distinction:
OAuth credentials identify and authenticate the software.
The apiCode provides the relevant test organisation context for the API request.
The two serve different purposes.
5. Understanding the different technical resources
Another part of the investigation has been understanding how DEFRA's different technical resources fit together.
There are several layers of documentation rather than one single document explaining the entire integration.
Authentication documentation
This explains how the software obtains an access token using the Client Credentials flow.
API specification
The Receipt API specification describes the formal structure of the API, including the requests, fields and responses that the system needs to work with.
Bruno collection
The Bruno collection provides executable example requests and test scenarios.
This is particularly useful because it moves the documentation from:
"This is what the API looks like."
to:
"This is an example of how a request can actually be made."
API-code documentation
The API-code documentation explains which codes can be used in the test and production environments.
Understanding the relationship between these resources has been an important part of the technical investigation.
6. Where the integration is now
The authentication layer has been successfully tested.
The test API-code requirement has also been identified.
The next stage is therefore the first actual Receipt API submission.
The current flow is:
DEFRA OAuth credentials
↓
OAuth authentication
↓
Access token obtained
↓
Test apiCode identified
↓
Construct /movements/receive request
↓
Submit request to the external test environment
↓
Analyse the response
↓
Iterate through any validation issues
The next milestone is to successfully construct and submit a valid test movement and receive the expected response from the external test environment.
What this investigation has shown so far
The interesting part of this process has been discovering that API integration is not one problem.
It is a series of smaller problems:
Authentication → token management → API identity → request structure → validation → successful submission.
Each stage has required a different part of the documentation and, in some cases, direct investigation when the expected behaviour was not immediately obvious.
This is still an active technical investigation.
The purpose of documenting it is to record what happens while the infrastructure is being built, including the problems encountered along the way, rather than only documenting the finished integration afterwards.