CoAP Overview

The Constrained Application Protocol (CoAP) is a specialized web transfer protocol for use with constrained nodes and constrained (e.g., low-power, lossy) networks. The protocol is designed for machine-to-machine (M2M) applications such as smart energy and building automation. On the other hand, the protocol easily interfaces with HTTP by providing familiar request-response model: URIs, content types and status codes.[1]
The 1NCE Data Broker CoAP endpoint is utilizing UDP as transport but provides reliable message delivery by using message confirmations, deduplication logic by validating Message IDs, and other functionality described in the protocol specification.

CoAP Communication

While using UDP protocol for transport, the CoAP protocol offers reliable communication by using message confirmation mechanism. Each CoAP request has to be acknowledged by the server, so that the client would be sure that the message was processed:

There are few key moments that allow reliable communication:

1. To maximize the chance that the message succeeds even in the lossy network environment, CoAP has a retransmission mechanism. The client would re-send the Confirmable message (CON) until the Acknowledgement (ACK) is received or the exchange lifetime has ended. The total exchange lifetime (EXCHANGE_LIFETIME) is the time from starting to send a Confirmable message to the time when an acknowledgment is no longer expected.
   By default, the EXCHANGE_LIFETIME value is 247 seconds.
2. CoAP Messages contain Message ID (also known as MID) to detect duplicates due to retransmissions. The Message ID has to be unique during the EXCHANGE_LIFETIME, so the client's endpoint should be able to specify a unique MID value if messages are being sent often enough. Most high-level CoAP clients are managing MID uniqueness internally, but for low-level clients like modem Quectel BG95, it can be specified in the AT command as msgID[2]:

AT+QCOAPHEADER=<clientID>,<msgID>,<mode>[,<TKL>,<token>]

There are two examples of the retransmission situation in a single exchange lifetime:

• client's CON message did not reach the server. The client is resending the same CON message with the same MID.
• server's ACK message did not reach the client. The client is resending the same CON message with the same MID (because there was no acknowledgment). The server is answering with the same ACK because it sees the already processed MID and does not process the request again.

1NCE CoAP Server

The CoAP Endpoint hosts a POST endpoint on the server root path. The Endpoint supports both translatable messages using the Translation Service and not.

The POST endpoint takes an optional query parameter (or Location-Query) “t“ to provide the MQTT topic used for forwarding this message to the MQTT broker. E.g. coap://coap-service:5683/?t=topicName. The Location-Query is limited to 255 characters by the CoAP protocol, hence the topic name itself can only contain up to 253 characters (“t“ and “=“ also count as characters). The topic name itself can only contain alphanumeric characters, underscores, and forward slashes (no two slashes in a row). If these constraints are violated, a Bad Request (4.01) will be returned.

When that Device could not be found or is in a non-active status, the CoAP service will return an Unauthorized (4.01) and no further processing of the message will take place.

DTLS encryption for CoAP

Making sure data is securely sent from a device to the 1NCE Connectivity Suite is an important part of gaining the trust of potential customers with their data. To be able to provide this secure connection 1NCE has implemented a DTLS layer in the CoAP communication from the device to the Connectivity Suite. This allows the device to securely send its data to the 1NCE Connectivity Suite without the possibility of messages being read or modified along the way. The picture above describes this process. First, when the device is ready to onboard itself it will call the CoAP onboarding endpoint. This will retrieve the necessary DTLS info to onboard itself securely and initialize the CoAP connection using a PSK.

Features & Limitations

DTLS as a security protocol provides secure and fast data streaming. This comes with some advantages but also has some limitations. We will explore these below in the context of a CoAP connection.

Features

DTLS is able to provide datastream functionality to devices with a low delay compared to TLS. It is able to provide this because it preserves the semantics of the underlying transport. DTLS also has better security. Because of that, communication between client-server applications cannot be eavesdropped on or tampered with. This makes sure that the data that is streamed is the same data that is received.

To make all this happen DTLS makes use of the UDP protocol. This means that the data is sent in a fire and forget way so no handshake occurs and the message is sent without any confirmation about it being received on the other end. UDP does also avoid the TCP meltdown problem where different transport layers compensating for each other cause delays in the data transfer.

Limitations

The main limitation of DTLS is the use of the UDP protocol. Even though it provides a couple of advantages over TLS, it also has a couple of drawbacks like having to deal with packet reordering, loss of datagram, and data larger than the size of a datagram network packet.

Encryption Key

To be able to use DTLS an encryption key is required to ensure security. For the 1NCE connectivity suite, a pre-shared key is used to encrypt and decrypt the data. The key can be obtained by the device calling the CoAP onboarding endpoint and the endpoint getting the key from the database. The result is a client identity and a pre-shared key in JSON format or CSV format. A request looks like the following:

curl --i --v https://device.connectivity-suite.cloud/device-api/onboarding/coap

First, the device is identified. Also the, “Accepts“ header is used to determine in what format the response should be sent back, this can be either application/JSON or text/CSV. The endpoint then is able to look up details about the device and will send the following response.

JSON
{ 
  clientIdentity: string, 
  preSharedKey: string 
}

CSV
"clientIdentity, preSharedKey"

This pre-shared key in the response can be used by the device to initialize a connection using DTLS. The key will be regenerated if an onboarding request is done more than 5 minutes after the last time onboarding was called to maintain integrity. To refresh the token the same endpoint is used. The endpoint reads the existing key (if provided) and determines if it is still valid. If it is not then it generates new information and returns that to the user.

Example

Below is an example of retrieving the pre-shared key and client identity. First, a request is done to the COAP onboarding endpoint, this request will result in the following CSV object:

preSharedKey, clientIdentity

For the endpoint to return CSV data instead of JSON an “Accept” header is set to text/csv.

The following piece of code is used to process this response so the device is fully onboarded with credentials and ready to send messages. In the security, parameters object the clientIdentity and preSharedKey will be set and these parameters will be passed to the CoAP DTLS client.

const coapDtls = require("node-coap-client").CoapClient;

async function coapPostDtls(clientIdentity, preSharedKey, postPayload) {
  
  const securityParameters = {
    psk: {
      [clientIdentity]: preSharedKey,
    },
  };
  coapDtls.setSecurityParams(coapsUrl, securityParameters);
  try {
   const result = await callPost(coapDtls, coapsUrl, postPayload);
   return result;
  } catch(e) {
    return e;
  }
}

In the following snippet, the initialized client will be used to send a message.

const coapDtls = require("node-coap-client").CoapClient;

async function callPost(client, url, postPayload) {
  console.log("Calling: ", url);
  try {
    const pingSuccess = await client.ping(url /* string | url | Origin */, [
      2000,
    ]);
    if (!pingSuccess) {
      console.log("Ping request failed");
      throw Error(`Ping to ${url} failed`);
    } else {
      console.log("Ping successfull");
    }
    const res = await client.tryToConnect(url);
    console.log("Connection attempt result: ", res);
    if (res) {
      const payload = Buffer.from(postPayload);
      const options = {
        /** Whether to keep the socket connection alive. Speeds up subsequent requests */
        keepAlive: true,
        /** Whether we expect a confirmation of the request */
        confirmable: confirmable,
        /** Whether this message will be retransmitted on loss */
        retransmit: false,
      };
      const result = await client.request(
        url /* string */,
        "post" /* "get" | "post" | "put" | "delete" */,
        payload /* Buffer */,
        options /* RequestOptions */
      );
      result.code = result.code.toString();
      printResponseHeaderBody(result);
      const resultObj = generateResponseObject(result.code, result.payload.toString());
      if (result.code !== "2.04") {
        
        throw new Error(resultObj);
      }
      return resultObj;
    }
  } catch (error) {
    console.log("Coap Error ", error);
    throw error;
  }
}

function generateResponseObject(respCode, body) {
  return {
    code: respCode,
    message: body,
  };
}

CoAP Endpoint Information

Base URL: device.connectivity-suite.cloud
Protocol: HTTPS
Supports HTTP Range Request
/device-api/onboarding/coap

Get Parameters

Responses

DTLS Onboarding Information

Information model:

The pre-shared key is valid indefinitely. In case the onboarding is called 5 or more minutes after the last time onboarding was called, the pre-shared key will be regenerated with a new value.

UnauthorizedResponse

API response when the license agreement is not accepted for the owner of the device:

ServerSideErrorResponse

API response in case of server-side errors:

References

[1] The Constrained Application Protocol (CoAP). https://datatracker.ietf.org/doc/html/rfc7252
[2] BG95&BG77&BG600L Series CoAP Application Note. https://www.quectel.com/wp-content/uploads/2021/10/Quectel_BG95BG77BG600L_Series_CoAP_Application_Note_V1.0.pdf