OpenWSN network stack

Provides a RIOT adaption of the OpenWSN network stack. More...

Detailed Description

Provides a RIOT adaption of the OpenWSN network stack.

See also

https://github.com/openwsn-berkeley/openwsn-fw

Warning

This feature is experimental!

OpenWSN RIOT Port

This implementation integrates the OpenWSN full network stack (UDP, IPv6 (6LoWPAN), RPL, 6TiSCH) into RIOT.

It can be used instead of GNRC on supported 802.15.4 radios, and compared to GNRC, provides a full 6TiSCH implementation. It supports RIOT's Sock API API, so it can be used as a drop-in replacement for application built on top of Sock API.

This port provides a new RIOT "board" to the OpenWSN software. In this way RIOT's hardware abstraction connects to OpenWSN's interfaces.

The simple scheduling mechanism in OpenWSN is run in a RIOT thread with second highest priority after the radio thread (THREAD_PRIORITY_MAIN - 4).

The current port of OpenWSN currently needs a root node that works along an external tool that performs routing and handles join procedure: Openvisualizer

Joining a network

The first thing a new mote will need to do is to find a network. On boot it will actively be listening for enhanced beacons. Once a beacon is received it will adjust its timers drift and synchronize with the network. Re-synchronization will be happening constantly to compensate for oscillator and timer drifts.

Once synchronized the node will need to join the network. OpenWSN uses CoJP constrained join protocol an the stack itself only handles Join Requests. The JRC (join registrar/coordinator, a central entity) is not running on the root node, but alongside it, in the OpenVisualizer external tool. This functionality is now optional but can be included through the openwsn_cjoin module.

Once joined the device has the required keys to start listening to DIS (DODAG Information Solicitation) messages and to send DIO (DODAG Information Object) requests. Once it knows about the topology of the network it is able to send packets.

OpenWSN uses source routing. This means that unless the recipient of a packet is one of the parents in the RPL tree the packet will have to go up the tree to the root node. But in OpenWSN RPL implementation the node does not know how to route, instead it is OpenVisualizer which generates an SRH (Source Routing Header), attaches to the incoming packet and sends it down the tree.

Hardware abstraction implementation

Following, details about the implementation of selected hardware modules.

sctimer

The sctimer ("single compare timer") in OpenWSN is the lowest timer abstraction which is used by the higher layer timer module opentimers. In the end it is responsible for scheduling on the MAC layer. To enable low power energy modes, this timer usually uses the RTC (real time clock) or RTT (real time timer) module.

This port has two possible implementations or sctimer, one on top of periph_rtt and another on top of ztimer, sctimer_rtt and sctimer_ztimer respectively. If possible ztimer should be preferred, being a virtual timer it will allow RIOT applications/modules to use the low level RTT timer. But ztimer (any virtual timer) has some overhead which can be costly depending on the different platforms used, specially when openserial is required. More on this in openserial and Known Issues.

sctimer_ztimer

In order to get the most portable code, this implementation uses ztimer and defines a new ztimer_clock (ZTIMER_32768) that operates at 32768Khz to have a resolution of ~30usec/tick (same as OpenWSN).

When available ZTIMER_32768 will be built on top of periph_rtt to get low power capabilities. If not it will be built on top of a regular timer. In either case it will be shifted up if the base frequency is lower than 32768Hz or frac if higher.

When next interrupt to schedule is already late, current time, implementations in OpenWSN directly trigger a hardware interrupt. Until able to trigger sw isr directly a callback is set 0 ticks in the future, which internally will be set to now + RTT_MIN_OFFSET.

sctimer_rtt

In order to reduce overhead this implementation uses bare RTT. It expects a RTT running at 32768Hz to have a resolution of ~30usec/tick (same as OpenWSN). If RTT_FREQUENCY is lower than 32768Hz then a simple time-division mechanism will be used to speed up the clock. This only works if RTT_FREQUENCY is 32768Hz/2.

When next interrupt to schedule is already late, current time, Implementations in OpenWSN directly trigger a hardware interrupt. Until able to trigger sw isr directly a callback is set RTT_MIN_OFFSET ticks in the future.

radio

The radio adaptation maps to RIOT's netdev API or on IEEE802.15.4 Radio Hardware Abstraction Layer API. The later is preferred but not yet supported by all boards.

Hardware MAC layer features such as CSMA/CA, ACK handling and retransmissions are handled by OpenWSN, so the radio driver must support disabling AUTOACK and CSMA handling by the hardware. Frame filtering must as well be disabled.

OpenWSN needs to be notified when a frame reception/transmission starts and when it ends. Without these synchronization can still work but in a deteriorated way.

OpenWSN expects to recover crc information on every received frame even if it will simply drop frames with invalid crc. The stack can function correctly if radio drivers automatically drop frames with an invalid crc (i.e. the stack doesn't get notified about these frames).

ieee802154_hal

The radio drivers should support the following caps:

- `IEEE802154_CAP_IRQ_RX_START`
- `IEEE802154_CAP_IRQ_TX_START`

For CRC error handling:

- `IEEE802154_CAP_IRQ_CRC_ERROR`

netdev

The radio adaptation preloads the buffer so NETOPT_PRELOADING must be supported.

The radio drivers should support the following netdev events:

- `NETDEV_EVENT_RX_STARTED`
- `NETDEV_EVENT_TX_STARTED`
- `NETDEV_EVENT_RX_COMPLETE`
- `NETDEV_EVENT_TX_COMPLETE`

uart

In RIOT, the first configured uart device is mapped to STDIO in most cases. In OpenWSN however, the openserial tool uses uart to feed external software running on a host computer such as Openvisualizer. To enable use of these tools, an uart adaptation is provided.

This is provided through the openwsn_serial (openserial) module. It will use the next available uart that is not used by STDIO (checking STDIO_UART_DEV). When multiple uart are available STDIO and openserial can be used in parallel. If stdio_null is used then openserial will use STDIO_UART_DEV, otherwise it will use the next available uart. e.g. If STDIO_UART_DEV = UART_DEV(1) OPENWSN_UART_DEV = UART_DEV(0) if there are uarts.

OpenWSN uart abstraction makes use of tx hardware interrupts to execute a previously registered callback after every byte is sent out. These interrupts are currently not defined in RIOT in a generic way, so instead a timer is set to fire shortly after a byte is written.

It uses ztimer to set the timer since it's already pulled in as a dependency.

Openserial

As was mentioned before any OpenWSN network will require a root node which is connected to an OpenVisualizer instance running on a host computer. Interaction between OpenVisualizer and the root-node is done over serial. As OpenWSN uses source routing, this means that ultimately all network traffic must go from the root node to OpenVisualizer and back.

OpenSerial uses software flow control (XonXoff) to turn off serial activity while time critical TSCH operation are ongoing. But software flow control can cause issues since delays in the serial pipe (either because of a remote connection, buffers, etc..) can lead to bytes being transmitted when one side is not yet ready.

Serial data is transmitted as High-Level Data Link Control (HDLC) frames. Since network traffic is tunneled through the serial pipe, in order to to have a stable connection these packets must not be lost. Packets can be lost in multiple ways but at the end it reduces to bytes being overridden in the uart reception buffer.

1. The last byte in an HDCL frame is received
2. Serial pipe delays hindering XonXoff operation
3. Interrupts are masked or higher/equal ISR are running

1. When the last byte of a frame is received some parsing and handling of the frame occur. This takes some time and during that time OpenSerial can't handle more incoming bytes. This in practice limits the baudrate depending on the CPU's speed.
2. This can occur when a debugger/virtual-port might act as a buffer between the mote and the host, or when there is a latency in the connection (for example tcp connection to iotlab).

3. Since uart reception is interrupted based if the uart ISR is not serviced for too long then bytes start overriding each other in the reception buffer. In an application where only OpenWSN is running then there are 3 functions/operations that run in ISR or with disabled ISR.

   (a) opentimers_timer_callback (OpenWSN timer abstraction callback) (b) sctimer_setCompare and sctimer_readCounter (c) ztimer_handler

(a) and (b) are closely related since opentimers_timer_callback will itself call sctimer_setCompare and sctimer_readCounter. And this itself will depend on ztimer_set and ztimer_now or rtt_get_counter or rtt_set_alarm execution time.

(c) is also related to (a) (b) since ztimer_handler will also call opentimers_timer_callback as well as the underlying rtt functions.

Since a 115200 baudrate means ~1byte every 10us, none of the above can take longer than that or bytes could be lost at that baudrate.

Because of the above mentioned reasons, it is preferable to use sctimer_rtt for the root node or "border router" on an OpenWSN network, since this reduces the likeliness of packets being lost. For non root nodes, OpenSerial only provides debugging information so no special care needs to be taken.

It is also recommended that the root node should act as a border router running only the OpenWSN stack to avoid other threads/ISR disrupting serial reception.

Tested Platforms and Pin configurations

So far, this has been successfully tested on iotlab-m3,nucleo-f103 and samr21-xpro, all based on at86rf23x radios. These radios make use of the radio_netdev adaptation.

Synchronization

To join a network a node must first receive EB (enhanced beacons). Once an EB is received the node will be synchronized with the network. Synchronization times are not deterministic, they depend on the following:

SLOTFRAME_LENGTH*SLOTDURATION*P_CHANNEL*P_EB

SLOTFRAME_LENGTH in OpenWSN is 101, and this port uses 20ms as the slotOffset duration. P_EB specifies the a probability for a node to transmit an EB. By default it's 10%, that means that on average it will take 10 tries before an EB is transmitted. P_CHANNEL is the probability for the transmitter's and receiver channel to match. If channel hopping is disabled this means that the average worst case scenario is 101*20ms*10 ~= 20s, so 20s for synchronization to take place. EB_PORTION can be changed to increase the likelihood of EB to be sent. This can also be achieved by reducing SLOTFRAME_LENGTH, but the later can have an impact on the MSF (Minimal Scheduling Function). If too few cells are available this could increase the likelihood of collisions.

On the other hand if channel hopping is enabled then the joining node picks a random channel to start listening on. The transmitter also picks a random channel to start transmitting on and then follows a channel hopping template. This would take on average ~8 tries for it to hit the correct channel for the first time, and then it would hit it every 16 hops, on average this could lead synchronization times of around 320s.

If nodes are having trouble in staying synchronized increasing P_EB by reducing the value of EB_PORTION can also be done. Note that EB_PORTION and SLOTFRAME_LENGTH are not configurable by default so need to be overridden with CFLAGS.

See Tune parameters in OpenWSN for more details.

Timing

Timing is essential for OpenWSN to work properly. For optimal results most parameters in board_info.h should be measured for the specific hardware used. OpenWSN has done that for most of their boards. These values can not be taken directly from OpenWSN since they do not necessarily use the same TIMER's or clock speeds.

For more details on those parameters refer to: https://openwsn.atlassian.net/wiki/spaces/OW/pages/688251/State+Machine

Since all these parameters are HW dependent, it also means that hybrid networks (different type of underlying hardware) might desynchronize often, or not manage to keep in sync at all.

Print messages during TSCH operation should be avoided since these can disrupt TSCH timings.

ledpins & debugpins

The OpenWSN software provides different hooks all over the stack to toggle different LEDs as well as debug pins to examine state and scheduling of a node. Default configuration files are provided for both. The LED configuration maps to RIOTs LEDX_PIN definitions, if available. On Nucleo boards LED0 line is shared SPI, so is not used.

The default configuration can be overwritten by setting OPENWSN_LEDS_DEFAULT in the form of leds_config_t. The debugpins work similarly by setting OPENWSN_DEBUGPINS_BOARD in the form of debugpins_config_t.

The default configuration maps to OpenWSN reference hardware openmote-b.

Optional Modules and Features

The following modules are optional and can be disabled if not needed, or to lower the stack footprint.

• openwsn_cjoin: this enabled the use of Constrained Join Protocol (CoJP)
• openwsn_6lo_fragmentation: this enable 6LoWPAN fragmentation
• openwsn_iee802154e_security: enable link layer security
• openwsn_adaptive_msf: allow the MSF algorithm to dynamically remove and allocate slots

Testing and debugging

List of some items which are helpful to explore the functionality of OpenWSN:

• LED pins and debug pins as mentioned above in combination with a logic analyzer. The expected behavior is described in: OpenWSN wiki.
• The provided test application provides a UDP client and server. If the UDP server is able to receive packets, the mechanism is considered to work correctly. You should also be able to ping the device from your host. See tests/pkg/openwsn/README.md for more details.
• To speed up synchronization and make sniffing easier you can disable channel hopping by setting (CFLAGS=-DIEEE802154E_SINGLE_CHANNEL=26).
• To sniff the packets either use a 802.15.4 capable board and follow at: https://github.com/RIOT-OS/applications/blob/master/sniffer/tools/README.md. Alternatively use a Raspberry Pi with an external radio such as Openlabs and incorporate Linux WPAN tools. In addition to that, there's also other hardware such as the ATUSB IEEE 802.15.4 USB Adapter which can directly be used on your Linux computer with WPAN tools installed. If you conduct your experiments on the IoT-LAB testbed you might want to use a: sniffer profile.
• To explore the channel hopping mechanism there are rather expensive multi-channel sniffers such as the BeamLogic 802.15.4 Site Analyzer that can sniff all channels simultaneously. Alternatively you can set up multiple separate sniffer devices locally or make use of the sniffer_aggregator on the IoT-LAB testbed.

• To test Openserial on a given platform the target make openv-serial can be used on a BOARD flashed with tests/pkg/openwsn (USEMODULE=openwsn_serial) must be included as well. The following output should appear:

  Test Setup:
  ------------------
  Iterations:    100
  Packet length: 100
  Echo timeout:    2
   
  Test Progress:
   
    [####################################]  100%
   
  Test Statistics:
  ------------------
  Pkts send:      100
  Echo success:   100
  Echo timeout:     0
  Echo corrupted:   0

The test should be considered passing if success rate is > 98%.

Known Issues

The following errors might be visible when using openwsn_serial:

• [OPENSERIAL] wrong CRC in input Buffer

Since a timer is set to simulate a uart transmit interrupt, it can happen that the interrupt is missed if another interrupt occurs during that time, this seems to lead to the input buffer missing a byte and so CRC fails. More details where given in the openserial section.

• [IEEE802154E] wdDataDuration overflows while at state 19 in slotOffset 0

This error can show up when the radio starts receiving (receives the SFD) and therefore triggers a NETDEV_RX_STARTED but then no NETDEV_TX_STARTED event follows. This happens when packets with invalid CRC are received. netdev currently silently drops these packets without notifying upper layers. But this does not affect the stack operation, so they can be ignored.

• [IEEE802154E] large timeCorr.: -18 ticks (code loc. 0)

Most crystals used to clock the RTT will drift even those with a very similarly drift (10-30ppm). It's is normal then for motes adjust their timerCorr as long as it stays within the above mentioned margins and if motes are able to stay synchronized over the long run. If there aren't then maybe board_info.h parameters require tuning for the specific platform.

• [JRC:ERROR] Type-error in conversion of 5N=ex

This errors happen when a node tries to rejoin the network. This error is only associated to a log print, so can be ignored.

• [coap:WARNING] coapRcBadRequest(reason=OSCOAP unprotect failed: oscoapError(reason=Replay protection failed))

The join procedure uses a replay window. If a node had already joined the network and for some reason attempts to rejoin again, then the replay windows will need to expire for it's join request to be accepted.

The following errors are platform specific.

• samr21-xpro issues:
  • The serial debugger hinders Openserial operation, an ftdi device must be used.
• sam0 issues:
  • sam0 requires 180us busy loops every time an alarm is set or the counter is read. Because of the later only sctimer_rtt can be used and the max. tested baudrate for openserial is of 19200 bauds.
• iotlab-m3 issues:
  • openserial does not work reliably over 57600 bauds or when using sctimer_ztimer.

Other errors:

• missed characters over stdio

TSCH state machine disable occurs in IRQ context and disables IRQ during time critical sections. This can cause bytes sent over stdio to be missed.

Todos

• sctimer to trigger an ISR immediately using software interrupts.
• The UART wrapper uses ztimer to fake an interrupt after one byte has been sent. This should also be done with software interrupts.

Future Steps

It would be desirable to achieve an extraction of the MAC layer.

Modules
	OpenWSN-specific implementation of the sock API
	Provides an implementation of the Sock API by the OpenWSN network stack.

Files
file	board_info.h
file	openwsn.h
file	openwsn_board.h
	RIOT HAL is provided as a new "board", a "RIOT board" to OpenWSN hardware abstraction interfaces.
file	openwsn_debugpins.h
	Provides an adaption of OpenWSN debug pin handling to RIOTs handling of GPIOs.
file	openwsn_debugpins_params.h
	Default configuration for the OpenWSN debugpins.
file	openwsn_leds.h
	Provides an adaption of OpenWSN led handling to RIOTs handling of LEDs and/or GPIOs.
file	openwsn_leds_params.h
	Default configuration for the OpenWSN leds.
file	openwsn_log.h
	System logging header OpenWSN definitions.
file	openwsn_radio.h
	RIOT adaption of the "radio" bsp module definitions.
file	openwsn_uart.h
	For details on the implementation check pkg/openwsn/doc.txt.
file	scheduler_types.h
	RIOT scheduler types variable declaration.