NEW!!! In 2021, SN4I was included in the SLICES-RI infraestructure of the ESFRI roadmap, which is a fully programmable and virtualized, remotely accessible, European-wide research infrastructure designed to support large-scale experimental research activities, in topics regarding innovative network architectures and services. SN4I research "flavour" SLICES-RI specifically targets 5G Advanced and 6G experimentation applied to the Manufacturing (Industry 4.0 & Beyond). On the September 1st 2022, the 39 months duration SLICES-PP (Preparatory Phase) project (HORIZON-INFRA-2021-DEV-02) has been launched to prepare the infrastructure.

NEW!!!In 2022, SN4I has been integrated into Fed4FIRE+, a federared infrastructure across Europe that provides open, accessible, and reliable interoperable testing facilities to researches and innovators. The integration of SN4I into Fed4Fire+ is being apporached by converging the orchestration tools used in SN4I with the framework and tools provided by Fed4Fire+. In this regard, OSM is being integrated as a new type of resource in order to support the lifecycle management of Network Services (including virtual, physical, and hybrid) on top of SN4I resources.

Introduction

Welcome to the Smart Networks for Industry (SN4I) facility, which is an NFV- and SDN-aware communication network that interconnects two locations, Bilbao and Leioa, of the University of the Basque Country (UPV/EHU) with the Aeronautics Advanced Manufacturing Centre (CFAA) in Zamudio. In this regard, NFV provides a flexible way to deploy virtual services, while SDN allows the programming of network elements in order to interconnect these virtual services by means of virtual links or virtual networks.

As a result, SN4I provides sliced network, computing, and storage resources for the deployment of isolated concurrent services and experimentation. This deployment also enables the integration of NFV technologies with the industrial Internet protocols and the coexistence with other industrial networks like Time Sensitive Networks (TSN). Therefore, SN4I complements traditional research and activity in the CFAA field of manufacturing, with state-of-the-art technologies like high bandwidth, low delay, on-demand service creation, SDN-based security, etc. The integration of all these technologies with cutting-edge machine tools and manufacturing processes will allow the provision of a mixed R+D+I centre in advanced connected manufacturing technologies that will become a reference at international level.

This infrastructure capitalizes the knowledge gathered in our previous facility, the University of the Basque Country's OpenFlow Enabled Facility (EHU-OEF), and represents the evolution of the application of SDN and NFV technologies to a vertical use case (Industry 4.0), but also to CyberSecurity and Generic Network Services deployment.

Infrastructure Location Description

The SN4I infrastructure has a 10 Gbps (40 Gbps capable) backbone supported by OpenFlow switches under the control of our DynPaC service (SDN-based Bandwidth on Demand Service). Three distant sites (around 15 km between each of them) are linked through a layer-2 network. The layer-2 network uses VLAN encapsulation and different VLANs are assigned to different services in order to isolate them from each other. As a result , the SN4I infrastructure links the Faculty of Engineering of Bilbao (EIB), the University’s Rectorate (where the PoP of GEANT, RedIRIS and I2Basque, European, national and local NRENs are located) and the CFAA.

Computing Architecture

The whole architecture is orchestrated by Open Source MANO (OSM), an ETSI NFV based MANO. Our own Wide Area Manager (WIM) solution DynPaC (the first available WIM in OSM since v5.0.3.13) is used to provide high performance, resiliente on demand bandwidth). A separate, out-of-band layer-3 network also exists across the three locations for management purposes.

Every SN4I PoP is equipped with hardware and software components that enable the deployment of virtual services in the infrastructure using NFV and SDN technologies. In each of these three locations, an OpenStack node has been deployed as a Virtualized Infrastructure Manager (VIM). Each OpenStack node consists of several physical servers (three or four depending on the site). One of them acts as the controller with the corresponding controller services, while all of the servers have computing and storage capabilities. The OpenStack nodes are managed by OSM which allows the instantiation of Network Services which deploy in turn Virtual Machines and Virtual Links. The OpenStack nodes support state-of-the-art high performance Enhanced Platform Awareness (EPA) features, such as SR-IOV networking and Non-uniform memory access (NUMA) awareness, ensuring optimal performance and resource allocation.

Furthermore, each OpenStack node is interconnected by the means of an OpenFlow switch under the control of a local ONOS SDN Controller, which manages the connectivity between virtual services in the OpenStack node and in and out of it. This local SDN domain leverages OSM’s SDN Assist feature towards ONOS. SDN Assist allocates the networking resources necessary to interconnect the different components of virtual services into individual SDN flows. This is done automatically by OSM whenever a Network Service is deployed in the infrastructure.

Recently, SN4I has been updated to include Kubernetes-based VIMs in order to deploy containerized applications that better align with the low-latency requirements of edge computing and with the microservices paradigm. OSM can manage Kubernetes as a VIM, in a similar way to how it works with OpenStack. The inclusion of Kubernetes enables the deployment of Kubernetes Network Functions (KNFs) as well as the traditional VM-based Virtual Network Functions (VNFs) in order to build more powerful hybrid Network Services. Specialized MEC platforms such as OpenNESS are currently being tested and deployed at beta stage in consideration of being integrated into SN4I. Also, the infrastructure supports the Computing in the Network (CoIN) paradigm through the programmable data-planes using several P4 switches, that are deployed on specific areas.

In rough numbers, the whole setup includes around a dozen OpenFlow switches (including core, SDN Assist, and spare/experimentation), three P4 high-performance switches and more than 20 servers which account for more than 1TB of RAM and 230 cores, many of which are dedicated permanently to the corresponding OpenStack and Kubernetes VIMs.

4G/5G and WSN Architecture

As of now, 5G full plant coverage in the CFAA is available in the context of the project 5G-Euskadi. Until now eMBB over NSA services is available. and a fully operative 5G URLLC over SA service is expected to (by the end of this year), available which will fulfill the low latency and critical data requirements of the Industrial IoT scenario. Additionally. a 5G Non-Public Network (NPN) with experimental gNodeB provides 5G connectivity in specific areas of the CFAA plant. This setup allows both the integration of 5G sensors and 5G routers in the machine tools to provide added value to the manufacturing processes. Another experimental eNodeb base station provides NB-IoT and LTE-M on-demand coverage for constrained device sensors.

In addition, a Wireless Sensor Network (WSN) has been integrated into the SN4I architecture, enabling the deployment of several constrained device sensors in the CFAA in order to monitor different environmental parameters such as temperature, humidity, or lightness. The interconnection between the WSN and the wired SN4I is performed through a 6LoWPAN Border Router, which is implemented by means of a Raspberry Pi running the necessary protocol translation and packet routing functions, and several Constrained Device Sensor (CDS) using the IEEE 802.15.4 protocol. To mitigate security issues, the Hidra protocol is used, providing a strong and dynamic access control solution that can be implemented even in the most resource-constrained devices.

Additional Equipment

We have the equipment to stress, measure (bandwidth, delay, BER, setup time and TCP throughput), and exercise the network at speeds up to 100 Gbps. As aforementioned, we also have several P4-enabled 100 Gbps switches that enable the offloading of network functions to the data plane for latency-critical services.

Publications

SN4I is actively being used in several research lines and thus appears in several academic papers.

A first description (very high level because of the targeted audience) of the initial deployment of SN4I appears in:
• Towards a 5G compliant and flexible connected manufacturing facility

The inclusion of the WSN, the support for secure network slicing, and a broaching machine use case are described in:
• Towards Flexible Integration of 5G and IIoT Technologies in Industry 4.0: A Practical Use Case

The integration of SN4I in a Nation-wide deployed experimental 5G infrastructure, the description of a solution to provide reconfiguration of IoT devices and the corresponding performance evaluation are described in:
• A Multi-Site NFV Testbed for Experimentation With SUAV-Based 5G Vertical Services

The identity management of IoT devices in an Industrial application is a complex problem that is many times neglected. In this paper we propose a solution that is evaluated in SN4I:
• How to survive identity management in the Industry 4.0 era

This paper presents a solution for industrial IoT (IIoT) network communications to reduce response times using in-network computing through data plane programming and P4. Our solution processes Message Queuing Telemetry Transport (MQTT) packets sent by a sensor in the data plane and generates an alarm in case of exceeding a threshold in the measured value. The implementation has been tested in the SN4I experimental facility, using a Netronome SmartNIC as a P4 programmable network device. Response times are reduced by 74% while processing, and delay introduced by the P4 network processing is insignificant
• Achieving Low Latency Communications in Smart Industrial Networks with Programmable Data Planes

Other papers that take advantage of SN4I are in preparation and will be presented here...

Additional material

Instructions to request and setup access to SN4I via VPN

Hands-on Open Source MANO (currently in spanish, to be translated soon!)

5G Euskadi demo. SN4I in the industrial process control and monitoring use case.