lighty.io Archives - lighty.io

[RESTCONF API] Topology Interaction

lighty.io — Thu, 23 Jan 2020 12:58:28 +0000

WHAT IS AN API REQUEST?

REST API request is an HTTP method. It lets you send a request from an endpoint to your server (controller). You can use it if you want to retrieve some information from the server or connected device, or if you want to do some changes (for example to datastore, topology, device, etc..). The most common HTTP methods are GET, PUT, POST or DELETE.

GET – used to read data from a server or connected device, when GET is successful you will get 200 OK response
PUT – used to create/update data in a server or connected device, when PUT is successful you should get either 201 Created(if something new was created) or 204 No content(if something was just modified) response
POST – used to send data to a server or connected device ( file upload, customer information, calling RPC, etc… ), when POST is successful you will get 200 OK response
DELETE – used to remove (delete) data from a server or connected device, when DELETE is successful you will get 204 No content response

PARTS OF REQUESTS

Default settings for the http://:/ are http://localhost:8888/, but for the purpose of this example, we will use : where ip_address and port are dependent on your running configuration.

The four parts are:

IP Adress & Port Nr.
Adding RESTCONF/Identifier
RESTCONF API Path
Controller Module Path

1. Part of every RESTCONF API request is an IP address and port number, on which our requested server is running:

http://:/ – ip_address, in this case, is IP address of the server we are sending a request to. port is the port number on which the RESTCONF plugin is running.

2. Add RESTCONF/identifier, because we are making requests to it (as described in the RFC 8040):

restconf/ – is the identifier that we are making a request to via the RESTCONF API

3. Path to either data or operations available through RESTCONF API

data (restconf/data/) – this leads to all data that can be requested, updated or deleted(if those are config data) from the controller datastore
operations (restconf/operations) – this contains all operations(called RPCs – Remote Procedure Calls) that can be invoked by the user via the RESTCONF interface

4. A path into the desired module in the controller:

unified path – restconf/data/module_name:container/ – where module_name is the name of the YANG module and container is the container in the particular YANG module
For example, we can make a request – restconf/data/network-topology:network-topology/ – which leads to a subtree of data store that is specified as network-topology. It basically returns a network-topology container with all its content
query parameters (network-topology:network-topology?content=nonconfig) – this is the query that returns data from path into module and container and in this case with query parameter content with value nonconfig that indicates it will return data from path stored only in nonconfig datastore(operational)

You can use zero or more query parameters in the request URI. For example:

content (in GET) – to select config or non-config data(stored in configurational or operational data store)
depth (in GET) – limited subtree depth in the reply content
fields (in GET) – a request for a subset of target
filter (in GET) – boolean notification filter for event stream
insert (in POST & PUT) – insertion mode for data resources
with-defaults (in GET) – control the retrieval of default values

If you need more information, take a look at RFC 8040.

RESPONSE VALUES OF REQUESTS:

Every request that was made via the REST API, returns a value with return code that indicates success or various types of errors for the request.

There are 5 return code classes:

1xx – informational character and indicates for example that request was received and the user should wait for the process, etc.
2xx – indicates that request was received, understood and accepted, for example, 200 OK or 201 Created, etc.
3xx – indicates that additional action has to be performed to finish the request, used mostly in redirections
4xx – indicates an error that might be caused on the client-side, for example, 400 Bad Request, 403 Forbidden, 404 Not Found, etc.
5xx – indicates an error that occurred on the server or that server cannot complete the request for some reason, for example, 500 Internal Server Error, 503 Service Unavailable, etc.

If these codes are insufficient because, for example, the error needs to be described more specifically, then the error message is usually more described and expanded in the return value of the request for example like this:

//Error Response Body
{
    "error": "err-login-01",
    "message": "Login failed",
    "detail": "Check your username and password"
}

EXAMPLES OF SOME REQUESTS FOR POSTMAN:

Default settings for the http://:/ are http://localhost:8888/, but for example purposes we will use : where ip_address and port are dependent on your running configuration.

Here are models that are used in this example, custom-yang-rpcs for simple-sum-rpc and network-topology for operations with network topologies and nodes.

custom-yang-rpcs@2019-11-14.yang

network-topology@2013-10-21.yang

To try some requests you can start our lighty.io example application and example topology (in mininet). An easy way to connect a controller via an HTTP request is using Postman. Here we show you some examples of how to get information from the controller:

to receive the operations(RPCs) that are available in the RESTCONF:

//GET
http://:/restconf/operations

here is the example of the part of the received list of available RESTCONF operations(RPCs) in JSON format :

//GET Response Body
{
    "ietf-restconf:operations": {
        "custom-yang-rpcs:simple-sum-rpc": [
            null
        ],
        "custom-yang-rpcs:custom-netconf-dev-connect": [
            null
        ],
...

To receive a list of available YANG models loaded into the controller

//GET
http://:/restconf/data/ietf-yang-library:modules-state

Here is the example of the part of the received list of YANG models loaded into the controller with its attributes like name, revision, namespace etc.

//GET Response Body
{
    "ietf-yang-library:modules-state": {
        "module": [
            {
                "name": "custom-yang-rpcs",
                "revision": "2019-11-14",
                "namespace": "tag:lighty.io.2019:lighty:custom:yang:rpcs",
                "conformance-type": "import",
                "schema": "/modules/custom-yang-rpcs/2019-11-14"
            },
...

To receive information about our topology

//GET
http://:/restconf/data/network-topology:network-topology?content=nonconfig

If you want to add new topology or node:

To create a new topology with topology-id test-topology. You need to write a body for this method. It is a JSON format with information about a list of network-topology container (topology) and key for this list (test-topology) that has to be unique:

//PUT
http://:/restconf/data/network-topology:network-topology/topology=test-topology

JSON body to PUT network-topology with id test-topology

//PUT Body
{
    "topology":
    {
        "topology-id: "test-topology"
    }
}

Here is the network-topology container in the datastore after a successful PUT request:

//GET Response Body
"network-topology:network-topology": {
    "topology": [
        {
            "topology-id: "test-topology"
        }
    ]
}

Alternatively, we can create the topologies by inserting whole list of topologies as JSON array in the network-topology container, like this(there can be more then three topologies), but it will rewrite whole container, that means every topology that was in there will be erased, so its best for complete lists or empty topologies:

//PUT
http://:/restconf/data/network-topology:network-topology

JSON body to PUT list of topologies

//PUT Body
{
    "network-topology:network-topology": {
        "topology": [
            {
                "topology-id": "test-topology-element1"
            },
            {
                "topology-id": "test-topology-element2"
            },
            {
                "topology-id": "test-topology-element3"
            }
        ]
    }
}

Here is the network-topology container in the datastore after successful PUT request:

//GET Response Body
"network-topology:network-topology": {
    "topology": [
        {
            "topology-id": "test-topology-element1"
        },
        {
            "topology-id": "test-topology-element2"
        },
        {
            "topology-id": "test-topology-element3"
        }
    ]
}

To create a new node in test-topology, you need to write a body for this method. Similarly to the method before, it is a JSON format with information about a topology list (node) and key for this list (test-node):

//PUT
http://:/restconf/data/network-topology:network-topology/topology=test-topology/node=test-node

JSON body to PUT node test-node into network-topology test-topology (or we can do it with the array of nodes just like in the previous example with the topologies )

//PUT Body
{
    "node": {
        "node-id": "test-node"
    }
}

If you want to delete a topology or node (like before it needs to specify the path to the particular node we want to delete):

//DELETE
http://:/restconf/data/network-topology:network-topology/topology=test-topology/node=test-node

And almost the same as with node, we just specify the path to the topology for deleting:

//DELETE
http://:/restconf/data/network-topology:network-topology/topology=test-topology

You might also invoke RPC that is implemented or you have implemented so when you will register it to lighty controller it will appear in the RESTCONF operations and you can invoke it like this:

I will make an example on custom RPC that takes two input values, sums them up and returns summed value, which is also stored in datastore. You can invoke RPC via POST request like this, you need to specify a path to RPC:

//POST
http://:/restconf/operations/custom-yang-rpcs:simple-sum-rpc

and also JSON body, if RPC has some input values:

//POST RPC Body
{   
    "input": {
        "input-a": 100,
        "input-b": 200
    }
}

It returns output value:

//POST RPC Output Body
{
    "custom-yang-rpcs:output": {
        "output-c": 300
    }
}

SUMMARY

In this blog post, we covered what are RESTCONF API requests and their basic principles, with a few examples of how to use them. Let’s do a quick summary:

what is RESTCONF API request and what is it used for(to retrieve or write information from/to server)
what types of requests RESTCONF API offers and what are the most common ones(GET, PUT, POST, DELETE)
how can we put together requests based on the models we use(IP address, port, path, query parameters, etc…)
that requests have response values which indicate a type of success or failure of request(server fault, user fault, informational character, etc…)
examples of GET requests and how GET request return value body can look like
examples of PUT request, how can we write data to the server with it and how it’s using JSON as input to particular requests(container, list, etc…)
examples of DELETE, how can we delete something from a server(in the example – topologies, nodes, etc…)
example of how can we invoke RPC via a request with input parameters and then what does the output from RPC look like

Those are the basic information that should help you to be able to start using RESTCONF API requests on your own – basic understanding of how it works, what are they used for and how you can put together your own requests along with where to look for troubleshooting in case of failure.

The post [RESTCONF API] Topology Interaction appeared first on lighty.io.

Integrate OpenDaylight Modules into lighty.io

lighty.io — Tue, 14 Jan 2020 12:34:57 +0000

Introduction

Not every OpenDaylight project is available in lighty.io. Since we are not dependent on the OSGi framework (Karaf), we can’t simply add and start project features from the Karaf command line.

Here is a list of all OpenDaylight Projects. Before starting the integration, make sure that your project is not already integrated within lighty.io.

Here is the list of already integrated modules. In this article, we will use the OpenFlow project as an example of how to integrate the new project to lighty.io.

We have already integrated the OpenFlow project into lighty.io – it is available under lighty-modules/lighty-openflow-sb.

Create a new project

Create a new Java project for your module. In this example, we will be using Maven.

Create a new lighty.io module

Create a new Java class, which will represent new lighty.io module. In our example, we will create the class OpenflowSouthboundPlugin , which corresponds with the OpenFlow module.

lighty.io provides an abstract AbstractLightyModule class, containing methods to start and shut down a module. By extending the AbstractLightyModule, you only need to implement two abstract methods:

initProcedure

stopProcedure

to initialize/stop services & beans in your module. Additionall,y we recommend to use Builder pattern to create instances of your new module, in case your module requires a larger amount of input parameters.

Initialize Beans & Services

This section will give you hints on how to find all beans and services from the original project, that you may need to start in your new lighty.io module.

Clone OpenDaylight Project

Clone the project from opendaylight-gerrit repository. Check out the version you would like to integrate.

Find all necessary Blueprint files

Check all project features for blueprint files. Usually, you’re looking for the autowire.xml file, which is located in the project:

target/generated-sources/blueprint/OSGI-INF.blueprint/autowire.xml

This file is generated during the build from the annotations and blueprint .xml files. In the project are blueprint files usually located in:

src/main/resources/OSGI-INF.blueprint

Here is the example of the interfacemanager-impl feature autowire.xml file from the genius module project.

As you can see, it contains 3 beans, where IfmDiagStatusProvider is a service. We need to create them within our lighty.io module project. Bean AlivenessMonitorListenerImpl has init and destroy methods, which we need to call as well – usually in the initProcedure and stopProcedure methods.

Sometimes, beans require classes from other projects. Always check beforehand, if the required class isn’t already being created in one of the present lighty.io modules. If it’s there, don’t create it in your module – just pass it to the module as a dependency. Some beans are also created by the builder pattern, which needs to set up class attributes to specific values. Check if the project has a resource file under src/main/resources/initial/config-file.xml, which contains the default values.

Create Beans & initProcedure

In the initProcedure, we are going to initialize all necessary classes, which are in the blueprints. Here is a snippet of the OpenFlow plugin method:

protected boolean initProcedure() {
    ForwardingPingPongDataBroker forwardingPingPongDataBroker =
          new ForwardingPingPongDataBroker(lightyServices.getBindingDataBroker());
    SwitchConnectionProviderList switchConnectionProviders = new SwitchConnectionProviderList(providers);
	this.mastershipChangeServiceManager = new MastershipChangeServiceManagerImpl();
    final OpenflowDiagStatusProvider diagStat = new OpenflowDiagStatusProviderImpl(this.lightyServices
          .getDiagStatusService());
    this.openFlowPluginProvider = new OpenFlowPluginProviderImpl(
          this.configurationService,
          switchConnectionProviders,
          forwardingPingPongDataBroker,
          this.lightyServices.getRpcProviderService(),
          this.lightyServices.getBindingNotificationPublishService(),
          this.lightyServices.getClusterSingletonServiceProvider(),
          this.lightyServices.getEntityOwnershipService(),
          this.mastershipChangeServiceManager, diagStat,
          this.lightyServices.getSystemReadyMonitor());
    this.openFlowPluginProvider.initialize();
...

We are creating 2 classes from the OpenFlow blueprints directly in the initProcedure. The OpenFlowPluginProviderImpl class requires some dependencies, which are provided by lightyServices – located in the lighty controller module. We are also calling the initialize method on the bean as well.

Close Beans in the stopProcedure

In stopProcedure, we are going to call the destroy-methods on all beans, which have it described in the blueprints.

@Override
protected boolean stopProcedure() {
    destroy(this.packetListenerNotificationRegistration);
    destroy(this.flowCapableTopologyProvider);
    destroy(this.operationProcessor);
    destroy(this.forwardingRulesManagerImpl);
    destroy(this.arbitratorReconciliationManager);
    destroy(this.openFlowPluginProvider);
    destroy(this.mastershipChangeServiceManager);
    destroy(this.terminationPointChangeListener);
    destroy(this.nodeChangeListener);
    return true;
}

Runnable application with the new module

A lighty.io application is basically just a plain old Java main method, that will run your module and all necessary dependencies. In order to create an application for your module, simply create and start all the necessary dependencies in the main method first, and then create and start your module.

We are now going to show you, how we have made the example OpenFlow application. First step is to get the configuration of the modules:


final Set modelPaths = Stream.concat(RestConfConfigUtils.YANG_MODELS.stream(),
	OpenflowConfigUtils.OFP_MODELS.stream())
	.collect(Collectors.toSet());
controllerConfiguration = ControllerConfigUtils.getDefaultSingleNodeConfiguration(modelPaths);
restConfConfiguration =  RestConfConfigUtils.getDefaultRestConfConfiguration();
openflowpluginConfiguration = OpenflowConfigUtils.getDefaultOfpConfiguration();

After we have the required configuration, we can start the modules one after the other.

//1. initialize and start Lighty controller (MD-SAL, Controller, YangTools, Akka)
final LightyControllerBuilder lightyControllerBuilder = new LightyControllerBuilder();
final LightyController lightyController = lightyControllerBuilder.from(controllerConfiguration).build();
lightyController.start().get();
//2. start RESTCONF server
final CommunityRestConfBuilder communityRestConfBuilder = new CommunityRestConfBuilder();
final CommunityRestConf communityRestConf = communityRestConfBuilder.from(RestConfConfigUtils
	.getRestConfConfiguration(restConfConfiguration, lightyController.getServices()))
	.build();
communityRestConf.start();
//3. start OpenFlow SBP
final OpenflowSouthboundPlugin plugin;
plugin = new OpenflowSouthboundPluginBuilder()
	.from(configuration, lightyController.getServices())
	.withPacketListener(new PacketInListener())
	.build();
ListenableFuture start = plugin.start();

Check out the example applications in lighty.io or other articles, for more information.

Summary

In this article, we have described the way to include other OpenDaylight projects, which are not integrated into our lighty.io. Steps of integrating the new module were:

Creating new project for the module
Initializing module classes from blueprints
Creating the runnable application in which a new module started

Don’t forget, some classes require dependencies from another module and its recommended to use existing implementations if they are available in lighty.io.

by Peter Válka

The post Integrate OpenDaylight Modules into lighty.io appeared first on lighty.io.

Spring.io Integration

lighty.io — Fri, 08 Jun 2018 09:25:37 +0000

lighty.io has been enabling OpenDaylight (ODL) components and applications to run in various environments and frameworks.

Those components and applications have never been useable by ODL developers before. Spring.io is a popular java ecosystem for application developers. Wouldn’t it be perfect to enable lighty.io for Spring as the target runtime environment is Java SE? This is exactly what we did.

lighty.io provides dependency injection extensions for Spring. That makes it easy to consume ODL core services via Spring’s dependency injection system. Let’s check how simple it is to create an OpenDaylight based NETCONF SDN controller application in the spring-boot environment.

The architecture of this example is in the picture.

Generate Spring Boot Skeleton Project

First, generate the Maven Spring Boot skeleton application with start.spring.io. In this guide, we will use Spring Boot release 2.0.2.

For now, we will need only standard web dependency, as we want to expose some REST endpoints.

Let’s use inputs as follows:

group id: io.lighty.core
artifact id: lighty-controller-springboot
name: lighty-controller-springboot
description: Demo lighty.io project for Spring Boot
package: io.lighty.core.controller.springboot
packaging: jar
java version: 8

Start.spring.io generates for us a basic maven project hierarchy with an application entry point in class LightyControllerSpringbootApplication.

lighty.io Project Setup

Next, we need to start lighty.io inside Spring, so just simply add lighty.io dependency injection extension for Spring to pom.xml.

This artifact is a bean provider for Spring’s @Configuration.

 	 	 	 	8.0.0-SNAPSHOT

    io.lighty.core
    lighty-controller-spring-di
    ${lighty.version}

And to use the default single node lighty.io configuration add a dependency


    io.lighty.resources
    singlenode-configuration
    ${lighty.version}

For the NETCONF devices, we will need also the NETCONF Southbound module


    io.lighty.modules.southbound.netconf
    lighty-netconf-sb
    ${lighty.version}

Since now we have lighty-core on the classpath, let’s create lighty.io bean definitions for Spring.

Create new @Configuration class LightyConfiguration extending LightyCoreSpringConfiguration which initializes all the lighty.io services as Spring beans. To be able to initialize all lighty.io services it is necessary to create lighty.io core controller as Spring bean, for example in class LightyConfiguration. In this example, we need to initialize LightyController and NETCONF Southbound Plugin too.

@Configuration
public class LightyConfiguration extends LightyCoreSpringConfiguration {
    private static final Logger LOG = LoggerFactory.getLogger(LightyConfiguration.class);
    @Bean
    LightyController initLightyController() throws Exception {
        LOG.info("Building LightyController Core");
        final LightyControllerBuilder lightyControllerBuilder = new LightyControllerBuilder();
        final Set mavenModelPaths = new HashSet<>();
        mavenModelPaths.addAll(NetconfConfigUtils.NETCONF_TOPOLOGY_MODELS);
        mavenModelPaths.add($YangModuleInfoImpl.getInstance()); //Add toaster model
        final LightyController lightyController = lightyControllerBuilder
            .from(ControllerConfigUtils.getDefaultSingleNodeConfiguration(mavenModelPaths))
            .build();
        LOG.info("Starting LightyController (waiting 10s after start)");
        final ListenableFuture started = lightyController.start();
        started.get();
        LOG.info("LightyController Core started");
        return lightyController;
    }
    @Bean
    NetconfSBPlugin initNetconfSBP(LightyController lightyController) throws ExecutionException, InterruptedException {
        final NetconfConfiguration netconfSBPConfiguration = NetconfConfigUtils.injectServicesToTopologyConfig(
            NetconfConfigUtils.createDefaultNetconfConfiguration(), lightyController.getServices());
        NetconfTopologyPluginBuilder netconfSBPBuilder = new NetconfTopologyPluginBuilder();
        final NetconfSBPlugin netconfSouthboundPlugin = netconfSBPBuilder
            .from(netconfSBPConfiguration, lightyController.getServices())
            .build();
        netconfSouthboundPlugin.start().get();
        return netconfSouthboundPlugin;
    }
}

From this point, we have created bean definitions for the Spring environment. Let’s add some basic logic.

Create a Basic lighty.io REST Service

In this step, we will create a simple REST service, exposing endpoints for an example of managing NETCONF devices.

Let’s add endpoints which provide us functionality to:

connect to a device
list all devices
disconnect device

Simply create a @RestController class skeleton for REST endpoints returning some default responses.

@RestController
@RequestMapping(path = "netconf")
public class NetconfDeviceRestService {
    @GetMapping(path = "/list")
    public ResponseEntity getNetconfDevicesIds() {
        // TODO add endpoin impl
        return ResponseEntity.noContent().build();
    }
    @PutMapping(path = "/id/{netconfDeviceId}")
    public ResponseEntity connectNetconfDevice(@PathVariable final String netconfDeviceId,
                                               @RequestBody final NetconfDeviceRequest deviceInfo) {
        // TODO add endpoin impl
        return ResponseEntity.noContent().build();
    }
    @DeleteMapping(path = "/id/{netconfDeviceId}")
    public ResponseEntity disconnectNetconfDevice(@PathVariable final String netconfDeviceId) {
        // TODO add endpoin impl
        return ResponseEntity.noContent().build();
    }
}

GET /netconf/list will accept no attributes and returns all stored NETCONF devices
PUT /netconf/id/{netconfDeviceId} will need device ID and device details needed to connect contained in the request body
DELETE /netconf/id/{netconfDeviceId} will disconnect device and removed it from datastore

All lighty.io core services are already defined as beans, so we can inject DataBroker and MountPointService into our REST Service class NetconfDeviceRestService, to provide access to OpenDaylight’s global data store.

@Autowired
@Qualifier("BindingDataBroker")
private DataBroker dataBroker;
@Autowired
private MountPointService mountPointService;

Now we can write and read from datastore so let’s update endpoints implementations in class NetconfDeviceRestService:

private static final InstanceIdentifier NETCONF_TOPOLOGY_IID = InstanceIdentifier
        .create(NetworkTopology.class)
        .child(Topology.class, new TopologyKey(new TopologyId("topology-netconf")));
private static final long TIMEOUT = 1;
@GetMapping(path = "/list")
public ResponseEntity getNetconfDevicesIds() throws InterruptedException, ExecutionException, TimeoutException {
    try (final ReadOnlyTransaction tx = dataBroker.newReadOnlyTransaction()) {
        final Optional netconfTopoOptional =
            tx.read(LogicalDatastoreType.OPERATIONAL, NETCONF_TOPOLOGY_IID).get(TIMEOUT, TimeUnit.SECONDS);
        if (netconfTopoOptional.isPresent() && netconfTopoOptional.get().getNode() != null) {
            final List response = new ArrayList<>();
            for (Node node : netconfTopoOptional.get().getNode()) {
                final NetconfDeviceResponse nodeResponse = NetconfDeviceResponse.from(node);
                response.add(nodeResponse);
                final Optional netconfMountPoint =
                    mountPointService.getMountPoint(
                        NETCONF_TOPOLOGY_IID.child(Node.class, new NodeKey(node.getNodeId()))
                    );
                if (netconfMountPoint.isPresent()) {
                    final Optional netconfDataBroker =
                        netconfMountPoint.get().getService(DataBroker.class);
                    if (netconfDataBroker.isPresent()) {
                        final ReadOnlyTransaction netconfReadTx =
                            netconfDataBroker.get().newReadOnlyTransaction();
                        final Optional toasterData = netconfReadTx
                            .read(LogicalDatastoreType.OPERATIONAL, TOASTER_IID).get(TIMEOUT, TimeUnit.SECONDS);
                        if (toasterData.isPresent() && toasterData.get().getDarknessFactor() != null) {
                            nodeResponse.setDarknessFactor(toasterData.get().getDarknessFactor());
                        }
                    }
                }
            }
            return ResponseEntity.ok(response);
        } else {
            return ResponseEntity.notFound().build();
        }
    }
}
@PutMapping(path = "/id/{netconfDeviceId}")
public ResponseEntity connectNetconfDevice(@PathVariable final String netconfDeviceId,
                                           @RequestBody final NetconfDeviceRequest deviceInfo)
    throws InterruptedException, ExecutionException, TimeoutException {
    final WriteTransaction tx = dataBroker.newWriteOnlyTransaction();
    final NodeId nodeId = new NodeId(netconfDeviceId);
    final InstanceIdentifier netconfDeviceIID = NETCONF_TOPOLOGY_IID
        .child(Node.class, new NodeKey(nodeId));
    final Node netconfDeviceData = new NodeBuilder()
        .setNodeId(nodeId)
        .addAugmentation(NetconfNode.class, new NetconfNodeBuilder()
            .setHost(new Host(new IpAddress(new Ipv4Address(deviceInfo.getAddress()))))
            .setPort(new PortNumber(deviceInfo.getPort()))
            .setCredentials(new LoginPasswordBuilder()
                    .setUsername(deviceInfo.getUsername())
                    .setPassword(deviceInfo.getPassword())
                    .build())
            .setTcpOnly(false)
            .build())
        .build();
    tx.put(LogicalDatastoreType.CONFIGURATION, netconfDeviceIID, netconfDeviceData);
    tx.submit().get(TIMEOUT, TimeUnit.SECONDS);
    return ResponseEntity.ok().build();
}
@DeleteMapping(path = "/id/{netconfDeviceId}")
public ResponseEntity disconnectNetconfDevice(@PathVariable final String netconfDeviceId)
    throws InterruptedException, ExecutionException, TimeoutException {
    final WriteTransaction tx = dataBroker.newWriteOnlyTransaction();
    final NodeId nodeId = new NodeId(netconfDeviceId);
    final InstanceIdentifier netconfDeviceIID = NETCONF_TOPOLOGY_IID
        .child(Node.class, new NodeKey(nodeId));
    tx.delete(LogicalDatastoreType.CONFIGURATION, netconfDeviceIID);
    tx.submit().get(TIMEOUT, TimeUnit.SECONDS);
    return ResponseEntity.ok().build();
}

Be sure you added the device yang model to PROJECT_WORKING_DIR/cache/schema/toaster@2009-11-20.yang (toaster@2009-11-20.yang).

Read Data from the NETCONF Device

Test the application through created REST endpoints:

start the Spring application
connect to NETCONF device with ID testDevice

curl -X PUT 
  http://localhost:8080/netconf/id/testDevice 
  -H 'Accept: application/json' 
  -H 'Content-Type: application/json' 
  -d '{
    "username": "admin",
    "password": "admin",
    "address": "127.0.0.1",
    "port": "17830"
}'

list all NETCONF devices

curl -X GET 
  http://localhost:8080/netconf/list 
  -H 'Accept: application/json' 
  -H 'Content-Type: application/json'

if the connection was successful, the response will be

[
    {
        "nodeId": "testDevice",
        "connectionStatus": "Connected",
        "darknessFactor": 1000
    }
]

Notice that the darknessFactor was read from the NETCONF device. Our testing device supports model toaster-model from OpenDaylight so we were able to read the value defined in the model through MountPoint.

Michal Baník

The post Spring.io Integration appeared first on lighty.io.

lighty.io Authentication Authorization & Accounting

lighty.io — Thu, 07 Jun 2018 10:19:34 +0000

Do you need an SDN controller that you can run quickly?
Is security one of your concerns?
Do you need to divide the users into groups & restrict their capabilities for certain tasks?

With lighty.io AAA, you can create password-protected users. You can assign a role and a domain and define its capabilities. You can restrict users to a certain method or permit them to access data only on a specific path.

Use this dependency to give lighty.io AAA a try:


   io.lighty.aaa
   lighty-aaa
   8.0.0-SNAPSHOT

To start this module, simply create a lighty AAA object and call a start method. You can use the shutdown method to stop this module. Once the lighty controller & RestConf with lighty AAA are up and running, you may start to create new users, roles, and domains.

You can assign roles to the users, create policies for, and grant permissions to the users.

How to create a user

Use this POST request to create a user:

path: /auth/v1/users

with body:

{
    "name": "u1",
    "description": "just another new user",
    "enabled": 1,
    "email": "u1@sdn.tech",
    "password": "foo",
    "salt": "foo",
    "domainid": "sdn"
}

lighty will automatically generate a user id for you and save it into the database. The expected response should be “201 Created“:

{
    "userid": "u1@sdn",
    "name": "u1",
    "description": "just another new user",
    "enabled": 1,
    "email": "u1@sdn.tech",
    "password": "**********",
    "salt": "**********",
    "domainid": "sdn"
}

How to list users

This GET request should bring up a list of users with their information:

path: /auth/v1/users

Now we expect a response with a list of users with their information, “200 OK”.

{
    "users": [
        {
            "userid": "admin@sdn",
            "name": "admin",
            "description": "admin user",
            "enabled": 1,
            "email": "",
            "password": "**********",
            "salt": "**********",
            "domainid": "sdn"
        },
        {
            "userid": "u1@sdn",
            "name": "u1",
            "description": "just another new user",
            "enabled": 1,
            "email": "u1@sdn.tech",
            "password": "**********",
            "salt": "**********",
            "domainid": "sdn"
        }
    ]
}

If we use the credentials of this newly created user, we would receive the response ‘unauthorized‘. To resolve this issue, we need to create a “grant” to the user. This means that we have to assign a role to the user.

How to create a “grant”

Use this POST request to create a grant:

path: /auth/v1/domains//users//roles

with the body:

{
    "roleid": ""
}

lighty will process this request, by writing new data to grant a table of the database with specific user-id and role-id under the chosen domain. The expected response should be the following: 201 Created

{
    "grantid": "@@",
    "domainid": "",
    "userid": "",
    "roleid": ""
}

How to list roles

This GET request will bring up a list of roles with their information.

path: /auth/v1/roles

Now we expect a response with a list of roles with their information, “200 OK”.

{
    "roles": [
        {
            "roleid": "admin@sdn",
            "name": "admin",
            "description": "a role for admins",
            "domainid": "sdn"
        },
        {
            "roleid": "user@sdn",
            "name": "user",
            "description": "a role for users",
            "domainid": "sdn"
        }
    ]
}

This is perfect if we need to divide our users into an admin role which would have the capability to create, remove, and update users. The user role would have the capability to do operations on datastore.

But what if we need to create more roles? Such as a role where the user can only read data or a role where the user can do operations only on a specific path. To do that, we need to follow these two steps:

1: Create a new role
2: Define a policy for the role

How to create a role

Use this POST request to create a role:´

path: /auth/v1/roles

with body:

{
    "name" : "read-only",
    "description": "This role is for users that have read only rights",
    "domainid": "sdn"
}

Lighty will write this role to the roles table in the database so it can be used to assign this role to the user or to update policy for this role. When creating a role we expect the following response: 201 Created.

{
    "roleid": "read-only@sdn",
    "name": "read-only",
    "description": "This role is for users that have read only rights",
    "domainid": "sdn"
}

How to define a policy for a specific role

The meaning of defining a policy to a role is to manage what the given role can do. By defining a policy you can restrict a role; to use specific HTTP methods or access data only under a specific list or container.

Create a POST request to define a policy:

path: /restconf/data/aaa:http-authorization/policies

with body:

{
    "policies": [
      {
        "resource": "/restconf/modules/**",
        "permissions": [
          {
            "role": "read-only",
            "actions": [
             "get"
            ]
          }
        ],
        "description": "read only policy on restconf/modules list"
      }
    ]
}

After this request, all the users who have assigned the role of “read-only”, will be able to do only GET request on path /restconf/modules and what “modules” contains.

How to list policies

Use the following GET request to see all the policies which are created:
path: /restconf/data/aaa:http-authorization/policies

Now we expect a response with a list of policies with their information:
200 OK

{
    "policies": {
        "policies": [
            {
                "resource": "/restconf/modules/**",
                "permissions": [
                    {
                        "role": "read-only",
                        "actions": [
                            "get"
                        ]
                    }
                ],
                "description": "read only policy on restconf/modules list"
            },
            {
                "resource": "/restconf/**",
                "permissions": [
                    {
                        "role": "users",
                        "actions": [
                            "get"
                        ]
                    }
                ],
                "description": "read only policy on restconf list"
            }
        ]
    }
}

How to create a domain

Use the following POST request to create a new domain:

path: /auth/v1/domains

with body:

{
    "name" : "",
    "description": "",
    "enabled": true
}

Lighty AAA will create a domain id and this domain will be written into the domains table of the database. The expected response should be the following:

{
    "domainid": "",
    "name": "",
    "description": "",
    "enabled": true
}

How to list domains

We can use the following GET request to see all the domains that are created:

path: /auth/v1/domains

Now we expect a response with a list of domains with their information:
200 OK

{
    "domains": [
        {
            "domainid": "",
            "name": "",
            "description": "",
            "enabled": true
        },
        {
            "domainid": "",
            "name": "",
            "description": "",
            "enabled": false
        }
    ]
}

How to authenticate

There are two ways to authenticate the user:

1: HTTP basic authentication
2: oauth2 to generate tokens

Method 1: HTTP basic authentication

For this, we need a username and password separated by column encrypted using base64 encryption. The resulting encryption can be used as a header with key “Authorization” and value “Basic”.

Method 2: Oauth2 authentication

The second option is to generate a Bearer token. To generate a token, create a POST request:

path: /oauth2/token
Content-Type : application/x-www-form-urlencoded

with body:

grant_type=password&username=&pard=&scope=

Now, you should be ready to use Authentication, Authorization & Accounting with lighty.io.

The post lighty.io Authentication Authorization & Accounting appeared first on lighty.io.

Rapid NETCONF Controller Integration Testing

lighty.io — Fri, 04 May 2018 11:02:46 +0000

Full NETCONF/RESTCONF controller integration test under 12s!

Can your SDN controller do that?

With lighty.io you can. Just like in a high-performance sports car world, the drag race is used to set winners and losers apart.

Faster is better, it is as simple as that. In SDN business, you have to test and re-test really hard before you deploy or deliver to the customer. High-performance integration testing is essential for cutting down the development costs and time-to-market to a minimum but keeping the high quality of the product.

lighty.io utilizes OpenDaylight components and adds agility and flexibility beyond the upstream OpenDaylight convention. Continuous integration, delivery, and deployment, known collectively as CI/CD, is an integral part of modern development intended to reduce errors during integration and deployment while increasing project velocity.

CI/CD is a philosophy and set of practices often augmented by robust tooling that emphasize automated testing at each stage of the software pipeline. CI/CD pipelines help shepherd changes through automated testing cycles, out to staging environments, and finally to production.

The more comprehensive your testing pipelines are, the greater assurance you have that changes won’t introduce unforeseen side effects into your production deployment.

Let’s take a look at how lighty.io achieves high-performance integration testing.

lighty.io components used in this test are:

lighty.io based SDN controller. RESTCONF on the northbound interface, NETCONF on the southbound interface.
lighty.io NETCONF toaster device simulator (Toaster device simulator is based on lighty.io NETCONF device library and internally uses the famous toaster model)
lighty.io NETCONF topology device simulator. (Network device simulator is based on lighty.io NETCONF device library and internally uses network-topology model)
lighty.io RESTCONF java client.

Test Architecture

As you can see, this integration test can start all 3 layers in a single JVM instance. This is possible because lighty.io provides not only components for the SDN controller (layer 2), but also RESTCONF client (layer 1) and NETCONF device simulators (layer 3). YANG models are shared between all layers to achieve data type safety. In fact, this test will start a small test network and exercise all layers involved using different ports on the localhost.

The mission of this test exercise will be

to start the SDN controller.
start the first NETCONF test device (Toaster).
start the second NETCONF test device (Topology Device).
connect both devices to the SDN controller.
connect java RESTCONF client to SDN controller.
get data from device models, listen for device notifications, call RPCs on devices.
evaluate test data.
shutdown this test setup.

JUnit test design

As all components involved in this test are written in java, we can start complete network simulation in a single unit test.

The advantage is, that the unit test logic has access to the internal APIs of all 3 layers. This means quite complex tests are very easy to perform. In fact, this test does not require any mocked components. All components involved are used in real deployment as well.

JUnit test initialization

Initializes test network consisting of the SDN controller and two NETCONF test devices.
Waits until all components are up and running.
Connect REST client to SDN controller using RESTCONF and HTTP2.
Connect Toaster and Topology device to SDN controller.
Duration ~10s

JUnit test execution

testRestClientCreateSubscriptionAll duration: ~0.3s
This is the RESTCONF notification test. The test subscribes to a notification on the Topology NETCONF device and triggers them via an RPC call.
Sequence diagram of testRestClientCreateSubscriptionAll
testToasterReadDeleteDatastoreOperations: ~0.1s
This test uses datastore operation of Toaster device in order to read and delete
Sequence diagram of testToasterReadDeleteDatastoreOperations
testTopologyDeviceRPCInputOutput: ~0.6s
This test calls RPCs on topology device.
Sequence diagram of testTopologyDeviceRPCInputOutput

JUnit test shutdown

Stop all components and wait until stopped.

Duration: ~0.08s
Total test run: ~12s
Hardware configuration: Intel i7-4810MQ CPU, 16GB DDR3 RAM, JDK 8, Ubuntu 18.04 LTS

Links to project source code:

Speed and flexibility matter. Lighty.io offers a set of components to achieve record-breaking speeds in integration testing and continuous integration. A

fter moving from upstream OpenDaylight to lighty.io, our projects are compiled and tested in seconds or minutes, rather than hours. How fast can your test system go from starting a small test network to running tests and full stop?

If you are interested, please register here for regular updates, click on the Contact Us button on the menu of this page.

Juraj Veverka

The post Rapid NETCONF Controller Integration Testing appeared first on lighty.io.

ONAP SDN-C

lighty.io — Thu, 29 Mar 2018 09:42:38 +0000

In the contemporary SDN world, ONAP (Open Network Automation Platform) is quite a trend. ONAP is a big project composed of a number of sub-projects (or components if you like), which together form a platform for orchestration and automation of network functions. There are big companies involved in ONAP and its development is speeding up rapidly.

SDN-C is one of many ONAP’s components, which is essentially OpenDaylight with added functionality for the execution of directed graphs.

SDN-C deployment architecture

ONAP SDN-C deployment is composed of several Docker containers. One of these containers runs Directed Graph Builder. It’s a web UI you can use to build directed graphs in a user-friendly manner.

Another container runs Admin Portal, which is another web UI for SDNC.

The next one runs the relational database, which is the central point of SNDC deployment and each container is using it.

Finally, there is a container that runs the SDNC controller itself. This is the one we are in particular interested in because it has all the logic behind the execution of directed graphs.

We found out that the SDNC container takes more than 300 seconds to initialize and it consumes 1,125 GiB of RAM after initialization. We were thinking – could we do something to make it more efficient? lighty.io comes to the rescue!

Migration of SDNC to lighty.io

As we already mentioned, SDNC is based on Opendaylight and what we wanted to achieve is make it more efficient and more suitable for microservice deployments. This is exactly the use case for lighty.io. We decided to migrate SDNC from Opendaylight to lighty.io and build our own Docker image for it.

We cover some of the technical aspects behind this migration in a separate article, here we will only summarize our achievements:

Ability to execute directed graphs (you can see it in action in our video)
Less memory usage
Quicker startup time

Here is a simple table with a comparison of our Pantheon lighty.io SDNC container with vanilla SDNC container:

	Vanilla SDN-C container	lighty.io SDN-C container
Initialization time[1]	388 s	147 s
Memory usage after startup	1,125 GiB	613,1 MiB
Memory usage after test graph[2] execution	1,246 GiB	937 MiB
Average execution time of test graph	1,49 s	1,2 s
Average cpu[3] usage during test graph execution	115,27 %	108,06 %

[1]The execution time of init script running in containers

[2]The graph used for testing reads data from MD-SAL and writes them to log 1000 times in a loop.

[3]Containers have available 4 CPU cores

Summary

Thanks to the migration exercise we proved that even a quite complex application can be migrated to lighty.io in a short time and achieve measurable improvements. Click here to read more about the migration process!

If you are interested in further SDN/C performance and architecture optimizations, contact us!

The post ONAP SDN-C appeared first on lighty.io.

OpenFlow Support in lighty.io

lighty.io — Thu, 29 Mar 2018 06:48:36 +0000

OpenFlow enables network controllers to determine the path of network packets across a network of switches.

Controllers are different from switches. Separation of control from the forwarding allows for more sophisticated traffic management than is feasible using access control lists (ACLs) and routing protocols.

Also, OpenFlow allows switches from different vendors — often each with their own proprietary interfaces and scripting languages — to be managed remotely using a single, open protocol. The protocol’s inventors consider OpenFlow an enabler of software-defined networking (SDN).

OpenFlow allows remote administration of a layer 3 switch’s packet forwarding tables, by adding, modifying, and removing packet matching rules and actions.

With this solution, you can make routing decisions periodically or ad hoc by the controller after which they can be translated into rules and actions with a configurable lifespan.

OpenFlow Capabilities

Later on, they can be deployed to a switch’s flow table, leaving the actual forwarding of matched packets to the switch at wire speed for the duration of those rules. Packets that are unmatched by the switch can be forwarded to the controller.

The controller can then decide to modify existing flow table rules on one or more switches or to deploy new rules, to prevent a structural flow of traffic between switch and controller.

It could even decide to forward the traffic itself, provided that it has told the switch to forward entire packets instead of just their header.

The OpenFlow protocol is layered on top of the Transmission Control Protocol (TCP) and prescribes the use of Transport Layer Security (TLS). Controllers should listen on TCP port 6653 for switches that want to set up a connection. Earlier versions of the OpenFlow protocol unofficially used port 6633.

Openflow usage in OpenDaylight is based on the OpenFlow plugin project. In order to connect OpenFlow devices to the lighty.io controller, we had to migrate the OpenFlow plugin project.

The Openflow project and its parts are initialized using blueprints in vanilla OpenDaylight, so to get the project up in lighty.io we have to initialize these explicitly. In addition, the OpenFlow plugin has a direct link to the Karaf bundle context and so we have to re-implement the OpenFlow plugin configuration part.

Although this sounds like a lot of work, it is still very straightforward.

Openflowplugin detail architecture

Blueprints in OpenFlow plugin

    
    
    
    
    
    
    
        
        
        
    
    
    
        
        
        
        
        
        
            
                
                
           
        
        
        
    
    
        
            org.opendaylight.openflowplugin.api.openflow.OpenFlowPluginProvider
            org.opendaylight.openflowplugin.extension.api.OpenFlowPluginExtensionRegistratorProvider

As you can see, the OpenFlow plugin uses OpenDaylight Controller dependencies and classes, like Entityownership, ClusterSingletonService, DataBroker, and NotificationService.

The Open Daylight Controller supports the OpenFlow protocol. The Controller architecture supports both the Hybrid Switch model as well as the classic OpenFlow model of having a fully centralized Control Plane. Service using bundle context

/**
 * Factory for creating ConfigurationService instances.
 */
public interface ConfigurationServiceFactory {
    ConfigurationService newInstance(OpenflowProviderConfig providerConfig,
                                     BundleContext bundleContext);
}

The OpenDaylight OpenFlow plugin implementation of this interface is using the Karaf bundle context, which we don’t have in lighty.io. Therefore, we need to re-implement it in lighty.io.

Configuration of OpenFlow plugin w/ lighty.io

OpenFlow plugin uses a separate configuration for each plugin configuration and separate configuration for switch connectors. In this lighty.io example use case we’ve put it together and here you can see an example JSON file to configure OpenFlow plugin.

{
  "openflow": {
    "isStatisticsPollingOn": true,
    "barrierCountLimit": "1234",
    "barrierIntervalTimeoutLimit": "3000",
    "basicTimerDelay": "2690",
    "echoReplyTimeout": "4000",
    "globalNotificationQuota": "9000",
    "maximumTimerDelay": "3679",
    "rpcRequestsQuota": "2500",
    "threadPoolMaxThreads": "1000",
    "threadPoolMinThreads": "3",
    "threadPoolTimeout": "60",
    "enableFlowRemovedNotification": "true",
    "isStatisticsRpcEnabled": "false",
    "skipTableFeatures": "true",
    "switchFeaturesMandatory": "false",
    "useSingleLayerSerialization": "true",
    "switchConfig": {
      "instanceName":"openflow-switch-connection-provider-default-impl",
      "port":1234,
      "transportProtocol":0,
      "useBarrier":true,
      "switchIdleTimeout":15001,
      "keystore":"configuration/ssl/ctl.jks",
      "keystoreType":0,
      "keystorePathType":1,
      "keystorePassword":"opendaylight",
      "truststore":"configuration/ssl/truststore.jks",
      "truststoreType":0,
      "truststorePathType":1,
      "truststorePassword":"opendaylight",
      "certificatePassword":"opendaylight"
    }
  }
}

Openflowplugin example application in lighty.io

We managed to create an example application using lighty.io. It might be a helping hand to those who wish to use OpenFlow their lighty.io controllers.

The structure of the example is the same as in the other examples: start the necessary dependencies such as Controller or Restconf. We need Restconf in this example only to be able to manage the resulting controller using operations like a post or to get flow and device information.

Program core structure:

//1. initialize and start Lighty controller (MD-SAL, Controller, YangTools, Akka)
final LightyControllerBuilder lightyControllerBuilder = new LightyControllerBuilder();
final LightyController lightyController = lightyControllerBuilder.from(controllerConfiguration).build();
lightyController.start().get();
//2. start Restconf server
final RestconfBuilder restconfBuilder = new RestconfBuilder();
final LightyModule restconf = restconfBuilder.from(RestconfConfigUtil
        .getRestConfConfiguration(restConfConfiguration, lightyController.getServices()))
        .build();
restconf.start();
//3. start openflow SBP
final OpenflowSouthboundPlugin plugin;
plugin = new OpenflowSouthboundPluginBuilder()
        .from(configuration, lightyController.getServices())
        .build();
plugin.start();

Controller start and device connection

After a build of a lighty-restconf-ofp-app in lighty.io, you can unzip the created file into a separate folder and run the controller with start script.

INFO [main] (Main.java:100) - Lighty and OFP started in 7649.765ms

The above logline tells us, that the controller and OpenFlow plugin are running. Now we can join an OpenFlow device using Mininet.

sudo mn --controller=remote,ip=127.0.0.1 --topo=tree,1 --switch ovsk,protocols=OpenFlow13

We can see in the log above, that the device openflow:1 is connected and mastered by the controller.

INFO [nioEventLoopGroup-10-2] (ContextChainHolderImpl.java:225) - Role MASTER was granted to device openflow:1

Post Flow / Check Device in the operational datastore

The last thing is to set a flow on the connected device (opeflow:1) and check if it is properly installed.

POST this flow:

curl -X POST -k -H 'Content-Type: application/xml' -H 'Authorization: Basic YWRtaW46YWRtaW4=' -i https://localhost:8888/restconf/operations/sal-flow:add-flow --data '

   false
   /inv:nodes/inv:node[inv:id="openflow:1"]
   55
   SEND_FLOW_REM
   0
   0
   false
   
    
     
       2048
     
    
    10.0.10.2/32
   
	
		
			0
			
				0
				
					INPORT
				
				
			
		
	
	0
   false
   0

And check the device:

curl -X GET -k -H 'Authorization: Basic YWRtaW46YWRtaW4=' -i 'https://localhost:8888/restconf/data/opendaylight-inventory:nodes/node=openflow%3A1/table=0?content=nonconfig'

After all this, you have to create proof that the device is connected with the newly created lighty.io controller using the OpenDaylight OpenFlow plugin.

{
  "flow-node-inventory:table": [{
    "id": 0,
    "flow": [{
      "id": "#UF$TABLE*0-1",
      "cookie_mask": 0,
      "priority": 0,
      "table_id": 0,
      "opendaylight-flow-statistics:flow-statistics": {
        "duration": {
          "second": 20,
          "nanosecond": 196000000
        },
        "byte-count": 0,
        "packet-count": 0
      },
      "match": {
        "ipv4-destination": "10.0.10.2/32",
        "ethernet-match": {
          "ethernet-type": {
            "type": 2048
          }
        }
      },
      "flags": "SEND_FLOW_REM",
      "idle-timeout": 0,
      "hard-timeout": 0,
      "cookie": 55,
      "instructions": {
        "instruction": [{
          "order": 0,
          "apply-actions": {
            "action": [{
              "order": 0,
              "output-action": {
                "output-node-connector": "IN_PORT",
                "max-length": 0
              }
            }]
          }
        }]
      }
    }]
  }]
}

Congratulations! You have successfully experimented with the OpenFlow plugin in lighty.io!

The post OpenFlow Support in lighty.io appeared first on lighty.io.

Migration of TransportPCE to lighty.io

lighty.io — Wed, 28 Mar 2018 09:42:41 +0000

Update 13/8/2019: lighty TransportPCE Controller is now available in the OpenDaylight upstream!

The migration of an OpenDaylight application to lighty.io is simple and straightforward. In this post, we are going to show you how to do this with help of the TransportPCE OpenDaylight project acting as an example.

Users can find many examples shipped along with the lighty.io package to help them jumpstart projects, but this post will detail the migration process in particular.

The major part of the migration is removing hard dependencies in OSGi / Karaf.

This exercise is good for the OpenDaylight project itself, as these dependencies should not exist in the first place. We recommend sending the resulting patch to the upstream track as a step towards cleaner code.

Since there are no OSGI dependencies in the TransportPCE project, we will focus on the rest of the work that needs to be done. To begin, you need to find out what other OpenDaylight modules (e.g. controller, NETCONF, RESTCONF) the given project uses (depends on).

TransportPCE’s dependencies are already migrated to lighty.io and therefore it is as easy as initialization of an object and calling a start function.

Initializing object and calling a start function:

        //1. initialize and start OpenDaylight controller (MD-SAL, Controller, YangTools, Akka)
        final LightyControllerBuilder lightyControllerBuilder = new LightyControllerBuilder();
        final LightyController lightyController = lightyControllerBuilder.from(controllerConfiguration).build();
        lightyController.start().get();

The second step is to build the lighty.io TransportPCE object. To do this you have to go to the original project and find all the blueprint files.

These will specify the dependency references that are needed by your newly created object constructor and beans, which are the objects that you will create using those references.

These beans also have names of functions that you need for initializing and destroying of the object blueprint example:

The next step is to initialize all the objects that TransportPCE needs to function. lighty.io has an abstract helper class called AbstractLightyModule which you should extend.

With this, you will be overriding an initProcedure function and whenever you call start on your lighty.io object, this function will be triggered. In this function, all you need to do is to create all the beans from blueprints and call their initializing function.

Start method from abstract class:

public synchronized ListenableFuture start() {
        if(this.running) {
            LOG.warn("LightyModule {} is already started.", this.getClass().getSimpleName());
            return Futures.immediateFuture(Boolean.valueOf(true));
        } else {
            if(this.executorService == null) {
                LOG.debug("Creating default single thread ExecutorService for LightyModule {}.", this.getClass().getSimpleName());
                this.executorService = MoreExecutors.listeningDecorator(Executors.newSingleThreadExecutor());
            }
            LOG.info("Submitted start of LightyModule {}.", this.getClass().getSimpleName());
            return this.executorService.submit(() -> {
                synchronized(this) {
                    LOG.debug("Starting initialization of LightyModule {}", this.getClass().getSimpleName());
                    boolean initResult = this.initProcedure();
                    this.running = true;
                    LOG.info("LightyModule {} started.", this.getClass().getSimpleName());
                    return Boolean.valueOf(initResult);
                }
            });
        }
    }

Initializing and calling start on lightyTransportPce object:

        //start TransportPce application
        final LightyTransportPce lightyTransportPce;
        final TransportPceBuilder lightyTransportPceBuilder = new TransportPceBuilder();
        lightyTransportPce = lightyTransportPceBuilder.from(lightyController.getServices()).build();
        lightyTransportPce.start().get();
overriding a initProcedure method:
    @Override
    protected boolean initProcedure() {
        final long startTime = System.nanoTime();
        LOG.info("Starting TransportPceProvider");
        transportpceProvider =
                new TransportpceProvider(dataBroker, mountPointService);
        transportpceProvider.init();
        LOG.info("Starting RendererProvider");
        rendererProvider = new RendererProvider(dataBroker, mountPointService,
                rpcProviderRegistry);
        rendererProvider.init();
        LOG.info("Starting OlmProvider");
        olmProvider = new OlmProvider(dataBroker, mountPointService, rpcProviderRegistry);
        olmProvider.init();
        LOG.info("Starting StubrendererProvider");
        stubrendererProvider = new StubrendererProvider(rpcProviderRegistry,
                notificationService,notificationPublishService);
        stubrendererProvider.init();
        LOG.info("Starting StubpceProvider");
        stubpceProvider = new StubpceProvider(rpcProviderRegistry, notificationService,
                notificationPublishService);
        stubpceProvider.init();
        LOG.info("Starting ServiceHandlerProvider");
        servicehandlerProvider = new ServicehandlerProvider(dataBroker, rpcProviderRegistry,
                notificationService, notificationPublishService);
        servicehandlerProvider.init();
        final float delay = (System.nanoTime() - startTime) / 1_000_000f;
        LOG.info("Lighty transportpce started in {}ms", delay);
        return true;
    }

The last step is to override the stopProcedure function. This function needs to call the destroy method of an object that you created. Whenever shutdown is called on a lighty.io object, this method will be triggered.

Override a stopProcedure method:

@Override
    protected boolean stopProcedure() {
        if (transportpceProvider != null) {
            transportpceProvider.close();
            LOG.info("TransportpceProvider closed");
        }
        if (olmProvider != null) {
            olmProvider.close();
            LOG.info("OlmProvider closed");
        }
        if (rendererProvider != null) {
            rendererProvider.close();
            LOG.info("RendererProvider stopped");
        }
        if (rendererProvider != null) {
            rendererProvider.close();
            LOG.info("RendererProvider stopped");
        }
        if (servicehandlerProvider != null) {
            servicehandlerProvider.close();
            LOG.info("ServicehandlerProvider stopped");
        }
        if (stubpceProvider != null) {
            stubpceProvider.close();
            LOG.info("StubpceProvider stopped");
        }
        if (stubrendererProvider != null) {
            stubrendererProvider.close();
            LOG.info("StubrendererProvider stopped");
        }
        return true;
    }
Shutdown method from abstract class
    public synchronized ListenableFuture shutdown() {
        if(!this.running) {
            LOG.warn("LightyModule {} is already shut down.", this.getClass().getSimpleName());
            return Futures.immediateFuture(Boolean.valueOf(true));
        } else {
            LOG.info("Submitted shutdown of LightyModule {}.", this.getClass().getSimpleName());
            ListenableFuture shutdownFuture = this.executorService.submit(() -> {
                synchronized(this) {
                    LOG.debug("Starting shutdown procedure of LightyModule {}.", this.getClass().getSimpleName());
                    boolean stopResult = this.stopProcedure();
                    this.shutdownLatch.countDown();
                    this.running = false;
                    LOG.info("LightyModule {} shutdown complete.", this.getClass().getSimpleName());
                    return Boolean.valueOf(stopResult);
                }
            });
            return !this.executorIsProvided?Futures.transform(shutdownFuture, (result) -> {
                LOG.debug("Shutdown default ExecutorService of LightyModule {}.", this.getClass().getSimpleName());
                this.executorService.shutdown();
                this.executorService = null;
                return Boolean.valueOf(true);
            }, MoreExecutors.directExecutor()):shutdownFuture;
        }
    }

Keep in mind, that your controller has to have access to all YANG model artifacts that you are using in your project. If you fail to do so, the schema context of the controller will be missing them and therefore some of your bean objects will not work properly.

As you can see, these steps are simple and straightforward. If all the OpenDaylight projects were this clean, a simple script would be able to take care of migration from OpenDaylight to lighty.io.

The post Migration of TransportPCE to lighty.io appeared first on lighty.io.

HTTP Server w/ YANG Modeled RPC in Java

lighty.io — Wed, 28 Mar 2018 08:54:21 +0000

YANG is a data modeling language, which allows modeling of Remote Procedure Calls (RPCs).

YANG is widely used in networking for modeling configurational and operational data of network devices (more than just routers and switches). These devices also implement some of the well-defined protocols for transport of the data modeled in YANG e.g.: NETCONF, RESTCONF, or GRPC.

Network controller software can manage such devices, by implementing these protocols and using the YANG models of the configuration and operational data.

This way, the controller or other software applications on top of the controller can configure and monitor a network of huge size. OpenDaylight is the leading open source SDN controller, which provides such functionalities.

Meet lighty.io, which is based on OpenDaylight components with additional PANTHEON.tech plugins & tools for fast development of lightweight SDN applications. One of the provided extensions by lighty.io is a unified API for YANG modeled data transformations.

This API is important for the integration of lighty.io with external systems. This article describes how to model RPC in YANG modeling language, how to generate Java classes from the YANG model, and how to implement the modeled RPC over HTTP protocol using the Codecs component of lighty.io.

The YANG model of the example RPC (see the yang model file for more details) is pretty straight forward:

    rpc compute-sum {
        input {
            leaf operand-a {
                type uint16;
                mandatory true;
            }
            leaf operand-b {
                type uint16;
                mandatory true;
            }
        }
        output {
            leaf sum {
                type uint32;
                mandatory true;
            }
        }
    }

The input of the RPC consists of two numbers and the output represents the sum of input numbers.

OpenDaylight – YANG – Maven

The ODL yang-maven-plugin is used to generate Java classes and interfaces from the Yang model (see the pom.xml of this example for more details).

An interface called LightyRpcExampleService is generated, which defines the RPC as a Java method with the input container as argument and output container as a return value (wrapped by Future>).

Now we need to provide an implementation for this freshly generated interface:

package tech.pantheon.lighty.examples.rpc;
import org.opendaylight.yang.gen.v1.urn.pantheon.params.xml.ns.yang.lighty.rpc.example.rev180219.ComputeSumInput;
import org.opendaylight.yang.gen.v1.urn.pantheon.params.xml.ns.yang.lighty.rpc.example.rev180219.ComputeSumOutput;
import org.opendaylight.yang.gen.v1.urn.pantheon.params.xml.ns.yang.lighty.rpc.example.rev180219.ComputeSumOutputBuilder;
import org.opendaylight.yang.gen.v1.urn.pantheon.params.xml.ns.yang.lighty.rpc.example.rev180219.LightyRpcExampleService;
import org.opendaylight.yangtools.yang.common.RpcResult;
import org.opendaylight.yangtools.yang.common.RpcResultBuilder;
import java.util.concurrent.Future;
public class LightyRpcExampleServiceImpl implements LightyRpcExampleService {
    public Future> computeSum(ComputeSumInput input) {
        return RpcResultBuilder.success(
                new ComputeSumOutputBuilder()
                        .setSum((long) input.getOperandA() + input.getOperandB())
                        .build())
                .buildFuture();
    }
}

The next step is to write a code, which will decode serialized input (received as HTTP payload), call the RPC, and serialize the resulting output to be sent back to the client.

In the beginning, it is needed to initialize a schema context including the YANG model which is needed for serialization and deserialization of modeled data.

This example shows how the schema context and RPC definition objects are initialized in the constructor of class RpcHttpServer from this example:

        System.out.println("Loading Yang model schema context");
        /* Load YANG models available at the class path */
        List moduleInfos = new LinkedList<>();
        ServiceLoader yangProviderLoader = ServiceLoader.load(YangModelBindingProvider.class);
        for (YangModelBindingProvider yangModelBindingProvider : yangProviderLoader) {
            moduleInfos.add(yangModelBindingProvider.getModuleInfo());
        }
        /* Create the schema context for loaded models */
        ModuleInfoBackedContext moduleInfoBackedCntxt = ModuleInfoBackedContext.create();
        moduleInfoBackedCntxt.addModuleInfos(moduleInfos);
        Optional schemaCtx =
                moduleInfoBackedCntxt.tryToCreateSchemaContext().toJavaUtil();
        if (!schemaCtx.isPresent()) {
            throw new IllegalStateException("Failed to load schema context");
        }
        this.schemaContext = schemaCtx.get();
        System.out.println("Yang model schema context loaded");
        /* Load the RPC used in this example */
        Optional loadRpc = ConverterUtils.loadRpc(schemaContext, rpcServiceQname);
        if (!loadRpc.isPresent()) {
            throw new IllegalStateException("RPC definition not found");
        }
        this.rpcDef = loadRpc.get();
        System.out.println("Loaded definition of RPC: " + this.rpcDef.getQName().getLocalName());

The last code snippet shows how to deserialize the incoming payload of HTTP request encoded in JSON using a coded object. The method implementing RPC is called with the input object as an argument, while the output object is obtained as a return value of the method.

The output object is then serialized to JSON using the same codec object as for input HTTP payload. The output JSON data can be written into HTTP response payload (see the example source code for more details).

                /* Read the payload from HTTP request */
                final String payload = com.google.common.io.CharStreams.toString(req.getReader());
                if (payload.isEmpty()) {
                    System.out.println("Nothing received");
                    return;
                } else {
                    System.out.println("Received payload: " + payload);
                }
                /* Create codec for the RPC input and load the RPC definition */
                DataCodec dataCodec = new DataCodec<>(schemaContext);
                /* Deserialize the payload to normalized node */
                NormalizedNode rpcInputNode =
                        dataCodec.withJson().deserialize(rpcDef, new StringReader(payload));
                /* Convert the normalized node to RPC's input object */
                ComputeSumInput rpcInput =
                        dataCodec.convertToBindingAwareRpc(rpcDef.getInput().getPath(),
                                                           (ContainerNode) rpcInputNode);
                /* Call the RPC and get RPC output */
                ComputeSumOutput rpcOutput =
                        rpcImpl.computeSum(rpcInput).get().getResult();
                /* Serialize the RPC output */
                Writer output = dataCodec.withJson().serializeRpc(rpcDef.getOutput(),
                                                                  dataCodec.convertToBindingIndependentRpc(rpcOutput));
                System.out.println("RPC output: " + output.toString());

You can use the following to test the RPC, and see the output:

curl -H “Content-Type: application/json” -X POST -d ‘{ “input”: { “operand-a”: 200, “operand-b”: 22 }}’ http://localhost:8080

{“output”:{“sum”:222}}

A sample output from the Java app above:

Loading Yang model schema context
Yang model schema context loaded
Loaded definition of RPC: compute-sum
HTTP server started at port 8080
Press any key to exit...
Received payload: { "input": { "operand-a": 200, "operand-b": 22 }}
RPC output: {"output":{"sum":222}}

The post HTTP Server w/ YANG Modeled RPC in Java appeared first on lighty.io.

lighty.io NETCONF Device on ARM

lighty.io — Tue, 27 Mar 2018 10:43:25 +0000

lighty.io components are versatile and reusable in different scenarios. Let’s take a closer look at the lighty-netconf-device java library.

This lighty.io component represents a fully programmable NETCONF communication stack, which can be integrated with hardware to achieve interoperability with lighty.io NETCONF – RESTCONF controllers.

The lighty-netconf-device library implements a server for RFC 6241 and provides related services like Datastore, RPC handling, and NETCONF protocol operations.

Raspberry Pi is a very popular platform for hardware prototyping. Raspberry Pi 3 utilizes Broadcom ARM v7 BCM2835 SoC.

In this example, we will build a simple device equipped with:

4 LED state indicators
4 buttons and sensors for measuring
- temperature
- relative humidity
- atmospheric pressure.

Device schematics and wiring is described on the picture below.

Hardware components

Raspberry PI 3 Model B v1.2
BMP-180 I2C Bosch temperature and atmospheric pressure sensor
HTU-21D I2C temperature and relative humidity sensor

Software Stack

The software stack used on the Raspberry PI device consists of following components:

The next step is to create a YANG model of the device:

module rpi-device {
    yang-version 1.1;
    namespace "urn:tech.pantheon.netconfdevice.rpid";
    prefix "dc";
    organization "Pantheon Technologies";
    description
        "This module represents API to Raspberry-Pi test device";
    revision "2018-02-25" {
        description
            "Initial revision";
    }
    grouping led-data {
        leaf led-id {
             type int8 {
                  range "0 .. 3";
             }
             description "Unique LED port ID. Allowed values are 0,1,2,3.";
        }
        leaf led-status {
             type boolean;
             description "LED port status true=high/on, false=low/off.";
        }
    }
    container RpiDevice {
        description "Data model of RPis test device.";
        leaf device-name {
            type string;
            description "Name of this device.";
        }
        leaf temperature {
            type decimal64 {
                 fraction-digits 3;
                 range "-200 .. 200";
            }
            config false;
            description "Ambient temperature in Celsius degree [C].";
        }
        leaf relative-humidity {
            type decimal64 {
                 fraction-digits 3;
                 range "0 .. 100";
            }
            config false;
            description "Ambient relative humidity in percentage [%].";
        }
        leaf atmospheric-pressure {
            type decimal64 {
                 fraction-digits 3;
                 range "0 .. 1000000";
            }
            config false;
            description "Atmospheric Pressure in pascals [Pa].";
        }
        list led-indicators {
             config false;
             description "0-3 LED port indicator list";
             key "led-id";
             uses led-data;
        }
    }
    rpc set-led {
        description "set LED port status high/on or low/off.";
        input {
           uses led-data;
        }
        output {
           uses led-data;
        }
    }
}

The last step is to implement the Java code, which initializes the NETCONF stack, datastore, and hardware bindings.

These interact with the sensors and IO ports of Raspberry PI. Example below starts NETCONF interface on port 17830. The rpi-device@2018-02-25.yang YANG model is used and the device Datastore is initialized with data from configuration.

Hardware drivers for sensors and IO ports are injected using constructors. The main class, which starts the NETCONF device application, looks like this:

public static void main(String[] args) {
    //1. Load models from classpath
    Set rpiModules = ModelUtils.getModelsFromClasspath(ModuleId.from("rpi-device", "2018-02-25"));
    //2. Initialize DataStores
    InputStream initialDataConfig = Main.class.getResourceAsStream("/initial-config.xml");
    InputStream initialDataOperational = Main.class.getResourceAsStream("/initial-operational.xml");
    //3. Initialize hardware driver
    HardwareDriver hardwareDriver = HardwareDriverFactory.create();
    //4. Initialize RPCs
    RpiDeviceServiceImpl rpiDeviceService = new RpiDeviceServiceImpl(hardwareDriver);
    SetLedProcessor setLedProcessor = new SetLedProcessor(rpiDeviceService);
    //5. Initialize Netconf device
    NetconfDevice netconfDevice = new NetconfDeviceBuilder()
            .setCredentials("admin", "admin")
            .setBindingPort(17830)
            .setInitialConfigurationData(initialDataConfig)
            .setInitialOperationalData(initialDataOperational)
            .withModels(rpiModules)
            .withDefaultRequestProcessors()
            .withRequestProcessor(setLedProcessor)
            .withDefaultCapabilities()
            .build();
    netconfDevice.start();
    //6. Initialize hardware scanner
    DataStoreService dataStoreService = new DataStoreService(netconfDevice.getNetconfDeviceServices().getDataBroker());
    hardwareDriver.setListener(dataStoreService);
    HardwareScanner hardwareScanner = new HardwareScanner(dataStoreService, hardwareDriver, TimeUnit.SECONDS, 5);
    new Thread(hardwareScanner).start();
}

Now we have NETCONF device up and running on network, let’s use lighty-netconf-restconf-app as a SDN controller to manage this device.

This lighty.io based controller uses OpenDaylight core components, NETCONF south-bound plugin and Pantheon’s RESTCONF RFC8040 implementation as a north-bound plugin. Whole SDN architecture looks like this:

Now we can use RESTCONF to connect and access Raspberry PI NETCONF device in 4 easy steps:

Connect the device to the SDN controller

POST https://localhost:8888/restconf/data/network-topology:network-topology/topology=topology-netconf

  {
    "netconf-topology:node" :[ {
      "node-id": "rpi-device",
      "host": "192.68.0.102",
      "port": 17830,
      "username": "admin",
      "password": "admin",
      "tcp-only": false,
      "keepalive-delay": 0,
      "netconf-node-configuration:schemaless": false
      }
    ]
  }
reply 201

Check the device connection status

GET https://localhost:8888/restconf/data/network-topology:network-topology/topology=topology-netconf?content=nonconfig

reply 200:
{
    "topology": [
        {
            "topology-id": "topology-netconf",
            "node": [
                {
                    "node-id": "toaster-device",
                    "netconf-node-topology:available-capabilities": {
                        "available-capability": [
                            {
                                "capability": "urn:ietf:params:netconf:base:1.1",
                                "capability-origin": "device-advertised"
                            },
                            {
                                "capability": "urn:ietf:params:netconf:base:1.0",
                                "capability-origin": "device-advertised"
                            },
                            {
                                "capability": "urn:ietf:params:netconf:capability:candidate:1.0",
                                "capability-origin": "device-advertised"
                            },
                            {
                                "capability": "(urn:tech.pantheon.netconfdevice.rpid?revision=2018-02-25)rpi-device",
                                "capability-origin": "device-advertised"
                            }
                        ]
                    },
                    "netconf-node-topology:host": "127.0.0.1",
                    "netconf-node-topology:unavailable-capabilities": {},
                    "netconf-node-topology:connection-status": "connected",
                    "netconf-node-topology:port": 17830
                }
            ]
        }
    ]
}

Access the device data

GET https://localhost:8888/restconf/data/network-topology:network-topology/topology=topology-netconf/node=rpi -device/yang-ext:mount/rpi-device:RpiDevice

reply 200
{
    "RpiDevice": {
        "atmospheric-pressure": 111111.109375,
        "device-name": "RPi device 01",
        "led-indicators": [
            {
                "led-id": 0,
                "led-status": true
            },
            {
                "led-id": 1,
                "led-status": false
            },
            {
                "led-id": 2,
                "led-status": false
            },
            {
                "led-id": 3,
                "led-status": false
            }
        ],
        "temperature": 22.2199993133544921875,
        "relative-humidity": 44.439998626708984375
    }
}

Set the device LED port status

POST https://localhost:8888/restconf/operations/network-topology:network-topology/topology=topology-netconf/node=rpi -device/yang-ext:mount/rpi-device:set-led

{
	"input": {
		"led-id": 0,
		"led-status": true
	}
}
reply 200:
{
    "output": {
        "led-status": true,
        "led-id": 0
    }
}

This example shows you, how easily you can create your own YANG model, to be able to manage your embedded devices with lighty.io SDN controller.

If you don’t dare to write your own YANG models, you can search for opensource models from standard bodies like IETF, or Openconfig.net.

Juraj Veverka

The post lighty.io NETCONF Device on ARM appeared first on lighty.io.