25 Mar 2018, 06:34

Pipe Routing Test 01

This is a test simulation of random pipe routing.

Initial test with 200mm pipe diameter.

temperature

temperature

temperature

temperature

Additional test with varying pipe diameters.

The following short video shows the pipe generation process. The animation is rendered using the povray export.

Another simulation, where the pipe diameter was 400mm.

temperature

21 Dec 2016, 15:40

IoT Device Simulation With IoSynth

Summary: In this article, I will show you how to build a test environment to simulate 1000 IoT devices connected to MQTT broker.

Update: New snapshot have new configuration. To get the new version and documentation go to IoSynth site.

When developing new IoT products it is usually useful to have controlled and reproducible environment of IoT devices for testing and experimenting. Scaling such environment to hundreds of thousands devices can help evaluating the products with realistic load.

MQTT protocol has been widely used in various industries to communicate between IoT devices. It is a machine-to-machine connectivity protocol.

I have created small Java package that simulates IoT devices. Each device contains a set of sensors with different behavior. The program connects to MQTT broker and publishes data from devices in predefined topics.

To start playing with the package you will need a MQTT broker. You may install one of the many available implementations such as Mosquitto, or you can use some of the publically accessible sandbox servers like iot.eclipse.org:1883.

The MQTT works around the concept of “topics”. That is a directory like structure where the MQTT clients publish messages to broker and other clients subscribe to these broker topics to receive messages.

IoSynth package simulate multiple devices that generate synthetic data and publish messages to e MQTT broker topics. I use the Paho client library to implement MQTT protocol.

The simulator library creates Device objects that are containers for Sensor objects. Each sensor have predefined generation function (Random Walk, Cycle through values, etc.…). There are generally two types of devices: devices with fixed sampling rate and devices with variable rate. The sensors produce numerical and categorical data. If you do not want to write java code to create devices and sensors, I have provided some ready to use applications with predefined sensors and devices. Currently appliications a very limited in configuration options.

You can run the application from command line like this:

/home/user> java -cp iosynth-0.0.1.jar net.iosynth.app.AppMqtt -c MqttConfig.json

Where the configuration file MqttConfig.json is like this:

{
  "topic":"iosynth/",
  "qos":2,
  "broker":"tcp://localhost:1883",
  "session":"mdejsighzne"
}

This will run the default application provided in the jar file assuming you have Java 7 installed in your PATH. The jar contains prepackaged dependencies (Paho Library), so you don’t need to search and install them. This will create a set of 10 devices that will publish the data on “iosynth.net/mdejighzne/devices/#” topic on you localhost:1883 MQTT broker. The session specify random string used to create unique topic. If omitted, one will be generted for each run.

If you do not have broker installed you can use one public broker like the following:

   {
      "topic":"iosynth/",
      "qos":2,
      "broker":"tcp://iot.eclipse.org:1883",
      "session":"mdejsighzne"
    }

Now you can see the published messages using a MQTT client by subscribing to the topics. I use the client provided with the Mosquitto installation:

/home/user> mosquitto_sub -v -t "iosynth/+/device/#" -h localhost -p 1883

The result should be like this:

iosynth/mdejsighzne/device/0010 {"time":"2016-12-23 15:42:12.870","door":"OF","1st floor":"elevator_02","state":18,"count":401}
iosynth/mdejsighzne/device/0003 {"time":"2016-12-23 15:42:13.436","state":7,"level":1.3000,"temp":18.2241,"semafor":"green","word":"Golf","prime":281}
iosynth/mdejsighzne/device/0001 {"time":"2016-12-23 15:42:13.667","state":7,"level":1.3000,"temp":16.1164,"semafor":"green","word":"Sierra","prime":181}
iosynth/mdejsighzne/device/0005 {"time":"2016-12-23 15:42:14.111","state":7,"level":1.3000,"temp":18.4741,"semafor":"green","word":"Xray","prime":313}
iosynth/mdejsighzne/device/0004 {"time":"2016-12-23 15:42:14.247","state":7,"level":1.3000,"temp":16.4652,"semafor":"green","word":"Lima","prime":461}
iosynth/mdejsighzne/device/0002 {"time":"2016-12-23 15:42:14.513","state":7,"level":1.3000,"temp":19.4194,"semafor":"green","word":"Foxtrot","prime":43}
iosynth/mdejsighzne/device/0007 {"time":"2016-12-23 15:42:15.270","door":"ON","1st floor":"elevator_03","state":1,"count":307}
iosynth/mdejsighzne/device/0009 {"time":"2016-12-23 15:42:16.973","door":"ON","1st floor":"elevator_03","state":13,"count":439}
iosynth/mdejsighzne/device/0003 {"time":"2016-12-23 15:42:23.436","state":8,"level":1.2000,"temp":18.2490,"semafor":"yellow","word":"Whiskey","prime":503}
iosynth/mdejsighzne/device/0001 {"time":"2016-12-23 15:42:23.667","state":8,"level":1.2000,"temp":16.0466,"semafor":"yellow","word":"Charlie","prime":3}
iosynth/mdejsighzne/device/0005 {"time":"2016-12-23 15:42:24.111","state":8,"level":1.2000,"temp":18.4225,"semafor":"yellow","word":"Papa","prime":467}
iosynth/mdejsighzne/device/0004 {"time":"2016-12-23 15:42:24.247","state":8,"level":1.2000,"temp":16.5070,"semafor":"yellow","word":"Kilo","prime":373}
iosynth/mdejsighzne/device/0002 {"time":"2016-12-23 15:42:24.513","state":8,"level":1.2000,"temp":19.4637,"semafor":"yellow","word":"Juliett","prime":149}
iosynth/mdejsighzne/device/0007 {"time":"2016-12-23 15:42:25.000","door":"OF","1st floor":"elevator_04","state":1,"count":79}
iosynth/mdejsighzne/device/0006 {"time":"2016-12-23 15:42:26.330","door":"ON","1st floor":"elevator_03","state":9,"count":37}
iosynth/mdejsighzne/device/0008 {"time":"2016-12-23 15:42:26.737","door":"ON","1st floor":"elevator_03","state":16,"count":307}
...

The “#” character at the end is topic filter that shows all topics starting with the given prefix.

You can subscribe to individual device like this:

/home/user> mosquitto_sub -v -t "iosynth/+/device/0006" -h localhost -p 1883

This will show the device with Id 0006:

iosynth/mdejsighzne/device/0006 {"time":"2016-12-23 16:20:11.215","door":"ON","1st floor":"elevator_03","state":20,"count":409}
iosynth/mdejsighzne/device/0006 {"time":"2016-12-23 16:20:21.865","door":"OF","1st floor":"elevator_04","state":21,"count":151}
iosynth/mdejsighzne/device/0006 {"time":"2016-12-23 16:20:38.786","door":"ON","1st floor":"elevator_01","state":20,"count":443}
iosynth/mdejsighzne/device/0006 {"time":"2016-12-23 16:20:58.338","door":"OF","1st floor":"elevator_02","state":20,"count":181}

In case you do not have Mosquitto client installed you can use one of the chrome mqtt plugins like MQTTBox.

Here is a list of simulated devices/sensors:

  • 5 devices with fixed sample rate (10s) with following sensors.
    • sensor that cycle through list of integer numbers (state)
    • sensor that cycle through list of double numbers (level)
    • sensor that produce random numbers between 15 and 25 (temp)
    • sensor that cycle through list of strings (semafor)
    • sensor that produce random string from list of strings (word)
    • sensor that random random integer number from list of integers (prime)
  • 5 devices with variable sample rate (with uniform arival times between 1s and 25s).
    • sensor that cycle through list of strings (door)
    • sensor that cycle through list of strings (1st floor)
    • sensor that produce random numbers between 0 and 25 (state)

I have also created some other applications with more sensors that preferably should be run only on your own servers.

/home/user> java -cp iosynth-0.0.1.jar net.iosynth.app.App1000 -c MqttConfig.json

The App1000 application is the same as AppMqtt but creates 500+500 devices.

List of public MQTT brokers

Server Port
iot.eclipse.org 1883 / 8883
broker.hivemq.com 1883
test.mosquitto.org 1883 / 8883 / 8884
test.mosca.io 1883
broker.mqttdashboard.com 1883

26 Nov 2016, 15:51

MQTT to Node-RED Dashboard

Summary: Showing the steps from device data simulation to plotting the data on dashboard.

Ingredients

For this recipe I will use the following ingredients: IoSynth device simulator, Node.js, Node-RED, Dashboard.

I am using a Ubuntu 16.04 laptop and commands are executed in Bash shell.

Installing Node.js

Node.js is event-driven I/O server-side JavaScript environment. In other words, it is a platform where you can create event-driven applications.

sudo apt-get update
sudo apt-get install nodejs

In addition, install npm. This is Node.js package manager.

sudo apt-get install npm

More installation details on: how-to-install-node-js-on-ubuntu-16-04

Installing Node-RED

Node-RED is a visual tool for wiring the Internet of Things. Using it, you can wire together hardware devices, APIs or online services using graphical blocks and connectors. Node-RED runs on top of Node.js.

sudo npm install -g --unsafe-perm node-red

After Node-RED is installed, the next step is running it.

node-red

After that, you can access the Node-RED editor at http://localhost:1880

More installation details on: getting-started

Installing node-red-dashboard

Run the following command in your Node-RED user directory (typically ~/.node-red):

npm i node-red-dashboard

Then, open your Node-RED instance and you should have UI nodes available in the palette and a new dashboard tab in right side panel. The UI interface is available at http://localhost:1880/ui (if default Node-RED settings are used).

More installation details on: node-red-dashboard

Installing IoSynth

IoSynth is device data simulator. You can gen the latest release from the GitHub repository. iosynth-0.0.7.zip

Unzip the archive and use the iosynth-0.0.7.jar file.

Alternative way is to build the Jar file from sources:

git clone https://github.com/rradev/iosynth.git
cd iosynth
mvn package

If you have Java 1.7 or above installed, run the following command:

java -cp iosynth.jar net.iosynth.Mqtt -c config-mqtt.json -d devices.json

Where the “config-mqtt.json” file is:

{
        "uri": "tcp://iot.eclipse.org:1883",
        "topic": "iosynth/",
        "qos": 2,
        "clients": 1,
        "seed": 123456
}

This will connect to the public Eclipse MQTT broker, or you can use your own if you have one.

The “devices.json” file contains the configuration information for the simulated devices:


[
    {
        "type":"simple",
        "uuid":"device",
        "topic":"{$uuid}",
        "sampling":{"type":"fixed", "interval":1000},
        "copy":1,
        "sensors":[
            {"type":"dev.timestamp",    "name":"ts", "format":"ms"},
            {"type":"double_oscillator", "name":"temp", "min":27.0, "max":34.0, "period":600000, "noise":5.0, "anomaly":0.001}
        ]
    }
]

You have created a device that will publish data on “iosynth/device” topic on Eclipse broker every second.

This device have the following payload:

{"ts":1486196918654,"temp":34.0000}

Where the first element is a timestamp and the second is simulated sensor data between 27.0 and 34.0 with some other sensor parameters.

You can check that the broker is receiving data if you have some MQTT client installed. For example with the Mosquitto client:

mosquitto_sub -v -t "iosynth/#" -h iot.eclipse.org

Now that you have the data streaming, lets create a Node-RED flow to collect the data and plot it on the dashboard.

Open the Node-RED editor and lets check that we can receive data from MQTT broker.

Drag a “mqtt” node to flow view and click on it to configure it: - server iot.eclipse.org:1883 - topic iosynth/device - name temp

Drag a “debug” node and connect it after “mqtt” node. This will allow you to check that you are receiving the data from MQTT broker. Now, deploy the flow by clicking on “Deploy” button at the top right. Then on the right panel check the “debug” tab and you should see the data similar to this:

2/4/2017, 9:21:24 AM  deb-temp
iosynth/device : msg.payload : string [35]
{"ts":1486192884335,"temp":35.0646}

2/4/2017, 9:21:34 AM  deb-temp
iosynth/device : msg.payload : string [35]
{"ts":1486192894335,"temp":22.0076}

2/4/2017, 9:21:44 AM  deb-temp
iosynth/device : msg.payload : string [35]
{"ts":1486192904335,"temp":19.0218}

Add a new “function” node after “mqtt” node and use the following code to configure it:

var p = JSON.parse(msg.payload);
return { payload: p.temp };

This will extract the “temp” value from the Json payload.

If you now attach a “debug” node after the “function” node, you will see the values in “debug” tab.

After, the “function” node add a “chart” node from the “dashboard” palette.

Use your preferences for the chart and deploy the new flow.

If everything is configured properly, you can see the chart on the dashboard by navigating to: http://localhost:1880/ui

temperature

Below is a flow configuration that can be imported using the menu at the top right corner of Node-RED editor.

[
	{
		"id": "41037743.f5cfc8",
		"type": "mqtt in",
		"z": "f940e6ae.d2f8c",
		"name": "temp",
		"topic": "iosynth/device",
		"qos": "2",
		"broker": "38b3d460.36a3fc",
		"x": 121.5,
		"y": 308,
		"wires": [
			[
				"192881c1.87132e"
			]
		]
	},
	{
		"id": "aa3097d9.6c5658",
		"type": "debug",
		"z": "f940e6ae.d2f8c",
		"name": "deb-temp",
		"active": true,
		"console": "false",
		"complete": "payload",
		"x": 290.5,
		"y": 414,
		"wires": []
	},
	{
		"id": "192881c1.87132e",
		"type": "function",
		"z": "f940e6ae.d2f8c",
		"name": "fun",
		"func": "var p = JSON.parse(msg.payload);\nreturn { payload: p.temp };",
		"outputs": 1,
		"noerr": 0,
		"x": 283.5,
		"y": 198,
		"wires": [
			[
				"167d88c.c16b177",
				"aa3097d9.6c5658"
			]
		]
	},
	{
		"id": "167d88c.c16b177",
		"type": "ui_chart",
		"z": "f940e6ae.d2f8c",
		"name": "temp",
		"group": "57fecc7.66fcf34",
		"order": 0,
		"width": "14",
		"height": "8",
		"label": "temp",
		"chartType": "line",
		"legend": "false",
		"xformat": "%H:%M:%S",
		"interpolate": "step-after",
		"nodata": "N/A",
		"ymin": "20",
		"ymax": "40",
		"removeOlder": "400",
		"removeOlderUnit": "1",
		"x": 473.5,
		"y": 198,
		"wires": [
			[],
			[]
		]
	},
	{
		"id": "38b3d460.36a3fc",
		"type": "mqtt-broker",
		"z": "",
		"broker": "iot.eclipse.org",
		"port": "1883",
		"clientid": "",
		"usetls": false,
		"compatmode": true,
		"keepalive": "60",
		"cleansession": true,
		"willTopic": "",
		"willQos": "0",
		"willPayload": "",
		"birthTopic": "",
		"birthQos": "0",
		"birthPayload": ""
	},
	{
		"id": "57fecc7.66fcf34",
		"type": "ui_group",
		"z": "",
		"name": "Default",
		"tab": "cc61ceff.378e",
		"disp": true,
		"width": "14"
	},
	{
		"id": "cc61ceff.378e",
		"type": "ui_tab",
		"z": "",
		"name": "Home",
		"icon": "dashboard",
		"order": "1"
	}
]