MQTT Integration

Is there the possibility to integrate a MQTT broker on the server side?
It could be interesting for IoT dashboards. I would like to publish and subscribe to topics like a client, using a centralized broker on the server (like Ubidots, for example)

Hello Ruky87, welcome to the forums and to this great Anvil community.
I am no admin here, but as long as there is a python package that can be installed server-side, admins are usually very happy to install it for you to use, if you’re a paying customer.
Another choice is to install and use those packages yourself in an uplink-ed server of your choice.
I see there are many clients, like paho-mqtt and all those listed in https://github.com/mqtt/mqtt.github.io/wiki/libraries under section Python

1 Like

Yes, I usually use paho, and I know I can use uplink and create a connector. I could also create a converter API<->MQTT but a third VPS is needed in both cases.
Suppose I am interested in a DIRECT communication from an IoT device based on MQTT and I can use only MQTT for communication (that is also energy efficient and has some useful features used with low power devices): if I want to connect to a broker I need an IP and a PORT and a topic “root” where i can publish or subscribe to. I think it is impossible with the Free or Individual/Business plans because there is no dedicated server in this case. Even if I could install paho package, a unique IP:PORT is needed.

The MQTT connector should be created by Anvil team, like the API connector. So then they can provide a MQTT port and a secret-key used to identify the user. The broker url and port will be the same for many/every user without the dedicated server. Then each user can manage its topics with some server side callbacks (like with the APIs), put data in the database, create logging system, simple analytics, triggering actions (eg. Telegram Bot with telepot). With the integrated plotly chart system and the possibility to create personalized widgets, this can be a game changer in the Industry.
In this way Anvil can compete with Thingsboard, Ubidots, and also the Adafruit IO dashboard used with Arduino Educational Kits. The difference is that Anvil App creator can be used to create everything from scratch without the limits of the cited platforms, usually limited to IoT sensor data monitoring/control.

The only missing thing will be a plan between Free and Individual, something like 10-15 $/month max (. With some limitation, but with python 3 instead of 2 maybe… Something that a maker or a small IoT company can use to create PoC, demonstrators… Where full python is needed, but custom domain not for example.

Sure, Anvil could expose a more generic TCP/IP connector just like they expose an HTTP one.
But I still think that uplinked server could be your saver here, if you need to proceed in the meanwhile.
If the uplinked server is exposed to the IoT devices (the clients) then you can run there a TCP/IP server and make clients connect to it.
A scheme like this:


where Anvil takes care of “blue” components, while you have to manage red components yourself.
Since I know nothing about MQTT I just built a generic TCP server listening on port 100000 (adapted code from https://pymotw.com/2/socket/tcp.html actually) on localhost, a simple client sending text, and an anvil app that will read the data sent from the client to the server, just to prove the basic reasoning I am doing here.

Sure, in this example client and servers communicate on localhost but that is not relevant when you server code runs on a real internet-exposed server.

Here is what I get.

  1. start the server
    cmd_2019-09-19_12-22-56

  2. check the data with the Anvil App

  3. start the client
    cmd_2019-09-19_12-23-17

  4. server receives data via the “red” network
    cmd_2019-09-19_12-23-27

  5. Anvil App reads data too, via the “blue” network

Does it make sense? Or I am completely out of scope here?

Follows server code, client code, App code and Anvil CopyApp link.

Server Code
=======================

import anvil.server
import socket
import sys
import time

anvil.server.connect('your app secret key',)

total_received_data = []

@anvil.server.callable
def get_received_data():
    return total_received_data

# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

server_name = 'localhost'   # user sys.argv[1] to bind the socket to the address given on the command line
server_address = (server_name, 10000)
print('starting up on %s port %s' % server_address, file=sys.stderr)
sock.bind(server_address)
sock.listen(1)

while True:             # instead of anvil.server.run_forever()
    time.sleep(1)
    print('waiting for a connection', file=sys.stderr)
    connection, client_address = sock.accept()
    try:
        print('client connected:', client_address, file=sys.stderr)
        total_received_data = []
        while True:
            data = connection.recv(16)
            print('received "%s"' % data, file=sys.stderr)
            if data:
                total_received_data += data
                connection.sendall(data)
            else:
                break
    finally:
        connection.close()

Client Code
=======================

import socket
import sys


def get_constants(prefix):
    """Create a dictionary mapping socket module constants to their names."""
    return dict((getattr(socket, n), n)
                for n in dir(socket)
                if n.startswith(prefix)
                )


families = get_constants('AF_')
types = get_constants('SOCK_')
protocols = get_constants('IPPROTO_')

# Create a TCP/IP socket
sock = socket.create_connection(('localhost', 10000))

print('Family  :', families[sock.family], file=sys.stderr)
print('Type    :', types[sock.type], file=sys.stderr)
print('Protocol:', protocols[sock.proto], file=sys.stderr)
print(file=sys.stderr)

try:

    # Send data
    message = 'This is the message.  It will be repeated.'
    print('sending "%s"' % message, file=sys.stderr)
    sock.sendall(bytearray(message, 'utf-8'))

    amount_received = 0
    amount_expected = len(message)

    while amount_received < amount_expected:
        data = sock.recv(16)
        amount_received += len(data)
        print('received "%s"' % data, file=sys.stderr)

finally:
    print('closing socket', file=sys.stderr)
    sock.close()

APP client code
=======================

from anvil import *
import anvil.server

class Form1(Form1Template):

  def __init__(self, **properties):
    # Set Form properties and Data Bindings.
    self.init_components(**properties)

    # Any code you write here will run when the form opens.

  def button_1_click(self, **event_args):
    """This method is called when the button is clicked"""
    recv_bytes = anvil.server.call('get_received_data')
    if len(recv_bytes)>0:
      self.text_box_1.text = "".join(map(chr, recv_bytes))
    else:
      self.text_box_1.text = 'None'






CopyAPP link
https://anvil.works/build#clone:4XNN3PR4KYSBJOA3=U2Y72NOCDD2RPJP6YZ52NFJN

2 Likes

Yes, this is a good solution, I will use an architecture like this in the meanwhile.
The MQTT feature request will be still useful, you can connect directly to Anvil without using a dedicated gateway or a VPS just for logging data and create simpler dashboards.

Is there somewhere a list of the modules available with the Python 2 free plan? I found only the packages for the python 3 plans.

However, Anvil can be a game changer as IoT web App development platform in my opinion

2 Likes

I’m glad this works for you! And many thanks to @aldo.ercolani for such a detailed solution!

The recommended way to use third-party libraries on the Free Plan is to use the Uplink and install them on your own machine.

I implemented some sample code: MQTT sample code