The project demonstrares real time plotting of sensor data using MQTT. A Rasperry Pi functions as a MQTT broker listening to a specified topic, and the concurrent (using threads) python code logs incoming data to a csv file and concurrently plots the data in real time with matplotlib.
All the code and examples are at smaroukis/realtime-plotter.
- Setting up a mosquitto MQTT broker on the RasPi
- Creating a Plotter wrapper class for the
- edge device code in C++/Arduino on an ESP32 to take sensor readings and send as MQTT messages to the broker
- Python code to handle multiple concurrent processes using the threading library, as well as using the
funcanimatefunction to update a plot on the fly
- Concurrently saves the data in csv or json to the disk
Demo video using a temperature sensor and covering it with my finger to affect the reading:
Real time data logging to csv and concurrent plotting with matplotlib, creating an mqtt client to listen to the specified topic on a specified mqtt broker/host.
Logger is under
raspi/app_logger/app.py. For example:
python raspi/app_logger/app.py -t # -d -h 192.168.0.17 -g
i.e. "all topics ("#"), for mqtt broker on host 192.168.0.17, include plotting (-g)
Seperate threads and callbacks are made for plotting and writing to the datafile
log_workerempties the data-to-log queue every 2 seconds
matplotlib.animate.funcanimationfunction that updates the graph every provided
interval(also has a queue to process data)
Esp32 code for sensor and mqtt client/publishing is also included.
Setting Up the ESP32 sensor
A BME temperature sensor is used over Two-Wire/I2C.
Setting up the RasPi/MQTT Broker
On local machine:
# use SSH from a terminal to get into the Pi server
ssh [email protected]
# ... enter user/pass ...
In a server terminal (or after SSHing in):
# print IP address (should match the one in `secrets.h`)
# check the `mosquitto` status:
systemctl status mosquitto.service
# ... if status is running try:
mosquitto_pub -d -t inTopic -m "ON"
# which should turn the builtin on the ESP32 ON if it is OFF
If necessary restart the mosquitto service with
sudo systemctl restart mosquitto
If we want to see the
outTopicresponses from the ESP32 we can open up another terminal on the MQTT broker and subscripe to this topic e.g.
mosquitto_sub -d -t outTopic
Or to subscribe to all topics do:
mosquitto_sub -d -t "#"
- the mosquitto broker has to be configured to accept anonymous connections (disabled by default)
- put the ESP32 in "flash" mode by (1) pressing and holding BOOT (2) pressing and releasing ENABLE (3) releasing BOOT
Code Description - ESP32
- publishes values to topic as e.g.
- was implemented using C style strings, should probably be refactored to smart pointers and std library strings
- currently implemented publishing from a bme280 temperature and humidity sensor (via Adafruit libary)
Establishes a connection to an MQTT broker, subscribes to specific topics, and publishes sensor data to MQTT topics. including potential buffer overflow issues when dealing with large float values, minimal error handling, and limited documentation. Additionally, it lacks configurability for MQTT client parameters, making it less flexible for different MQTT broker setups
Need to add more error handling and confirugability for different MQTT client parameters (e.g. name, etc).
Sets up wifi with the