Introduction
In the moving world we live in today, real-time data has become essential for businesses and organizations to stay ahead of the competition. Whether it's tracking live financial markets or monitoring sensor data in industrial applications, the ability to visualize and respond to data in real time provides a significant advantage.
In this tech bite, we will explore how Socket.IO can seamlessly stream live data directly into Plotly, enabling us to create interactive real-time visual experiences. We’ll demonstrate this through a simulation of temperature sensor data, illustrating how real-time updates can deepen our understanding of environmental changes and improve the effectiveness of our visualizations.
What Are Socket.IO and Plotly?
Socket.IO is a powerful library designed to enable real-time, two-way communication between client and server, in contrast to traditional HTTP requests, where the client initiates the communication and waits for a response from the server. It operates on top of the WebSocket protocol while adding additional features, such as automatic reconnection, multiplexing, and handling network interruptions.
These capabilities make it ideal for real-time applications like chat systems, live data feeds, and collaborative tools, where continuous communication and instant updates are required. By using Socket.IO, we can ensure that our visualizations stay up-to-date as new data streams in without requiring the user to refresh the page.
Plotly is an open-source library that allows developers to create highly interactive, visually appealing data visualizations with minimal effort. With support for a wide range of chart types—including line charts, bar graphs, scatter plots, and even 3D visualizations—Plotly is a powerful tool for turning raw data into meaningful insights. What sets Plotly apart is its ability to handle real-time data, making it an ideal choice for live dashboards and dynamic charts. Whether you're building complex visualizations or simply adding interactivity to a basic graph, Plotly provides the flexibility and tools to meet your needs.
Socket.IO and Plotly together create a powerful combination for real-time data visualizations. Socket.IO handles the continuous data flow between server and client, while Plotly transforms that data into rich, interactive charts that respond in real-time. This seamless integration allows users to track live data streams without refreshing the page, making it ideal for applications where timely updates are critical, such as stock market dashboards, live event monitoring, and real-time analytics tools. By the end of this tech bite, you’ll be able to create live-updating charts using Socket.IO and Plotly and optimize them for performance and smooth user experience.
Setting Up the Environment
Prerequisites
To build real-time visualizations with WebSockets and Plotly, we need to ensure we have the following tools and libraries set up:
Python (for the backend)
Flask (to create the web server)
Flask-SocketIO (to handle WebSocket communication)
Flask-CORS (to manage Cross-Origin Resource Sharing)
Plotly (for interactive visualizations)
Eventlet (for asynchronous operations)
JavaScript (for frontend logic and WebSocket integration)
HTML/CSS (for the frontend structure and styling)
Installation Commands
We can install the required libraries using pip:
pip install flask
pip install flask-socketio
pip install flask-cors
pip install eventlet
For Plotly, we can include it via CDN in our HTML, so no additional installation is needed.
Setting Up the Basic Project Structure
Create the following directory structure for your project:
real-time-plotly-app/
│
├── app.py
├── static/
│ ├── script.js
│ └── style.css
└── templates/
└── index.html
Backend (Flask and Socket.IO Server):
Create a file named app.py in the project folder. This file will configure Flask, Flask-SocketIO, and Flask-CORS to handle SocketIO connections and real-time data updates.
Frontend (Plotly Visualization):
HTML File: Create a file named index.html inside the templates/ directory. This file will define the structure of your web page and include references to Plotly and Socket.IO libraries.
JavaScript File: Create a file named script.js inside the static/ directory. This file will handle the SocketIO connection and update the Plotly chart with real-time data.
CSS File: Create a file named style.css inside the static/ directory to style your web page.
With this setup, you’ll be ready to create and visualize real-time data using SocketIO and Plotly.
Creating a Flask Server and Setting Up a Socket Connection
In this section, we'll begin by setting up a simple Flask server that enables real-time communication using Flask-SocketIO.
Flask is a lightweight web framework that will serve our web pages and handle API requests. By adding Flask-SocketIO, we enhance Flask with support for real-time communication, which allows data to be exchanged between the client and server instantly—without needing to refresh the page.
Flask-SocketIO uses WebSockets as its primary method for real-time communication. WebSockets enable full-duplex (two-way) communication between the server and client. If WebSockets are not available, Flask-SocketIO can seamlessly switch to alternative methods, such as HTTP long-polling, to maintain real-time functionality.
By combining Flask and Flask-SocketIO, we can build applications that update dynamically, ensuring users get the latest data without delay.
from flask import Flask, render_template
from flask_socketio import SocketIO
from flask_cors import CORS
app = Flask(__name__)
CORS(app) # Allows Cross-Origin Resource Sharing
# Initialize SocketIO with the Flask app
socketio = SocketIO(app, cors_allowed_origins="*")
# Serve the main page
@app.route('/')
def index():
return render_template('index.html')
# Handle client connection
@socketio.on('connect')
def connect():
print(f'Client connected: {request.sid}')
socketio.emit("Connected", {'message': 'Welcome!'}, to=request.sid)
# Handle client disconnection
@socketio.on('disconnect')
def disconnect():
print(f'Client disconnected: {request.sid}')
if __name__ == '__main__':
socketio.run(app, debug=True, host='127.0.0.1', port=5000)
Let’s explain this code in detail:
First, we need to import Flask, Flask-SocketIO, and Flask-CORS. Flask-CORS ensures that your web app can access the server even if it’s hosted on different origins.
The SocketIO(app) method wraps your Flask app to enable real-time communication using WebSockets.
@socketio.on('connect') decorator listens for when a client connects to the server and logs a message to the console.
@socketio.on('disconnect') decorator listens for when a client disconnects.
The server is started using socketio.run() with the host set to 127.0.0.1 and port 5000.
At this point, the server is ready to handle WebSocket connections, but we haven't yet sent any real-time data. That will be covered in the next section.
Simulating Sensor Data with Random Value Generation
Now that we have established the server and WebSocket connections let's simulate real-time sensor data by generating random values.
To make the example more interesting, we will simulate a real-time temperature sensor by generating random values and sending them to connected clients using WebSocket messages. Here’s how we’ll do it:
from random import uniform
from datetime import datetime
from threading import Lock
import time
# Global variables
thread = None
thread_lock = Lock()
last_value = None # To keep track of last emitted value
# Function to get current date and time
def get_current_datetime():
now = datetime.now()
return now.strftime("%m/%d/%Y %H:%M:%S")
# Function to generate random values and send them to the client
def background_thread():
print("Starting background thread for random value generation")
# Start with an initial value
current_value = round(uniform(16.0, 26.0), 2)
while True:
# Generate a random change between -1.0 and 1.0
change = round(uniform(-1.0, 1.0), 2)
# Update the current value within the range 16.0 - 26.0
current_value = max(16.0, min(26.0, current_value + change))
# Get the current time
current_time = datetime.now().strftime("%m/%d/%Y %H:%M:%S")
print(f"Generated value: {current_value} at {current_time}")
# Send the data to the client via WebSocket
socketio.emit('updateValue', {'value': current_value, 'time': current_time})
# Sleep for 1 second before generating the next value
time.sleep(1)
How do we get those random generated values?
We use Python's random.uniform() to generate random values between 16.0 and 26.0 (usual room temperature range) to simulate sensor data.
The current_value is updated in each iteration by adding a random change between -1.0 and 1.0, ensuring the value remains between 16.0 and 26.0.
After the value is generated, the function retrieves the current date and time using datetime.now() to include a timestamp with each value.
The generated data is emitted to the client using socketio.emit() every second.
To keep generating data continuously, we run this code in a background thread using socketio.start_background_task(), which we will initialize when a client connects.
Now that we have our Flask server, WebSocket connection, and sensor data simulation, let's combine everything and create the real-time visualizations with Plotly on the frontend.
Integrating the Background Thread
We need to ensure the background thread starts as soon as a client connects.
# Handle client connection
@socketio.on('connect')
def connect():
global thread
print(f'Client connected: {request.sid}')
# Start background thread if it's not already running
with thread_lock:
if thread is None:
thread = socketio.start_background_task(background_thread)
# Optionally, send some initial data or message on connect
socketio.emit("Connected", {'message': 'Welcome! You are now connected.'}, to=request.sid)
Frontend Setup with Plotly
In the frontend, we’ll use Plotly to visualize the incoming data from the WebSocket connection.
Firstly, inside an index.html file, we’ll make a structure of our page that will contain a Plotly chart:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Real-Time Sensor Data</title>
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js"></script>
</head>
<body>
<h1>Real-Time Temperature Sensor Data</h1>
<div id="plotly-chart" style="width: 100%; height: 500px;"></div>
<script type="text/javascript" src="{{ url_for('static', filename = 'script.js') }}"></script>
</body>
</html>
In the JavaScript part of our application, we will integrate Plotly to create a dynamic, real-time line chart. Plotly is an extremely versatile and powerful library for generating interactive visualizations. Here’s a more detailed look at how we use Plotly to display the real-time data streamed from the Flask-SocketIO server:
Setting Up the Plotly Chart
The first step is initializing an empty chart where the data will be plotted:
Plotly.newPlot('plotly-chart', [{
x: [],
y: [],
mode: 'lines+markers',
type: 'scatter'
}], {
title: 'Real-Time Data',
xaxis: {
title: 'Time',
automargin: true
},
yaxis: {
title: 'Value',
}
});
The Plotly.newPlot() function is used to create a new chart. It takes three arguments: the ID of the HTML element where the chart will be rendered, the data (as a list of traces), and layout options.
The x and y arrays are initialized as empty, and the data will be filled as real-time values arrive.
The mode: 'lines+markers' ensures that both a line and points are shown on the graph.
We also specify axes titles to give context to the data (Time on the x-axis, Value on the y-axis).
WebSocket Event Handling
Once the chart is initialized, we need to handle real-time data updates that come through WebSockets:
socket.on('updateValue', function (data) {
console.log("Received updateValue:", data);
// Add new data point
times.push(data.time);
values.push(data.value);
// Keep only the last 20 values
if (times.length > 20) {
times.shift();
values.shift();
}
// Update Plotly chart
Plotly.update('plotly-chart', { x: [times], y: [values] });
});
The socket.on('updateValue', function (data)) listens for the updateValue event emitted by the server. This event carries the real-time data (data.value) and the timestamp (data.time).
Each new data point is appended to the times and values arrays.
To keep the chart concise and focused on recent data, we ensure that only the last 20 data points are kept by using the shift() method, which removes the oldest data point once we exceed this limit.
Finally, the Plotly.update() method is called, which efficiently updates the chart with the new data without re-plotting the entire graph.
Efficient Real-Time Updates with Plotly
One of the key features of Plotly is its efficient update capabilities. By using Plotly.update() rather than recreating the entire chart each time new data arrives, we optimize performance and ensure the chart remains responsive. This approach is crucial when dealing with high-frequency data streams, as it minimizes the load on the browser and keeps the user experience smooth.
You can further customize the look and behavior of the chart by modifying the layout options, adding annotations, or even switching to different chart types (e.g., bar charts, heatmaps, etc.). The flexibility of Plotly makes it an excellent choice for real-time applications like this one.
In the snippet below, you can see the real-time output of our app, demonstrating how data is dynamically visualized and updated every second.
Conclusion
In this tech bite, we walked through creating real-time data visualizations with Flask-SocketIO and Plotly. We set up a Flask server to handle real-time communication and simulated live sensor data, and we used Plotly to dynamically update charts on the client side.
This combination of Flask-SocketIO for smooth real-time data flow and Plotly for interactive visualizations, although simple, opens up a number of possibilities. Whether you're working on financial dashboards to track market trends, monitoring sensor data in real-time for industrial or environmental applications, or displaying live updates from events, this setup provides a powerful foundation.
Looking ahead, there are several ways to enhance and expand this system:
Security: Implement secure WebSocket connections (WSS) and consider adding authentication to safeguard your data.
Scaling: Explore strategies to manage a large number of connections and distribute the load across multiple servers.
Advanced Data Processing: Integrate with data processing frameworks to handle more complex transformations and analyses before visualizing the data.
With this foundation, you’re ready to explore more advanced features and tailor the system to your specific requirements. Play around with different data sources, refine your visualizations, and continue pushing the limits of what you can achieve with real-time data.
"Real-Time Data Visualisation with Plotly and Socket.IO" Tech Bite was brought to you by Elma Turčinović, Data Analyst at Atlantbh.
Tech Bites are tips, tricks, snippets or explanations about various programming technologies and paradigms, which can help engineers with their everyday job.
![Figure 1 - Bandwidth of human senses [T.Nørretranders, “The User Illusion: Cutting Consciousness Down to Size”, 1999]](https://www.atlantbh.com/wp-content/uploads/2023/03/Bandwidth-of-Human-Senses.png)
