Implementation of a centralized system for data transfer and monitoring of FLAME stations

Abstract

This technical report details the architecture and implementation of a centralized client-server platform, “File Transfer,” designed for the secure transfer and real-time monitoring of data files from remote stations. The need for a unified system arises from the requirement to automate the data acquisition pipeline, guarantee file integrity during transfer (via hash verification), and provide personnel with a single interface (dashboard) for network health monitoring and configuration.
The implemented solution is a Python-based application, featuring a central Flask web dashboard and a lightweight FastAPI server on each remote station. The system utilizes a SQLite database with monthly rotation for persistent logging and status tracking. The system utilizes a central background service that employs a multi-threaded ThreadPoolExecutor for parallel and continuous data acquisition from all configured stations.
Key features include:
1. A “live” dashboard with KPIs for network health (Online, Error, Offline) and daily transfer statistics.
2. A secure transfer protocol using SHA-256 hash verification to ensure zero data corruption.
3. An automatic anomaly detection system based on a consecutive error counter to identify and flag unresponsive stations.
4. A dynamic storage architecture that sorts received files based on pre-configured location tags (Etna, Stromboli, Vulcano) and the file’s original date (format yymmdd).
5. A flexible configuration system, including a per-station toggle (checkbox) to switch between “Copy” (test) and “Move” (production) modes.