From Raw Data to the Dashboard: Thoughts on an Industrial Project

Johannes Falk on May 19, 2026 · 6 min read

For a leading manufacturer of high-precision analytical equipment, I built the entire workflow, from analyzing measurement data and consolidating and storing it in a database to creating a dashboard for rapid visualization. Here, I explain the technical decisions I made during the process; research code - my usual area of ​​expertise - differs significantly from production code.

For a leading manufacturer of high-precision analytical equipment, I built a system that imports, makes accessible, and analyzes complex device diagnostics and measurement logs: a database for measurement logs and supplier information, an API on top of it, and a dashboard that allows managers to view this information at a glance and analyze it statistically.

The project actually began in a different context. The original assignment was a data analysis with several objectives:

• Parameter Optimization: High-precision instruments rely on complex, interdependent calibration settings. Analyzing historical operational logs helps identify optimal operating conditions faster and more reliably.

• Root-Cause Diagnostics: Complex hardware platforms consist of numerous interacting sub-components. Systematic data analysis makes it easier to trace error conditions back to specific physical causes rather than relying on trial-and-error.

• Quality & Cost Control: By correlating component manufacturing tolerances with final device performance, the team can pinpoint exactly where strict specifications are necessary and where standard tolerances suffice.

In the course of the project, it became clear that the real hurdle was not the algorithm, but access to the data: The data was available, but not in a form that allowed for systematic comparison and analysis across many instruments. As is often the case with projects that have evolved heterogeneously and been merged, the various data sets were stored in different databases and in different formats. A significant challenge was handling legacy telemetry data stored across fragmented file systems, where schema drift over several hardware and software generations had introduced inconsistencies.

Based on an ER model developed in collaboration with domain experts, I consolidated all data sources into a relational database. I then created a dashboard to facilitate clear analysis. In the long term, this serves as the foundation for two things: better optimization of the devices and reduced maintenance costs, because errors can be pinpointed more precisely rather than narrowed down by process of elimination.

The entire workflow from a single developer

What made this project so extraordinary and exciting for me wasn’t so much the individual components, but rather the breadth: from the merging of the data and the associated data model, through statistical analysis and the API, all the way to the frontend. These areas are typically handled by three or four different experts. However, important information is often lost at each interface.

However, someone who understands how a measurement is performed and which physical quantities are truly comparable will make different decisions regarding where data is stored in the database and how it is normalized - as well as what the dashboard ultimately displays side-by-side. A frontend that overlays two incorrect curves may be technically flawless, but it is useless.

The Technical Decisions

For the database, I used SQLModel, an abstraction layer on top of SQLAlchemy. An SQLModel model is also a Pydantic model. So the same class definition describes both the table schema in the database and the validation schema in the API. This means you only have to maintain one definition instead of two.

This allows FastAPI to work right away. FastAPI validates incoming data internally via Pydantic and automatically generates an OpenAPI schema from the type annotations. From this schema, in turn, a typed client for the frontend can be generated. If a field changes in the backend, the TypeScript compiler in the frontend flags it during the next build: errors are shifted from runtime to compile time.

The frontend itself is implemented in React, using Vite as the build tool and shadcn/ui for the components. For the basic framework, I closely followed the official Full-Stack FastAPI Template. This was a deliberate decision against building my own: A developer who joins the project later will find a project structure they’re familiar with or can look up in the documentation. A custom, potentially more elegant structure would have sacrificed this advantage.

What Sets Research Code Apart from Production Code

Research code has a straightforward purpose: It must compute correctly, the result must be reproducible, and then it goes into the archive or into the repository as an appendix to a publication. Throughout, I am usually the only reader. No team pushes to the same repository, no one else needs to familiarize themselves with my algorithms, and no one but me runs the analysis. A terminal command whose arguments only I know is perfectly sufficient as long as I’m the only one who needs it.

When developing code together with a client, what also matters is whether an IT specialist looking at it for the first time can quickly figure out what each function does and where to start. Everything related to handoff thus becomes a measure of quality: naming conventions, project structure, data types, documentation, and the question of whether a dependency will still be maintained two years from now.

This isn’t a higher requirement, but a different one. However, it’s one that rarely arises in research, and that’s why it was the part from which I learned the most.

Why I Chose Boring Technology

Here’s an example of how this difference between Research Code and Production Code affects decision-making.

A few years ago, I implemented a research project entirely in Julia. The speed advantage in the simulations was impressive, and that was the first time I worked with multiple dispatch (and I am still not sure if I don’t actually prefer this concept to classic object-oriented programming). The cost, however, was noticeable: Many of the packages I needed were still under development, interfaces changed between versions, and I had to rewrite my code several times.

For a research project, this trade-off is justifiable. I was the only one affected, the time frame was manageable, and the speed gain directly accelerated my work. For a system that’s going to be delivered and then operated by others for years to come things are different.

That’s why my stack for the industry project is deliberately unspectacular: widely used, well-documented, with a large community, and a foreseeable lifespan. My experience with Julia was a prerequisite for this well-considered choice. You have to have paid the price of “bleeding edge” yourself to know when it’s worth it (or not).