OPC UA ETL: What It Is and How OPC Data Wrangler Does It
OPC UA ETL means extracting data from OPC UA servers, transforming it into the units, names and shapes your other systems expect, and loading it to a destination. OPC UA gets the data moving. ETL makes it usable.
Why OPC UA needs ETL
OPC UA solved the protocol problem: secure, cross-platform access to industrial data without COM/DCOM. It did not solve the data problem. A PLC publishes Temperature_Tank1 in Celsius. The SCADA wants Plant1.Temp.Tank01 in Fahrenheit. A European pressure sensor reports Bar while the rest of the plant works in PSI.
Somebody has to bridge those differences. Left to hand-written scripts, the conversion factors get hard-coded and nobody reviews them. ETL is the discipline of doing that bridging deliberately, in one visible place.
Extract: reading from OPC UA servers
The extract step connects to one or more OPC UA servers, browses the address space, and selects the tags you care about. You read values from the live server, for example from different PLCs or vendors, and feed them into the pipeline.
Extraction is usually the easy part. The questions worth asking up front are which servers you need to read, which tags matter, and what each tag's unit and naming convention is.
Transform: units, formulas and Python
Most OPC UA transformations fall into three levels.
Level 1: unit conversion
Bar to PSI, Celsius to Fahrenheit, L/min to GPM. The math is simple and the risk is high, because manual conversions are easy to get wrong. A built-in, validated set of conversions removes the guesswork.
Level 2: formula evaluation
Scaling, ratios and compensation, such as temperature-compensating a gas flow reading:
compensated_flow = flow * (273.15 + temperature) / 293.15Level 3: custom logic in Python
Smoothing noisy sensors, range checks, outlier detection and anything that needs real programming. For example, a 10-sample moving average:
recent_values = []
def transform(sensor_reading):
recent_values.append(sensor_reading)
if len(recent_values) > 10:
recent_values.pop(0)
return sum(recent_values) / len(recent_values)Load: writing to OPC UA servers
The load step writes the transformed values to their destination, mapping source tags to destination tags and setting write intervals. With OPC Data Wrangler the destination is an OPC UA server, so your SCADA or other OPC UA clients read clean, normalized tags without knowing anything about the original sources. CSV import and export is also available.
A typical OPC UA ETL pipeline
Consider three pressure sensors from three vendors reporting in Bar, PSI and kPa, with a SCADA that expects PSI everywhere:
- Source: read all three tags from their OPC UA servers.
- Transform: convert Bar to PSI, pass PSI through, convert kPa to PSI.
- Sink: write the three normalized tags to the OPC UA server the SCADA reads.
Drawn as a directed acyclic graph (DAG), that flow is visible at a glance: sources on the left, transforms in the middle, sinks on the right.
Ways to approach OPC UA ETL
| Approach | Strength | Trade-off |
|---|---|---|
| Custom scripts | Total flexibility | Logic is hidden in code, hard to review, and owned by whoever wrote it |
| Config-file based ETL tools | Repeatable deployments | You read config syntax rather than seeing the data flow; debugging means digging through logs |
| Visual DAG-based ETL | The pipeline is the documentation; failures are easy to locate | Best suited to flows that map cleanly to nodes, with code for the exceptions |
How OPC Data Wrangler does it
OPC Data Wrangler is an OPC UA ETL tool and data broker built on a visual DAG editor. You drag nodes onto a canvas and connect them:
- OPC Source connects to an OPC UA server so you can browse and select tags to read.
- Unit Conversion covers 11 industrial unit categories with over 50 units.
- Formula Eval evaluates mathematical expressions with named variables.
- Python Transform handles complex logic with the Python standard library.
- OPC Sink writes the results to an OPC UA server.
- CSV Source and Sink import and export data.
The common cases are covered visually, and Python handles the rest, so you are not locked into a proprietary scripting language. The OPC Data Wrangler Copilot can also build a DAG from a plain-English description of the pipeline, which you then adjust before deploying.
OPC UA is available now. Modbus TCP, MQTT, Ethernet/IP and REST API integration are coming soon. For a walkthrough with screenshots of each node, read The OPC UA ETL Problem Nobody Talks About, or see how the OPC UA broker fits around it. Unfamiliar with a term? Check the OPC UA ETL glossary.
FAQ
What is OPC UA ETL?
OPC UA ETL is the process of extracting data from OPC UA servers, transforming it (unit conversion, formulas, smoothing, renaming and validation), and loading the result to a destination such as another OPC UA server.
Does OPC UA already convert units?
OPC UA standardises how data moves between systems. It does not decide which units your SCADA expects or apply conversions for you. Different vendors publish different units and tag names, so the transformation step is still yours to build.
Do I need to write code to transform OPC UA data?
Not for common cases. Unit conversion and formula nodes are configured visually. When you need custom logic, such as a moving average, you can write a Python transform.
What is a DAG in an OPC UA ETL tool?
A directed acyclic graph (DAG) is a flow diagram of your pipeline: sources feed transforms, transforms feed sinks, and data never loops back. It makes the whole pipeline visible and doubles as documentation.
Which protocols does OPC Data Wrangler support?
OPC UA is available now. Modbus TCP, MQTT, Ethernet/IP and REST API integration are listed as coming soon.