Direct Answer
You have just taken over an energy platform. How do you quickly judge whether this batch of data can be fed directly to the algorithms, which parts need cleaning, and which parts should stay? First identify the failure mode of the dirty data, then decide how to clean it — packet loss appears as a time hole, jumps are isolated wild points, dead values are long-term unchanging readings, timestamp corruption is reversed order, and unit drift is a sudden magnitude change; the five modes differ in signature, cause and treatment — judge the mode first and then prescribe the medicine, and washing them all together will only make the data dirtier and messier. The digital electrical intelligent supervision system puts mode identification into the first step of the data-cleaning pipeline: judge the mode first, then clean.
A scene: a factory installed an energy-monitoring system and fed the raw data directly to the consumption-analysis algorithm — the report said "the switch room consumes 0 energy" while the site was clearly running. Investigation: the data was entirely missing from 2:00 to 4:00 a.m. (gateway restart), and the algorithm treated "no data" as "zero consumption". Without first identifying the "loss hole", the algorithm treats the hole as a real value — cleaning dirty data without classifying it is the same as not cleaning it at all.
A second scene: a park's monthly report showed "air-conditioning consumption suddenly down 30%", and management was about to praise it — the operations team checked the raw data and found that one measuring point's unit had drifted from kWh to Wh, shrinking the value by 1000 times — the "30% energy saving" was an illusion created by unit drift. With the same data, if the mode is not identified: loss becomes zero and drift becomes savings — without judging the mode, every conclusion is an illusion.
1. Packet Loss: A Time Hole
Packet loss appears as a whole period of missing data, forming a "hole":
Communication interruption — gateway offline, network flash break: the hole usually lasts minutes to hours, and after recovery the data continues normally, with the values before and after the hole continuous; Device power loss — device loses power: the hole can be very long (hours to days), and during the hole the device "vanishes"; How to tell — check whether the values on both sides of the hole are continuous: continuous means communication interruption (data was not lost, just not transmitted — it can be filled), a jump means device power loss (the outage event itself is valuable — record it as an event).
In one sentence: packet loss is a "hole in time" — first judge whether "the network or the power went down", then decide whether to fill the data or record the event; if the judgment is wrong, what should be filled is not filled and what should be recorded is not recorded.
A packet-loss example: data from a switch room was entirely missing from 2:00 to 4:00 a.m., and the current curve was continuous on both sides of the hole — judged as gateway communication interruption (the data did not come up, not that it did not happen); meanwhile another workshop's data was missing from 14:00 to 16:00, with 300 A before the hole and 0 after — judged as device power loss (an outage event). Both are "holes", but one gets data filling and the other gets an event record — judge wrongly, and the treatment is all wrong.
2. Jumps: Isolated Wild Points
Jumps appear as one or a few points suddenly deviating from the normal range, like "wild points":
Loose sensor — poor probe contact: the jump is usually a single instantaneous anomaly, with the points before and after normal, and the value can be absurd (e.g., current jumping to 10,000 A); Real electrical impact — motor starting, large-load switching: the jump often lasts from several to dozens of points and follows a pattern (starting current is usually 5–7 times rated); How to tell — check whether the jump is "a single isolated point" or "a multi-point segment": a single isolated point with an absurd value is probably a sensor problem; a multi-point segment consistent with physical laws (starting current, inrush current) is a real event.
In one sentence: a jump is an "isolated wild point" — a single point is a fault, a segment is an event; tell them apart first, then decide whether to remove or keep; deleting a real event is more dangerous than keeping a fake wild point.
A jump example: on a busbar current curve, an isolated 8000 A point appeared with 500 A before and after — single, isolated and absurd, judged as a loose sensor probe; on another curve, 20 consecutive points at 2500 A (5 times the 500 A rating) during motor starting matched the starting-current law — judged as a real start event. Both are "large currents", but one is removed and one is kept — judge the mode wrongly, and the real event is washed away while the fake wild point enters the report.
3. Dead Values: Long-Term Unchanging
Dead values appear as the value staying unchanged for a long time, as if "the data froze":
Device hung — the acquisition module froze, communication dead: the value stays at the last reading and no longer updates, lasting minutes to days; Real steady state — the device is stopped but acquisition is normal: the value stays in a reasonable range (e.g., 0 or no-load current), with small fluctuation but not completely static; How to tell — check whether the value is "completely unchanged" or "slightly fluctuating": completely unchanged, with even the decimal digits motionless, over consecutive N points (e.g., 30+ points) is probably a hung device and should raise a "data frozen" alarm; slightly fluctuating and consistent with physical trends is a real steady state that needs no treatment.
In one sentence: a dead value is "frozen data" — completely unchanged means false death, slight fluctuation means true steady state; if N consecutive points (e.g., 30) are unchanged, raise an alarm — do not wait until a fault becomes an accident.
A dead-value example: a meter's current stayed at 320 A, motionless for 40 minutes, while the voltage fluctuated normally in the same period — the acquisition module had hung, the value "frozen" at the last reading; by comparison, another stopped meter's current fluctuated slightly between 0 and 2 A — a real shutdown. Both are "almost static", but one is a false death and one is a real stop — look at "completely unchanged" vs "slight fluctuation", and it is clear at a glance.
4. Timestamp Corruption: Reversed Order
Timestamp corruption appears as data timestamps not matching the real time, with order reversed:
Gateway clock not calibrated — gateway battery depleted, NTP failed: the new data's timestamp may be earlier than the old data, and the two curves "fight"; Collector timezone error — timezone misconfigured (UTC used as local time): all data shifts by 8 hours, and daytime data appears at midnight; How to tell — check whether the timestamps are strictly increasing and match the local time: reversed order means a clock problem (calibrate the clock); a wholesale shift means a timezone problem (fix the configuration).
In one sentence: timestamp corruption is a "misplacement of time" — the data itself is not wrong; the clock was mislabeled, and the analysis is entirely misaligned; re-sorting can only save reversed order, not a wholesale shift.
A timestamp-corruption example: in a park, the harmonic analyzer and the meter clocks differed by 3 minutes, and the analysis report said "harmonics strongly correlate with load" — on-site verification found the two curves were 3 minutes apart, making the correlation a "misaligned correlation", pure coincidence; another batch of data shifted 8 hours as a whole, with daytime data appearing at midnight — the timezone was wrong, using UTC as local time. Reversed order is a clock problem, a shift is a timezone problem — judge wrongly, and even re-sorting cannot save it.
5. Unit Drift: A Sudden Magnitude Change
Unit drift appears as a sudden change in the order of magnitude of the value, while the device itself has not changed:
kWh/Wh mixing — a measuring point's configuration was changed: the same point changes from kWh to Wh, and the value grows 1000 times; A/mA mixing — sensor range misconfigured: current changes from A to mA, and the value grows 1000 times; How to tell — check whether the magnitude changes "as a multiple": an "integer-multiple drift" like exactly 1000 times or 100 times (kWh↔Wh, A↔mA) is ninety percent a misconfigured unit; check the configuration first, then talk about the device.
In one sentence: unit drift is a "magnitude mutation" — for an integer-multiple jump, check the unit configuration first and do not rush to convict the device; change the configuration and the data returns to place immediately.
A unit-drift example: a measuring point's current suddenly changed from 250 A to 0.25 A, with the site device unchanged — investigation found the sensor range configuration had been changed from "A" to "mA", shrinking the value by 1000 times; another point's energy changed from 35 kWh to 35,000 Wh, again exactly 1000 times — the configuration had changed kWh to Wh. Both "1000 times" were configuration problems, not device problems — the signature of unit drift is exactly the "integer-multiple mutation"; on seeing an integer multiple, check the configuration first.
6. Rapid Identification: The Five-Step Comparison
Step 1, look for holes — a whole period missing → packet loss; judge communication vs power loss; Step 2, look for wild points — a single isolated anomaly → jump; judge sensor vs real event; Step 3, look for dead values — long-term unchanged → dead value; judge hung device vs real steady state; Step 4, look for reversed order — timestamps not increasing → timestamp corruption; judge clock vs timezone; Step 5, look at the magnitude — an integer-multiple mutation → unit drift; check whether the configuration was changed.
In one sentence: identify the mode first and then prescribe the medicine — hole, wild point, dead value, reversed order, magnitude; after the five steps, whether the data can feed the algorithm is clear at a glance, and each treatment channel is in place.
A complete five-step example: taking over the data of 200 devices in a park — first scan for holes: 3 devices had holes in the early morning, judged as communication interruption; then look at wild points: 2 isolated jumps, judged as loose sensors; then check dead values: 1 meter motionless for 40 minutes, judged as a hung device; then check reversed order: some timestamps were not increasing, judged as an uncalibrated gateway clock; finally look at magnitude: 1 point exactly 1000 times smaller, judged as unit-configuration drift. The five modes each go to their own category and treatment channel — a 5-minute scan is a hundred times better than blind washing.
A common pitfall — mixed modes: the same curve can have holes and wild points at once, and someone "cuts all with one knife": treating the hole as zero and deleting the real starting current as a wild point — the real event is washed away and the fake data enters the report. The right approach for mixed modes is "segment-by-segment identification": first separate the hole segment from the normal segment, then look at wild points in the normal segment alone — one mode per segment, one dirt cleaned at a time, and only then is it clean.
7. Back to FEXLINK: How Mode Identification Lands
To land this logic on products: the digital electrical intelligent supervision system builds dirty-data mode identification into the data link — before storage, data passes a "mode classifier": packet loss records events, jumps are marked as wild points, dead values raise alarms, reversed order is re-sorted, and drift verifies the configuration; the five modes each take their own treatment channel. Dirty data is not scary; what is scary is "not knowing which kind of dirt it is" — mode identification is the first step of data cleaning and the first gate of data credibility; with the gate held, the algorithms dare to work. During selection, ask "how does your platform identify dirty data" — the one answering "classify and handle the five modes of packet loss, jumps, dead values, timestamps and units" is the platform treating data as engineering; the one only answering "our data is very clean" has probably never even seen what dirty data looks like.
In one sentence: five modes, five medicines — holes get data filling or event records, wild points get removal or retention, dead values get alarms, reversed order gets re-sorting, and drift gets configuration verification — when mode identification is right, cleaning does not backfire, and the algorithms dare to trust this batch of data with confidence.
Scope and Limitations
- The conclusions in this article are intended for LV distribution and electrical-safety sites. Criteria are compiled from public codes and FEXLINK product documentation, and product parameters follow the latest selection handbook and product knowledge base.
- Quantitative criteria (protection zone, discharge and residual voltage, earth resistance, thresholds and grading) hold under the preconditions stated in this article. Where the site voltage level, earthing arrangement, environmental category or load profile differs, the figures must be recalculated.
- Specific product models and parameters are subject to the latest selection handbook and product knowledge base; this article does not constitute selection or procurement advice.
- Standard numbers cited here refer to the currently effective editions; when a standard is revised, the related quantitative conclusions must be re-checked.
- Deployment and implementation for specific projects must be carried out by qualified engineers based on site survey and design drawings.
Frequently Asked Questions
Five Typical Failure Modes of Raw IoT Data
You have just taken over an energy platform