Keep raw sensor data for 7–30 days as a practical starting range, then create lower-resolution tiers for long-term analysis, subject to investigation, compliance, and equipment requirements.
The correct retention period depends on sampling rate, signal bandwidth, event duration, storage cost, and how long high-resolution data must support failure analysis. No universal retention interval applies to every industrial sensor.
Raw retention by fault-analysis need
Industrial data acquisition systems commonly generate samples at intervals from milliseconds to seconds, and high-frequency systems can produce terabytes of data per day. Raw retention should cover at least one complete troubleshooting cycle: 7 days for routine trend work, 30 days for equipment reliability programs, and longer when investigation of rare failures or contractual evidence is required [S1].
The source material does not establish a mandatory 7-day or 30-day threshold, so these are engineering starting points rather than standards. A machine that fails once every 18 months cannot be supported reliably by a 30-day raw window unless relevant waveforms are captured at the time of the event.
For rotating machinery, vibration, torque, and transient event records, full-rate data is most valuable during startup, shutdown, load changes, alarms, and detected anomalies. Long-term layers can support energy, utilization, and drift analysis, but they can conceal the edge shape and timing that explain a mechanical failure. A detailed comparison of edge and cloud vibration analytics appears in Edge vs Cloud Analytics for Vibration Data: 2026 Deployment Matrix.
Selection criteria for a retention policy
A defensible policy is built around six measurable criteria: required diagnostic window, fastest event of interest, sensor bandwidth, acquisition rate, acceptable information loss, and storage cost. For slow-changing process variables such as temperature or pressure, daily values may retain enough trend information; for vibration and transient waveforms, seconds of raw data can contain the fault mechanism. [S1]
Set the raw tier from the fastest diagnostic requirement, not from the storage budget. If investigators must examine a 20-millisecond transient, a 100-millisecond acquisition interval already fails that requirement. The source example uses 100-millisecond readings to explain when hourly averages are sufficient for long-term trends, while retaining detailed data for anomaly detection [S1].
Storage architecture also matters. Grafana Mimir can retain all metric data indefinitely if no retention time is configured, which confirms that downsampling is not automatically imposed by a time-series platform. A plant may use unlimited object storage, but that does not make every resolution equally useful [S2].
Raw, minute, hourly, and daily tiers compared
Use separate retention and resolution tiers because each supports a different engineering decision. The following comparison is a practical policy framework, with the source material supporting the hierarchy of 1-minute, 1-hour, and 1-day aggregation levels [S1].
Do not replace a raw record with an average unless the loss of information is intentional. The aggregation method should be selected from mean, maximum, minimum, median, or standard deviation, with the retained statistic documented alongside the time window. Maxima and minima are often more useful than means for alerts, while medians can reduce the influence of spikes [S1].
Decimation, aggregation, and adaptive reduction
Decimation selects every nth sample and preserves original values, while aggregation replaces groups of samples with statistical summaries. A 60:1 reduction from one sample per second to one sample per minute is acceptable only when the discarded interval contains no diagnostically relevant transient. [S1]
Adaptive downsampling retains more samples when local variance is high and reduces low-variance periods more aggressively. The published implementation uses a 10-sample window and a variance threshold of 0.1, keeping every second sample in a high-variance window and retaining the window mean in a low-variance window [S1]. Those values are examples, not universal settings.
For high-rate waveform data, the original waveform must be low-pass filtered before decimation to prevent aliasing and maintain the mean. Filtering reduces bandwidth but smears sharp edges, making event-to-event energy comparison less reliable when events are closer together than the filter duration [S3].
Filter limits and storage penalties
Downsampled waveform storage can consume 3.75 times the space of a compact raw format in the cited implementation because each sample stores three 32-bit floating-point values and three 8-bit metadata values. That penalty is separate from the sample-count reduction and must be included in storage calculations [S3].
For a 50 kS/s output, the cited FIR filter duration is 650 microseconds. Two events separated by less than 650 microseconds cannot be compared cleanly after filtering because their energy is spread across the filter response [S3].
The published storage measurements provide a useful check: a one-minute capture measured 470 MB at 2 MSPS, 881 MB at 1 MSPS, 441 MB at 500 kS/s, 176 MB at 200 kS/s, and 88 MB at 100 kS/s. These results reflect a particular format and device implementation, not universal storage rates [S3].
Who needs raw data, and who does not
Raw retention is appropriate for rotating equipment, electrical waveforms, high-speed transient sensors, process upset investigations, quality events, and regulated evidence. A 50 kS/s filter with 650 microseconds of smearing illustrates why transient-heavy measurements require careful band limiting before reduction [S3].
Long-term raw retention is less valuable for slow process variables, occupancy counts, energy meters, and status changes when the operational question concerns hourly or daily trends. A data logger configured for minute-level reporting can be economical, but it should not replace an event-triggered high-rate capture system when sudden failures must be reconstructed.
Decimation is suitable when individual spikes remain meaningful, aggregation is better for trend and reporting work, and adaptive reduction is useful when behavior changes substantially over time. A sensor connected through a digital multimeter may also require separate retention rules for logged measurements and triggered waveform captures.
Implementation controls and failure modes
Implement downsampling as an automated lifecycle rule with an explicit raw-retention deadline, a defined rollup statistic, and traceability from each summary back to its source interval. Preserve original units, timestamps, calibration state, sensor identifiers, and event flags; a mean without engineering units or provenance is not usable historical data [S1].
The main failure modes are aliasing, loss of peaks, inconsistent aggregation, and false assumptions that platform retention equals analytical adequacy. Filtering addresses aliasing but can alter sharp-edge behavior, while mean-only rollups can understate mechanical stress and process excursions [S3].
Track storage growth per signal, query latency, percentage of time spent in a downsampled tier, and the number of investigations lacking adequate raw context. For laboratory measurements, the electronic test and measurement workflow should define the sampling and capture rate before the retention schedule is approved.
Set the first raw-retention window at 30 days for a new system, reduce it to 7 days only when the analysis need is demonstrably low, and preserve high-rate data around alarms, trips, and maintenance events. Review the policy when acquisition rate, storage cost, failure frequency, or investigation requirements change.