EmonPi CSV logging too coarse: check and reduce interval

Part of our guide on track home energy consumption

EmonPi ships with 10-second CSV intervals by default, which masks the rapid cycling and defrost spikes that heat pump troubleshooting depends on.

What the default CSV logging interval is and why it feels coarse

EmonPi logs CSV data at 10-second intervals out of the box. That default strikes a balance between capturing general household consumption trends and keeping SD card writes manageable. For tracking solar yield, overnight baseload or daily kWh totals, 10 seconds works fine.

The interval feels coarse when you monitor heat pumps, electric vehicle chargers or immersion heaters. A heat pump defrost cycle lasts two to four minutes, and the power spike can hit three to four kilowatts before dropping back. At 10-second resolution you capture perhaps 12 to 24 samples across the whole event, which makes it hard to distinguish a normal defrost from a compressor stutter or a faulty reversing valve.

EmonPi also logs locally to CSV files and posts data to Emoncms feeds. The interval you set in emonhub.conf governs both. If you see stepped, blocky graphs in Emoncms or miss short transient peaks, the logging interval is the first place to look. A coarser interval smooths over the detail that retrofit decisions depend on.

The default exists because most EmonPi users start with whole-house monitoring, where appliance-level transients don't matter. Once you add CT clamps to individual circuits or track a heat pump's coefficient of performance hour by hour, you need finer time resolution to see what's really happening. We cover using pulse output for energy readings in its own article.

Where to find and interpret your current logging interval settings

Your current interval lives in /etc/emonhub/emonhub.conf. SSH into the EmonPi or access it via the local Emoncms admin interface, then open the file in a text editor. Look for the section headed [[EmonHubEmoncmsReporter]]. Inside that block you'll see a line that reads interval = 10 or another integer. That number is seconds.

If the line is missing or commented out with a hash, EmonHub falls back to its compiled default, which is also 10 seconds. Some older images used 30 seconds, so check rather than assume. The section also contains a url pointing to your local or remote Emoncms instance and a buffersize that holds samples when the network drops. Neither affects the logging rate.

To confirm the interval actually in use, log into Emoncms and open the Feeds page. Click any feed and check the timestamp gap between consecutive entries. If you see 10-second jumps and your config says 5, either the edit didn't save or you forgot to restart the service. Feeds don't lie; the config file is just intent until EmonHub reads it. The other half of this decision is choosing ammeters for surge measurement.

A quick diagnostic: if your feed shows a timestamp gap larger than your configured interval, EmonHub is buffering or losing samples. That points to CPU load, a slow SD card or network latency rather than a config problem. Match the feed timestamps to the config value before you change anything.

1. SSH into the EmonPi or use the Emoncms admin interface file editor.
2. Open /etc/emonhub/emonhub.conf in nano or vi.
3. Scroll to the [[EmonHubEmoncmsReporter]] section.
4. Read the interval line; the number is your current logging interval in seconds.
5. Cross-check by opening Emoncms Feeds and comparing consecutive timestamps.

What you'll need

SANDISK 32GB High Endurance Video microSDHC Card with Adapter for dash cam

Supports continuous recording for up to 2,500 hours, suitable for long-term CSV logging with high endurance.

See the one to get

How to modify EmonPi configuration for finer CSV logging intervals

Edit /etc/emonhub/emonhub.conf and change the interval line under [[EmonHubEmoncmsReporter]] to your target value in seconds. For heat pump monitoring I'd set 5 seconds; for detailed defrost analysis or fault diagnosis, 2 seconds. Save the file, then restart EmonHub with sudo systemctl restart emonhub. The new interval takes effect immediately.

If you run multiple reporters (one local, one pushing to a remote Emoncms server) each reporter block has its own interval line. Edit both if you want matching resolution everywhere. Mismatched intervals aren't wrong, but they make it harder to correlate data when you export CSVs from two sources. We cover reducing wasted heating energy in its own article.

After the restart, wait 30 seconds and refresh the Emoncms Feeds page. Check the timestamp gap on a live feed. If it still shows 10-second jumps, either the config syntax is wrong or the service didn't pick up the change. Run sudo systemctl status emonhub to check for errors. A typo in emonhub.conf often stops the service entirely, leaving feeds frozen.

For persistent changes, make sure the filesystem isn't mounted read-only. Older EmonPi images remount the root partition read-only after boot to protect the SD card. Run mount | grep ' / ' and look for ro in the options. If it's there, remount with sudo mount -o remount,rw / before you edit, then remount read-only afterward if that's your preferred setup.

• 5 seconds: good general choice for heat pumps, captures defrost cycles and cycling behaviour without overwhelming storage.
• 2 seconds: use for short-term fault diagnosis, detailed COP analysis during defrost, or troubleshooting compressor starts.
• 1 second: only for bench testing or debugging - generates large CSVs quickly and raises SD write load noticeably.

Trade-offs in data size and processing when you reduce intervals

Halving the interval doubles the number of samples per hour, which doubles CSV file size and the number of writes to the SD card. A single CT channel at 10-second intervals generates roughly 8,640 samples per day, or about 170 KB in a plain CSV. At 5 seconds that becomes 340 KB; at 2 seconds, 850 KB. Add six CT channels and a pulse counter and you're writing several megabytes per day. For the detail, see our notes on how low sunlight affects solar export readings.

SD card wear matters on a Raspberry Pi. The EmonPi uses an industrial-grade card, but write endurance is finite. Dropping from 10 seconds to 5 raises write cycles by a factor of two, but that still falls well within the card's rating for continuous operation over several years. Going to 1-second logging on eight channels, running 24/7, will shorten card life noticeably: expect replacement every two to three years rather than five.

CPU load rises slightly because EmonHub has to process and buffer more samples. On a Raspberry Pi 3 or 4 the difference is negligible at 5-second intervals. At 1 second with a full set of inputs you might see the emonhub process climb from 2% to 5% CPU usage. That's not a problem unless you're also running other services on the same Pi: Node-RED, InfluxDB or a VPN server can push total load high enough that buffering starts.

Storage rotation helps. EmonPi images often include a cron job that compresses and archives old CSV files weekly. If you're logging at 2-second intervals, check that the archive job is active and consider shortening the retention window. You rarely need second-by-second data older than a month; daily averages suffice for long-term trends. It helps to understand official Energy Star status details before going further.

Recommended intervals for heat pump monitoring and what they reveal

For day-to-day heat pump monitoring, 5 seconds is the sweet spot. It captures compressor starts, defrost cycles and cycling frequency without burying you in data. You'll see the power spike at the start of a defrost, the plateau during the cycle and the drop when it ends. That's enough to calculate defrost energy, spot excessive cycling and correlate performance with outdoor temperature.

If you're diagnosing a fault (hunting for short-cycling, checking whether the compressor is stalling on start, or measuring the exact duration of a defrost) drop to 2 seconds. At that resolution you can see the inrush current as the compressor motor spins up, distinguish between a soft-start ramp and a hard start, and catch brief power dips that suggest a weak capacitor or low voltage at the board.

For whole-house monitoring where the heat pump is just one load among many, 10 seconds remains fine. You care about daily kWh and average power, not the shape of every defrost. Save the finer intervals for the heat pump CT only, and leave the other channels at the default. EmonHub lets you set different intervals per input if you configure multiple reporters or use emonCMS input processing to downsample selectively.

One practical test: log at 2 seconds for a week, then review the CSVs. If you find yourself aggregating samples into 10-second or 30-second bins for every graph you make, you've gone too fine. The right interval is the one where the raw data is directly useful without further averaging.

• 10 seconds: whole-house monitoring, daily totals, baseload tracking - keeps file sizes small and SD wear low.
• 5 seconds: heat pump COP tracking, defrost cycle capture, general troubleshooting - best all-round choice for retrofit monitoring.
• 2 seconds: fault diagnosis, compressor start analysis, detailed defrost study - use for short campaigns, not continuous logging.
• 1 second: bench testing, sensor calibration, debugging EmonHub itself - not practical for long-term field deployment.

Common mistakes when changing the logging interval

The most common mistake is editing the config but forgetting to restart emonhub. The service reads emonhub.conf on startup and caches the settings in memory. Your edit sits in the file doing nothing until you restart. Always run sudo systemctl restart emonhub and check the feed timestamps afterward.

Another is editing the wrong reporter block. If you have multiple [[reporter]] sections (one for local logging, one for a remote server, one for MQTT) make sure you're changing the interval in the block that writes your CSVs. The section names are similar, and it's easy to edit EmonHubMqttReporter by mistake when you meant EmonHubEmoncmsReporter.

Some users set an interval so low that EmonHub can't keep up, and samples start buffering. The symptom is a feed that looks fine for a few minutes, then suddenly jumps forward by 30 or 60 seconds as the buffer flushes. If that happens, raise the interval or reduce the number of active inputs. A Raspberry Pi Zero struggles with 1-second logging on more than three or four channels.

Setting different intervals on different feeds and then trying to merge the CSVs later. If one feed logs every 5 seconds and another every 10, you'll have mismatched timestamps when you export. Either set all feeds to the same interval or use Emoncms input processing to align them before export. Stitching together CSVs with different time bases is harder than it sounds.

Questions people still ask

I ran an EmonPi at 10-second intervals for two years monitoring whole-house load, then dropped to 5 seconds when I added a heat pump CT: the defrost cycles finally made sense on the graphs, and the SD card is still fine three years later.