On this page 8 sections
- Missing time is not the same thing as no trading
- Why one hour can matter
- Missing ticks can be partly hidden by a tester
- The wrong way to detect gaps
- A practical gap-analysis workflow
- DST creates fake-looking gaps and real mapping errors
- So should you reject a dataset with one missing hour?
- The deeper lesson
Imagine downloading fifteen years of data.
Everything looks complete except for one hour.
One hour out of more than 130,000.
Would you care?
The tempting answer is:
“Probably not.”
The better answer is:
“What happened during that hour?”
Missing time is not the same thing as no trading
Before calling anything a gap, distinguish between two situations.
Expected absence
The market was closed, the instrument was outside its quoting session or activity was genuinely absent.
Unexpected absence
The source should have contained market updates, but the data are missing.
Only the second case is a true data-integrity problem.
This sounds simple until weekends, holidays, session schedules and daylight-saving changes enter the picture.
Why one hour can matter
Suppose your strategy had no position and no signal anywhere near that interval.
The effect may be zero.
Now suppose the missing hour contained:
- a stop-loss event,
- a pending-order trigger,
- a volatility spike,
- an indicator extreme,
- the session high used later that day,
- a spread expansion,
- the bar that armed a trading condition.
The effect can propagate far beyond the missing interval.
A stateful EA may behave differently hours or days later because one internal condition changed.
This is why gap impact can be nonlinear.
Missing ticks can be partly hidden by a tester
MetaTrader has mechanisms for working with incomplete historical data.
Its Strategy Tester uses minute bars alongside real tick history and can generate tick sequences in situations where minute history exists but tick detail is unavailable.
That is useful because testing does not necessarily have to stop.
But it creates an important distinction:
a test successfully running is not the same thing as proving every minute contained original historical ticks.
The tester journal can provide valuable information about real-tick availability and inconsistencies.
The wrong way to detect gaps
A naive rule might say:
“Flag every hour with zero ticks.”
That will create false alarms across:
- weekends,
- market holidays,
- scheduled trading breaks,
- instruments with limited sessions.
A better system asks whether a gap conflicts with the expected market schedule and neighbouring data.
A practical gap-analysis workflow
1. Build an hourly coverage map
Count ticks per hour.
This quickly exposes large holes and sudden feed changes.
2. Add market-session context
Classify expected closures separately from suspicious absences.
3. Check neighbouring periods
Did activity disappear suddenly and return normally one hour later?
That can be a useful warning signal.
4. Re-query the source
If the provider supports it, fetch that interval again.
A temporary download failure may disappear.
5. Cross-check bars
If independent M1 or broker candles exist, compare whether the supposedly empty period contained market activity.
6. Preserve unresolved cases
If an interval cannot be conclusively verified, label it unverified rather than pretending it passed.
DST creates fake-looking gaps and real mapping errors
Daylight-saving transitions deserve special attention.
A timestamp conversion error can make an hour appear duplicated, skipped or moved.
The underlying price stream can be perfectly intact.
The clock mapping is what failed.
That is one reason TickArk's broker-aware MT5 workflow explicitly validates candles around DST-change periods rather than relying only on a permanent fixed-hour assumption.
So should you reject a dataset with one missing hour?
Not automatically.
First determine:
- Was the market expected to be open?
- Can the source recover the interval?
- Does another representation confirm activity?
- Does the missing period overlap strategy exposure?
- Could it affect indicators or strategy state?
Then decide whether the dataset remains appropriate for the experiment.
The deeper lesson
Data quality is not about chasing a magical “100%” badge.
It is about knowing where uncertainty exists.
A documented imperfection can be managed.
An invisible one cannot.
PRIMARY SOURCES