QQ 301, 6 min read

Data

Note. References below marked (qqx-archive) are to the archive reader and importer, which were added after the rest of this document's line numbers were taken.

5.1 What .qqx is

.qqx is the QubitQuant market-data archive format. One file holds one symbol's ticks or bars for one year. Prices are stored as integers (the price times 10^digits) in independent blocks. Within a block each column is delta-coded, the deltas are cut into quantile bins, the bin numbers are entropy-coded (rANS) and the offsets inside a bin are bit-packed. A file is: a header (magic QQX1, version, record kind, digits, a volumes flag, the symbol and a source string), the blocks, an index (per block: first timestamp, record count, byte offset, byte length) and a 56-byte footer (index position, time span, record count, an xxh64 value, the magic again). Timestamps are Unix milliseconds. Because blocks are independent, reading a date range decodes only the blocks that overlap it.

The terminal contains a read-only decoder. The writer and the format's specification live in a separate repository (qqdata), which was not read for this document. The reader refuses a file with bad magic, a truncated footer, an unsupported major version, an index that runs past the file, a block that does not match its index entry, or a decoded span that does not match the footer.

5.2 Bid bars versus ticks

A tick file decodes to ts, bid, ask, and bid and ask volumes when they were stored. A bar file decodes to ts, open, high, low, close, volume, spread (in points).

When a tick file is imported into the library it is folded to one-minute bars built on the bid: open, high, low and close are the first, highest, lowest and last bid in the minute. volume is the number of ticks in the minute, not units traded. spread is the mean of ask − bid in price units. The code names this "the MT5 chart convention".

Three things follow for backtests on these bars:

1. The default fill is a bid price. A long that "buys at the open" buys at the bid, which no one can do; the real cost is the ask. Set spread_bps (or run the realistic preset) or long entries and short exits are understated by the spread.
2. Volume is tick count. Square-root impact and the capacity check read it as units traded, so their output on this data is an order of magnitude at best.
3. The spread column does not reach the engine. The library import keeps only ts, open, high, low, close, volume (providers/userdata.py:98-145), so a per-bar spread stored in the archive is not available to spread: "data".

A bar holds four prices. Whether the high or the low came first inside the bar, and what the spread was at that moment, is not in a bar. Anything that depends on the path inside a bar needs ticks, and this engine runs on bars.

5.3 Dukascopy as the source of record, and why a broker feed is not

This is a project rule, not something the engine enforces. The terminal will import and backtest any file you give it. The rule, from the workspace's cross-cutting rules: "Dukascopy, not broker data. Broker data doesn't survive a slippage check. Always label the source on charts." And: every published metric must reconcile with a raw report a user can open.

The reasoning recorded in this repository (API-RESEARCH.md, section 3):

• A published number has to trace to data whose origin is stated. The Dukascopy archive is "Dukascopy-specific by construction": source="dukascopy" is written in every .qqx header and manifest.
• The platform's own data vault has an empty source field on every catalogue row. Its provenance is unstated, and its FX tape "is consistent with an aggregated retail feed". The document's conclusion is that vault data "cannot back any published Trident number", and that its legitimate uses are research and screening.

That is all the repository writes down. The rule is stated in one line ("broker data doesn't survive a slippage check") with no measurement behind it in this codebase, and the recorded argument is about traceability: one named, labelled tick history that anyone can be pointed to, rather than a feed whose origin is not stated. Nothing in the terminal's code compares a broker feed with Dukascopy.

What ships today, stated plainly:

• The in-app guide says the QubitQuant backend the app connects to by default "generates simulated prices" and that "nothing it shows is a live market quote" (ui/static/guide.md:23).
• The ten bundled sample datasets are recorded with source bundled-sample. Their upstream origin is not recorded in the manifest, and this document does not claim one.
• A file you import yourself is recorded as user-import. The terminal does not know or check where it came from.

5.4 How provenance is recorded on a library dataset

Every library dataset has an entry in manifest.json in the data folder: symbol, name, kind, folder, file, timeframe, row count, first and last timestamp, import time, and source.

For an archive import, the entry also gets a provenance object:

The file hash is not decorative. The backend only says where each file is and what it must hash to; the bytes come from the file host. A downloaded file is kept only when its length and its SHA-256 both match the catalogue. A response that is not a .qqx file (a web page from a link that is not shared) is an error, never data. After decoding, the record count and digits must match the catalogue entry too.

Two limits to be clear about:

• The hash recorded is that of the archive file. The dataset in the library is a CSV produced from it (windowed, and for ticks folded to bars), and no hash of that CSV is stored in the manifest. A validation report hashes the bars it actually ran on (candles_sha256, 4.9), but the report's manifest does not carry the dataset's source or provenance.
• The reader parses the footer's xxh64 value but, as far as this reading found, does not recompute it. Integrity on download rests on the SHA-256 and the structural checks listed in 5.1.

Source. Format and reader: providers/qqx.py:1-23, providers/qqx.py:408-481, providers/qqx.py:490-529 (qqx-archive). Bid bars: providers/qqx.py:543-555; candle shape for import providers/qqx.py:557-568 (qqx-archive). Download and verification: providers/archive.py:1-12, providers/archive.py:60-102; provenance providers/archive.py:146-191; caps providers/archive.py:33-34 (qqx-archive). Manifest entry: providers/userdata.py:400-449 (qqx-archive); on this tree providers/userdata.py:388-391, providers/userdata.py:402-439. Import column set: providers/userdata.py:98-145. Project rule: workspace CLAUDE.md ("Cross-cutting rules" 1 and 2); API-RESEARCH.md:139-171. Guaranteed by tests on that branch: tests/test_qqx.py (matches the reference decoder; truncated and corrupt files are refused; ticks fold to the archive's own one-minute bars) and tests/test_archive.py (bytes that do not match the catalogue are refused; provenance is recorded).