From synthetic CSV to inspectable factor-research artifacts with JQ Factor Lab
A practical, synthetic-data walkthrough for Python developers building local research harnesses. Prepared with AI assistance for the project owner. This is project outreach, not an independent review or investment advice.
The problem: inspectable runs, not another return chart
A factor experiment is hard to debug when its inputs, calendar assumptions and outputs are scattered across notebooks. JQ Factor Lab is an Apache-2.0 Python toolkit whose configuration-driven CLI can return a JSON summary and write a separate directory of analysis artifacts. Its data layer uses DuckDB. The surrounding scheduler or agent still owns experiment decisions; this is not an autonomous profitable trading system.
Project source and installation documentation
Reproduce the small example
This walkthrough was verified on Linux with Python 3.12 against commit 7e24b714e0b82c7f786b16fd27ad836b08ac9212. The documented supported environments are Python 3.11–3.12 on macOS or Linux; native Windows is not supported. Use a fresh output directory.
git clone https://github.com/GSX-Eis/jq-factor-lab.git cd jq-factor-lab git checkout 7e24b714e0b82c7f786b16fd27ad836b08ac9212 python -m venv .venv source .venv/bin/activate python -m pip install -e '.[dev,synthesis]' python -m examples.generate_data --output data/demo.csv python -m tools.demo_cli single-factor-analysis \ --config config/demo.json --output-dir outputs/first-analysis
The verification used equivalent absolute paths for its temporary input and output locations. The example completed with exit status 0 on 3,600 synthetic input rows covering 20 fictional assets. This observation is not a runtime benchmark or a claim that the full test suite passes.
What the run actually produces
summary.json factor_values.csv ic.csv ir.csv quantile_returns.csv std_err.csv
The JSON stdout identifies DemoMomentum, row and asset counts, forward periods 1/5/10, execution lag 1, and calendar identity DEMO_SYNTHETIC_WEEKDAYS. Progress logs go to stderr. A harness can check the exit code, parse stdout with json.loads, and retain the output directory as the experiment artifact. A failed process must not be treated as a completed analysis.
Where DuckDB fits, and one easy mistake
The daily CSV evaluation path loads its input into an in-memory DuckDB database and calls the factor and evaluation modules. This is distinct from the persistent local-data import commands. Importing files into a separate database does not automatically redirect the demo CLI to that database: the caller must explicitly configure the intended path or data binding.
The project import guide also documents an important operational distinction: price-file resume tracking is based on frequency and filename. Editing an already recorded CSV does not automatically cause a reimport. For a revised complete price dataset, the documented approach is to import into a new database rather than mistake resume tracking for an incremental update service. Transaction boundaries vary by importer; an entire multi-dataset command is not one global transaction. These are documented contracts, not additional claims tested by the single-factor run above.
Read the pinned data-import contracts and the complete local-workflow example.
Limits to keep visible
All prices in this demonstration are synthetic, and its calendar is fictional. The results do not establish investment performance, predictive value, a valid real-market backtest, or production readiness. Real research requires suitable market data, an explicit verified trading calendar, validated label timing, costs and independent checks. Existing research factors, trained models and production strategies are not included.
A focused feedback question
If you connect this to your own research harness, which contract is hardest to understand first: the CSV shape, the calendar, the database binding, or the output artifacts? Concrete setup failures and unclear contracts are more useful feedback than a star request. The repository documentation is the source of truth for current behavior.