# 01 · Architecture & Canonical Schemas This document is the **contract**. Every other module — every pattern detector, every alert channel, every broker adapter — is built against the types and interfaces defined here. Build this first. --- ## 1. System layers MarketPulse is a pipeline of seven layers. Data flows **down**; nothing in a lower layer knows which pattern or which channel produced the work. ``` ┌───────────────────────────────────────────────────────────────────┐ │ 1. MARKET DATA LAYER │ │ Real-time streams + historical OHLCV. Normalised to Candle[]. │ │ Adapters: broker feeds, exchange WS, Polygon/Finnhub/CCXT. │ └───────────────────────────────────────────────────────────────────┘ │ Candle[] ▼ ┌───────────────────────────────────────────────────────────────────┐ │ 2. INDICATOR / PATTERN ENGINE │ │ Pure functions. IndicatorSeries + Detector[] → Signal[]. │ │ (docs 02 & 03) │ └───────────────────────────────────────────────────────────────────┘ │ Signal ▼ ┌───────────────────────────────────────────────────────────────────┐ │ 3. STRATEGY & RISK GATE │ │ Combines signals, applies filters, position sizing, risk caps. │ │ Signal → Intent (or reject). (doc 06) │ └───────────────────────────────────────────────────────────────────┘ │ Intent ▼ ┌───────────────────────────────────────────────────────────────────┐ │ 4. ALERT / NOTIFICATION LAYER │ │ Fan-out to web · mobile · email · WhatsApp · Telegram. │ │ Renders Alert, waits for approve/reject (or auto). (doc 04) │ └───────────────────────────────────────────────────────────────────┘ │ ApprovedIntent ▼ ┌───────────────────────────────────────────────────────────────────┐ │ 5. ORDER ROUTER / OMS │ │ Idempotency, order lifecycle, OCO/bracket assembly. (doc 05) │ └───────────────────────────────────────────────────────────────────┘ │ Order │ Order ▼ ▼ ┌───────────────────────┐ ┌───────────────────────────────┐ │ 6a. PAPER SIMULATOR │ │ 6b. LIVE BROKER ADAPTER │ │ fills against feed │ │ Alpaca/IBKR/Binance/Kite… │ └───────────────────────┘ └───────────────────────────────┘ │ Fill │ Fill ▼ ▼ ┌───────────────────────────────────────────────────────────────────┐ │ 7. LEDGER + AUDIT + FEEDBACK │ │ Positions, P&L, reconciliation, append-only audit, notify back. │ └───────────────────────────────────────────────────────────────────┘ ``` **Design rule:** layers communicate only through the schemas below. A detector never calls a broker; a broker never inspects RSI. This is what lets you add a new pattern or a new channel without touching the rest of the system. --- ## 2. Core data types > Types are shown in TypeScript for precision. The same shapes map cleanly to > Python `dataclass`/`pydantic`, Go structs, or JSON schema. Pick one language > for the engine and keep the wire format JSON. ### 2.1 Candle (the atom) ```ts type Candle = { symbol: string; // "AAPL", "BTCUSDT", "RELIANCE.NS" tf: Timeframe; // "1m" | "5m" | "15m" | "1h" | "4h" | "1d" ts: number; // bar OPEN time, epoch ms, UTC open: number; high: number; low: number; close: number; volume: number; closed: boolean; // false while the bar is still forming }; type Timeframe = "1m" | "5m" | "15m" | "30m" | "1h" | "4h" | "1d" | "1w"; ``` **Rules that prevent 90% of live-trading bugs:** - Detectors run on **closed bars only** (`closed === true`). Acting on a forming bar causes repainting — a signal that appears then vanishes. - `ts` is always the bar's **open** time, UTC, milliseconds. Convert at the edges, never in the middle. - Feeds emit *append-only*. If a bar is revised, emit a correction event; never mutate history in place downstream. ### 2.2 Signal schema Every detector emits exactly this object. This is the most important type in the system. ```ts type Signal = { id: string; // uuid detector: string; // "harmonic.gartley" | "ma.golden_cross" | ... symbol: string; tf: Timeframe; ts: number; // close time of the bar that confirmed it side: "long" | "short"; // direction the pattern implies action: "entry" | "exit" | "scale_in" | "scale_out"; strength: number; // 0..1 detector-normalised confidence price: number; // reference/confirmation price // Trade geometry — optional but strongly preferred: suggested: { entry: number; // limit/trigger price stop: number; // protective stop targets: number[]; // one or more take-profit levels rr: number; // reward:risk at first target } | null; // Provenance so alerts can explain themselves and you can debug: evidence: { points?: Record; // X,A,B,C,D... ratios?: Record; // {"B/XA": 0.618, "D/XA": 0.786} indicators?: Record; // {"rsi": 27.4, "adx": 31} note: string; // human sentence for the alert body }; meta: Record; }; ``` ### 2.3 Intent (post-strategy, pre-order) The strategy layer converts a `Signal` into an `Intent` by attaching size and account, or drops it. An Intent is what a human approves. ```ts type Intent = { id: string; signalId: string; account: string; // logical account key mode: "paper" | "live"; symbol: string; side: "buy" | "sell"; qty: number; // resolved by position sizing (doc 06) type: "market" | "limit" | "stop" | "stop_limit"; limitPrice?: number; stopPrice?: number; bracket?: { // optional protective children takeProfit?: number; stopLoss?: number; }; timeInForce: "day" | "gtc" | "ioc" | "fok"; risk: { // filled by the risk gate, shown in the alert riskAmount: number; // currency at risk to the stop riskPct: number; // % of account equity rr: number; }; expiresAt: number; // approval window; auto-cancel if unapproved }; ``` ### 2.4 Order & Fill (broker-facing) ```ts type Order = { id: string; intentId: string; clientOrderId: string; // IDEMPOTENCY KEY — see §4 account: string; mode: "paper" | "live"; broker: string; // "alpaca" | "paper" | "binance" | "kite" | ... symbol: string; side: "buy" | "sell"; qty: number; type: Intent["type"]; limitPrice?: number; stopPrice?: number; status: "new" | "submitted" | "partially_filled" | "filled" | "cancelled" | "rejected" | "expired"; brokerOrderId?: string; // id returned by the venue createdAt: number; updatedAt: number; }; type Fill = { orderId: string; brokerOrderId: string; qty: number; price: number; fee: number; ts: number; }; ``` ### 2.5 Alert (what the user sees) ```ts type Alert = { id: string; intentId: string; channels: Channel[]; // where it was sent title: string; // "🟢 Gartley (bullish) · AAPL 1h" body: string; // from Signal.evidence.note + geometry actions: AlertAction[]; // approve / reject / snooze / view chartUrl?: string; // rendered annotated chart status: "pending" | "approved" | "rejected" | "expired" | "auto"; createdAt: number; respondedAt?: number; respondedVia?: Channel; }; type Channel = "web" | "mobile" | "email" | "whatsapp" | "telegram"; type AlertAction = { id: string; label: string; kind: "approve"|"reject"|"snooze"|"link" }; ``` --- ## 3. The `Detector` interface (build every pattern to this) This single interface is why the engine is extensible. Adding a pattern = adding one function that satisfies this contract. It never touches I/O. ```ts interface Detector { /** stable id, e.g. "harmonic.butterfly" */ readonly id: string; /** which timeframes it is valid on (or "*") */ readonly timeframes: Timeframe[] | "*"; /** minimum bars of history needed before it can emit */ readonly warmup: number; /** tunables with sane defaults; surfaced to the UI/config */ readonly params: Record; /** * Pure function. Given the closed-candle history for ONE symbol/timeframe * (oldest→newest) and precomputed indicator series, return 0..n Signals for * the LAST bar only. Must be deterministic and side-effect free. */ detect(ctx: DetectContext): Signal[]; } type DetectContext = { symbol: string; tf: Timeframe; candles: Candle[]; // closed bars, oldest→newest ind: IndicatorBook; // memoised indicator series (see §5) params: Record; }; ``` **Contract requirements** 1. **Pure & deterministic** — same input ⇒ same output. No clock, no network, no randomness. This makes detectors unit-testable and backtestable. 2. **Emit for the last bar only** — the engine walks bars forward and calls `detect` per new closed bar; a detector returns signals *confirmed on that bar*. This prevents look-ahead bias. 3. **No repainting** — never rely on data after `ts`. If confirmation needs a later bar (e.g. neckline break), the signal fires on *that* later bar. 4. **Normalise `strength` to 0..1** — so the strategy layer can rank signals from different families on one scale. 5. **Fill `evidence`** — points, ratios and a human `note`. The alert layer turns this into the message; your future self turns it into a debug trace. **Engine loop (reference):** ``` for each (symbol, tf): history = closedCandles(symbol, tf) ind = buildIndicatorBook(history) // memoised, incremental for each detector enabled for (symbol, tf): if history.length < detector.warmup: continue signals = detector.detect({symbol, tf, candles: history, ind, params}) for s in signals: emit(s) // → strategy & risk gate ``` --- ## 4. Idempotency (the rule that stops double-fills) Alerts can be retried. Webhooks can be delivered twice. A user can tap "approve" on both Telegram and email. **Money must not move twice.** - Every `Intent` deterministically derives a **`clientOrderId`**: `hash(signalId + account + side + roundedQty + bar_ts)`. - The Order Router keeps a **unique index on `clientOrderId`**. A second insert with the same key is a no-op that returns the existing order. - All live broker APIs accept a client order id (`client_order_id`, `newClientOrderId`, `tag`…). Always pass it through so the venue also dedupes. - Approvals are idempotent too: the first approve transitions the alert; later approves return the same result. --- ## 5. Indicator book (shared, memoised) Detectors must not each recompute EMAs. Compute an `IndicatorBook` once per `(symbol, tf)` per new bar and pass it in. ```ts interface IndicatorBook { sma(period: number): number[]; ema(period: number): number[]; rsi(period: number): number[]; atr(period: number): number[]; macd(fast: number, slow: number, signal: number): { macd: number[]; signal: number[]; hist: number[] }; bollinger(period: number, mult: number): { mid: number[]; upper: number[]; lower: number[]; pctB: number[]; bandwidth: number[] }; stoch(kP: number, kSmooth: number, dP: number): { k: number[]; d: number[] }; adx(period: number): { adx: number[]; plusDI: number[]; minusDI: number[] }; swings(depth: number): Swing[]; // pivot highs/lows — see doc 02 // …extend as detectors require } ``` Implementations should be **incremental** where possible (EMA, RSI, ATR all update O(1) per new bar) so the engine can run on 1-minute data across many symbols. See doc 03 for exact formulas. --- ## 6. Recommended tech stack Nothing here is mandatory — the contracts are language-agnostic — but this combination is proven and keeps the moving parts small. | Concern | Recommended | Why | |---------|-------------|-----| | Engine language | **Python** (pandas/numpy, or `pandas-ta`/TA-Lib) or **TypeScript** (Node) | Both have mature TA libs; pick the team's strength. | | Real-time transport | WebSocket in; internal event bus (Redis Streams / NATS / Kafka) | Backpressure + replay. | | State / ledger | **PostgreSQL** (orders, fills, positions, audit) | ACID + the unique index that guarantees idempotency. | | Cache / dedup | **Redis** | Alert state, rate limits, `clientOrderId` locks. | | API / webhooks | FastAPI / Express | Receives WhatsApp/Telegram callbacks (doc 04). | | Scheduler | Cron / Temporal | Bar-close ticks, approval-window expiry. | | Charts for alerts | Server-rendered PNG (lightweight-charts + headless Chromium, or mplfinance) | Annotated chart in the message. | | Secrets | Vault / cloud secret manager / KMS | Broker keys never in the repo (doc 06). | | Deploy | One worker per role: `ingest`, `engine`, `alerts`, `oms` | Scale and fail independently. | **Backtest = live, minus I/O.** Because detectors are pure and the engine loops bar-by-bar, the *same* detector code runs in the backtester (feed historical candles) and in production (feed live candles). Never write a "backtest version" of a strategy — that is how live/backtest drift bugs are born. --- ## 7. Module / folder map ``` marketpulse-app/ ├── ingest/ # data adapters → Candle[] on the bus ├── engine/ │ ├── indicators/ # sma, ema, rsi, macd, atr, bbands, adx, swings… │ ├── detectors/ # one file per pattern; all implement Detector │ └── loop.ts # the engine loop from §3 ├── strategy/ # signal combination, filters, position sizing (doc 06) ├── risk/ # caps, kill switch, promotion gates (doc 06) ├── alerts/ │ ├── render.ts # Signal → Alert (title/body/chart) │ ├── web.ts mobile.ts email.ts whatsapp.ts telegram.ts │ └── webhooks/ # inbound approve/reject callbacks (doc 04) ├── oms/ │ ├── router.ts # idempotency, lifecycle (doc 05) │ ├── paper.ts # simulator │ └── brokers/ # alpaca.ts ibkr.ts binance.ts kite.ts angelone.ts ├── ledger/ # positions, P&L, reconciliation, audit └── config/ # symbols, strategies, params, EXECUTION_MODE ``` Proceed to the pattern catalog: [`02-patterns-price-structure.md`](02-patterns-price-structure.md) and [`03-indicators.md`](03-indicators.md).