Personal Finance Dashboard from Multiple Providers

0) Problem Restatement

Amazon asked: design a financial dashboard that shows a user's bank account balances, stock holdings, cryptocurrency holdings, and derived portfolio metrics (total net worth, allocation by asset type, daily change). Data comes from multiple external providers (banks via aggregators like Plaid, brokers, crypto exchanges, price feeds), each with different formats, update frequencies, rate limits and reliability.

1) Architecture

flowchart LR
    U["User"] --> API["Dashboard API"]
    API --> DB[("Normalized holdings + balances")]
    API --> DER["Derived metrics service"]
    DER --> PR[("Price cache - stocks, crypto, FX")]
    SCH["Refresh scheduler"] --> AD["Provider adapters"]
    AD --> P1["Bank aggregator"]
    AD --> P2["Broker API"]
    AD --> P3["Crypto exchange API"]
    AD --> DB
    PF["Market price feeds"] --> PR
    VAULT[("Token vault - encrypted credentials")] --> AD

2) Key Design Points

• Adapters per provider: each converts provider data into a normalized model: accounts (type, currency, balance, as_of), holdings (asset, quantity, cost_basis, as_of). New providers are just new adapters.
• Separate quantities from prices: holdings (how many AAPL shares, how much BTC) change rarely, while prices change constantly. Store quantities from providers, and get prices from a shared market price cache (one feed for all users). The portfolio value = quantity × latest price, computed on read or updated by a stream.
• Refresh scheduling:
• When the user opens the app: refresh stale connections (subject to rate limits) and show cached data immediately.
• In the background: refresh by provider tier (banks a few times a day, brokers on market close or intraday, crypto more often).
• Per-provider rate limiters and jittered schedules so we don't burst at the top of the hour.
• Staleness indicators: every number shows as_of, e.g., "Chase balance updated 3 hours ago". If a provider fails, show the last known value with a warning, never a silent wrong value.
• Derived metrics: net worth, allocation, daily change (today's value vs yesterday's close snapshot), all converted to the user's currency with FX rates. Store a daily snapshot of net worth for history charts.

3) Reliability and Security

• Provider outages: circuit breakers per provider, retries with backoff, and "reconnect required" states when tokens expire (the user must re-authorize).
• Security: provider credentials and OAuth tokens are encrypted in a vault (never in logs), with read-only scopes where possible. MFA for the app, and encryption at rest.
• Idempotent upserts of balances and holdings keyed by (user, provider, account/asset), with events for changes (alerts like "large deposit").

4) Scale

• Millions of users × several connections each. The refresh workload is the main cost, so prioritize active users, lower the frequency for inactive ones, and use provider webhooks where offered (push instead of poll).
• Price computations are shared: 10K distinct assets priced once, then applied to millions of holdings.

5) Wrap-Up

Connect each provider through an adapter that normalizes accounts and holdings into one model, refreshed by a rate-limited, jittered scheduler (and on app open) with secrets kept in a vault. Keep quantities per user but prices in a shared market cache, compute net worth, allocation and daily change with FX conversion, and snapshot daily history. Always show "as of" freshness and last-known values with warnings when providers fail.