Desktop README.md Observatory layers.sys
gpu_stack.exe

gpu_stack/readme/frontdoor.txt

gpu_stack

A virtual AI datacenter you can question. It estimates what a training run costs in time, power, and money, states how sure it is, and shows you what every number is built from.

Open observatory Research program Inspect symbolic engine

read_graph.exe

Read it like a receipt, not a magic answer.

Start with a human question, then follow the named dependencies upstream. Every hop should tell you whether you are looking at an equation, a scenario value, or an unresolved root input.

  • The registry currently names 1517 variables and 950 equations.
  • Root debt is the pile of inputs the model cannot yet derive. All 619 stay named instead of being smoothed over.
target cost.per_token The number a human asks about.
equations run cost, tokens, power The graph walks upstream through what defines it.
root debt 619 named inputs Every unknown keeps its name.
scenario explicit assignments A fixture is a test anchor, not a price.
Target selected: start with the question. The page keeps the output attached to the labels underneath it.
1517registered variables
950equations connecting them
619root inputs, named instead of hidden
884equations with unit checks

token_journey.exe

Follow one token from math to money.

Each station is one layer of the model. The lamp lights as the token passes through, and the trip always ends at an invoice.

Model math attention, FFN, parameters
Kernels instructions, tiling, occupancy
Memory HBM traffic, caches, bandwidth
Silicon transistors, lithography, atoms
Power board, rack, cooling, PUE
Invoice: ~$3 per million tokens synthetic fixture value, not a market price

Symbolic causal backbone

A model earns trust when a measurement can prove it wrong.

The registry keeps every equation next to its units, sources, constraints, scenario assumptions, and the boundaries it has not resolved. The research layer adds the things that can contradict it: real observations, timed events, interventions, uncertainty, residuals, held-out splits, and regret. A held-out split is data the model never saw while it was being fitted, so a prediction on that data is a real test. Regret is what the choice cost you compared with the choice you would have made knowing the outcome. A complete graph is not the same thing as a tested one.

  • Only universal physics constants belong in Constant.
  • Everything else stays a Variable: clocks, voltages, tariffs, GPU counts, batch sizes, and facility assumptions.
  • A root input is modeling debt you can see. That beats hidden debt papered over with a fudge factor.

One answer at the top rests on every layer beneath it. Click a layer to see what it owes.

layers.sys

Click a layer. Watch the dependency chain move.

The visible machine is a building, but the model treats it as a constraint bundle: grid interconnect, substations, cooling loops, water, occupancy, capex, operations, and uptime.

  • Inputs include power envelope, PUE, utilization, cooling load, and build cost.
  • Outputs feed cluster capacity, cost allocation, emissions, and schedule pressure.

trace_target.exe

Choose a question and follow what it depends on.

Cost per token is not a lone price. It depends on run cost, token count, facility power, throughput, utilization, hardware choices, and root assumptions that still need better evidence.

  • This is a picture of how the resolver works, the code that walks the graph and computes a value. It is not a live solver.
  • The moving gold segment shows which way the dependencies run.

Every hop keeps its label, and that is what makes the graph useful. If a hop cannot be worked out from equations or scenario assignments, it comes back as a named missing boundary instead of a guess.

synthetic fixture resolves 4 of 4 targets
scenario-report status: ok

cone_browser.exe

Inspect any variable's upstream cone.

Pick a target, then click any node to open its direct dependencies. Each hop shows its unit, its scope, meaning the part of the system it belongs to, and whether it is an equation, a root input, or a physics constant. Root inputs carry a gold badge because they are the debt you can still see.

  • The tree loads from a JSON snapshot of the registry, generated ahead of time.
  • Click a node to open or close its direct dependencies.
  • Constants carry their own mark, so universal physics values are easy to tell apart.
Loading dependency data...

root_debt.dat

Root inputs are the visible unpaid invoices.

root-debt ranks unresolved root inputs by how much depends on them. A large family is not automatically a bad one. The ranking exists so you know which unknowns are expensive right now.

  • Total roots in the observed summary: 619.
  • Grouped root families in the observed summary: 151.
  • The heaviest shown family is physical.lithography.medium with total weight 3014.
physical.lithography.medium
weight 3014, roots 15
physical.lithography
weight 2185, roots 11
physical.lithography.source_plasma_drive
weight 1943, roots 8
physical.mosfet
weight 1866, roots 18
physical.process
weight 1293, roots 8

The bars scale the five README weights against the heaviest family. Nothing new is being measured here.

CLI.exe

Use the command line as a microscope.

This is a research instrument, not a polished app, and right now that is the useful thing to be. Ask it what exists, what is still unresolved, and where a claim bottoms out. 

python -m gpu_stack.cli stats
python -m gpu_stack.cli verify --profile fast
python -m gpu_stack.cli resolve econ.cost.per_token --preset scenarios.dense_training_cost_fixture --trace --missing
python -m gpu_stack.cli root-debt --families --limit 5
python -m gpu_stack.cli scenario-report scenarios.dense_training_cost_fixture --json
python -m gpu_stack.cli next-work
python -m gpu_stack.cli experiment-protocol E001 --json
python -m gpu_stack.cli experiment-run E001 --scenario experiments/e001-beyond-one-datacenter/screening-scenario-v1.json --output result.json --observatory-output docs/data/e001-screening-v1.json

Root input

A variable the model cannot yet compute from anything else. It may be a genuine edge of the scenario, or physics that nobody has broken down yet.

Dependency cone

Everything upstream of one number: the variables, equations, assumptions, and constants you need in order to explain it.

Scenario fixture

A named set of fixed input values, so a target resolves the same way every time. Synthetic fixtures are test anchors, not claims about real prices.

MFU

Model FLOPs Utilization: the share of the machine's peak math rate that training actually uses. When it is low, the run is usually waiting on memory instead of doing math, which is exactly what a roofline chart is drawn to show.

HBM

High Bandwidth Memory: the fast memory stacked right next to the accelerator. Moving data in and out of it is often the real speed limit, not the math.

PUE

Power Usage Effectiveness: total facility power divided by IT equipment power. Cooling and everything else the building draws shows up here.

Good for now

Tracing a result back to mechanics and evidence. Replaying compute, communication, checkpoint, outage, and recovery events one at a time. Keeping observations, assumptions, priors, modeled values, and unmeasured claims clearly apart.

Not finished yet

E001 now has explicit recovery mechanics and three measured learning stages. LC3 held useful work equal across policies: adaptive continuation kept the learning and saved both attempted work and opportunity ticks, the scheduled time it no longer needed, but it went past its frozen device-energy bound. None of that shows a frontier-scale multi-site run converges, and energy measured at one GPU is not energy measured at a facility.

next_work.exe

The frontier program is six questions a result could prove wrong.

1. Beyond One Datacenter · energy gate failedAt equal useful work, adaptive continuation learned just as well and saved work and scheduled time, but its measured RTX energy interval went past the frozen bound. Scaling stays blocked.
2. Shape the Power Waveform · nextSeparate checkpoint cadence from survivor continuation, attribute power phase by phase, and test whether scheduling that respects dependencies removes the energy penalty without giving back the measured learning, work, or time gains.
3. Semantic Fault ToleranceSpend canaries, replay, and redundancy on the faults that would really have hurt learning, not on whatever the fault is labeled.
4. Fluid Inference TopologyMeasure what is gained, and where the behavior flips, when the serving topology changes for every request.
5. Architecture as a Datacenter VariableDesign the model's parts and the mixed hardware together, under a single facility power and time budget.
6. Firm Grid-responsive InferenceMeasure meter-verified demand response with output quality, tail latency, rebound, and hidden work all counted inside the same boundary.