[All resources](https://goodmem.ai/resources) Interactive technical brief # The Architecture of Agent Optimization Source: Title: The Architecture of Agent Optimization | Goodmem > An interactive map of the agent stack: what each layer contributes to quality and cost, and the measured evidence for engineering it. Memory, retrieval, model, harness: four layers decide what an agent can do and what it costs. GoodMem is the memory layer, and it fine-tunes its own retrieval models automatically. PAIR Systems, the team behind it, engineers custom LLMs and harnesses when stock parts aren’t enough. PAIR Systems engineering · July 2026 [Architecture](https://goodmem.ai/resources/agent-optimization-architecture#architecture) [Memory](https://goodmem.ai/resources/agent-optimization-architecture#goodmem) [Retrieval models](https://goodmem.ai/resources/agent-optimization-architecture#retrieval-models) [LLM](https://goodmem.ai/resources/agent-optimization-architecture#llm) [Harness](https://goodmem.ai/resources/agent-optimization-architecture#harness) Explore the agent architecture **Layer 03 · Agent component LLM** PAIR trained a 35B-class MoE that outscored GLM‑5.2 on a controlled BI-agent benchmark. It reached 91% of Opus 4.8’s measured quality in medium-thinking mode at 35× lower serving cost. [View model benchmark](https://goodmem.ai/resources/agent-optimization-architecture#llm) **Layer 03 · Agent component Harness** [Zhang et al.](https://arxiv.org/abs/2605.23950) report 7.8× more performance variance from harness configuration than model choice. PAIR engineers custom harnesses for GoodMem customers as a service. **Layer 02 · Retrieval models Rerankers** GoodMem’s Fine-Tuning automatically derives optimized embedding and reranking models from two inputs: memory content and agents’ access patterns. [View Fine-Tuning study](https://goodmem.ai/resources/agent-optimization-architecture#retrieval-models) **Layer 02 · Retrieval models Embedders** GoodMem’s Pipeline Optimization automatically selects the best combination of embedders and rerankers for each agent’s unique context. **Layer 02 GoodMem Context / Memory** Controlled tests cut aggregate token burn 28% and agent steps 23%, with no measurable change in quality. Heavy retrieval tasks saved 30–66%. [View memory study](https://goodmem.ai/resources/agent-optimization-architecture#goodmem) **Layer 01 PostgreSQL Neon · AlloyDB** Battle-tested reliability, security, and industry support. An open data layer with hundreds of deployment paths. Benchmark · Memory ## Memory On vs Memory Off −28% aggregate token burn, memory on versus memory off A controlled, paired A/B on a production enterprise BI agent: 17 matched scenarios, three replicates each, identical model and harness, with GoodMem memory as the only variable. Memory also cut agent steps 23%, with no measurable change in answer quality, and heavy retrieval tasks saved 30–66%. Confidence intervals are nonparametric bootstrap estimates, 10,000 resamples, clustered by scenario. Pilot-scale benchmark; broader multi-tenant validation is in progress. [GoodMem · Benchmark](https://goodmem.ai/resources/cutting-agent-token-burn) ### [Memory That Pays for Itself](https://goodmem.ai/resources/cutting-agent-token-burn) [Controlled A/B benchmark — GoodMem's memory layer cut aggregate token use 28% and reasoning iterations 23%, at no measurable loss of answer quality.](https://goodmem.ai/resources/cutting-agent-token-burn) [**token burn**](https://goodmem.ai/resources/cutting-agent-token-burn) [−28%](https://goodmem.ai/resources/cutting-agent-token-burn) [**reasoning steps**](https://goodmem.ai/resources/cutting-agent-token-burn) [−23%](https://goodmem.ai/resources/cutting-agent-token-burn) [**quality change**](https://goodmem.ai/resources/cutting-agent-token-burn) [≈ 0](https://goodmem.ai/resources/cutting-agent-token-burn) [June 2026 Read the report](https://goodmem.ai/resources/cutting-agent-token-burn) Benchmark · Retrieval models ## Cloud Fine-Tuner on CRUMB Mean reciprocal rank on eight public CRUMB retrieval tasks, before and after automated fine-tuning. The benchmark tests the embedder; the fine-tuner also tunes rerankers. **8 / 8** Tasks improved **+0.232** Largest MRR gain · Code retrieval **35%** Median relative gain **Base embedder versus GoodMem Cloud Fine-Tuner on eight CRUMB retrieval tasks** | Retrieval task | Base MRR | Fine-tuned | Delta | | ------------------------------------------------- | -------- | ---------- | ---------- | | Clinical trial matching clinical\_trial | 0.633 | 0.746 | **+0.113** | | Code solution retrieval code\_retrieval | 0.389 | 0.621 | **+0.232** | | State-specific legal statutes legal\_qa | 0.232 | 0.284 | **+0.053** | | Scientific paper retrieval paper\_retrieval | 0.464 | 0.514 | **+0.050** | | Set-based entity retrieval set\_operation | 0.198 | 0.256 | **+0.058** | | Reasoning-heavy community QA stack\_exchange | 0.338 | 0.481 | **+0.143** | | Mathematical theorem retrieval theorem\_retrieval | 0.236 | 0.330 | **+0.095** | | Vague movie / TV retrieval tip\_of\_the\_tongue | 0.067 | 0.139 | **+0.071** | Eight public CRUMB retrieval tasks, mean reciprocal rank of the base embedder versus the fine-tuner output. Fully automated, with no per-task human tuning. CRUMB was introduced by Killingback and Zamani. [Read the CRUMB paper](https://arxiv.org/abs/2509.07253) Benchmark · LLM ## BI Harness LLM vs Frontier Models A 20-scenario production BI agent suite spanning Oracle EBS workloads: accounts payable, sales, inventory, and multi-step drill-downs. Three replicates per scenario, memory off, identical harness, independent LLM judge. The only variable is the model. **Production BI agent benchmark with the harness held constant** | Model | Median tokens | Tool calls | Pass rate | Cost per 1,000 conversations | | --------------------------------------------------------------------------------------------------------------------- | ------------- | ---------- | --------- | ---------------------------- | | Claude Opus 4.8* (medium thinking)* Proprietary frontier model, API only, cannot be self-hosted | 233k | 6 | **61.7%** | **$2,380** $1,780–$3,080 | | PAIR Systems BI Harness LLM**PAIR** Custom-trained for each customer, becomes the customer’s IP. Self-hosts on 2×H100 | 219k | 14 | **55.9%** | **$67** $55–$80 | | GLM-5.2 744B MoE, 40B active, roughly 16×H100 to self-host at FP8 | 342k | 11 | **54.4%** | **$260** $205–$320 | | GPT-OSS-120B 117B MoE, 5.1B active, self-hosts on 1×H100; the customer’s prior on-premises harness model | 182k | 8 | **30.0%** | **$10** $8–$12 | Pass rates and 95% confidence intervals are nonparametric bootstrap estimates (10,000 resamples) over scored replicates (57 to 68 per model). Cost per 1,000 conversations is measured per-conversation token usage priced at posted July 2026 provider rates: Anthropic list for Opus 4.8 ($5 input / $25 output per million tokens), OpenRouter for GLM-5.2 ($0.56/$1.76) and GPT-OSS-120B ($0.036/$0.18), and prevailing open-weight market rates for the PAIR model’s size class ($0.14/$1.00) as its serving proxy. Prompt caching is not modeled. Cross-check: even in the most conservative fully-serial self-hosting scenario (2×H100 at Lambda on-demand rates, $2.99 per GPU-hour), the PAIR model costs $367 per 1,000 conversations, still roughly six times cheaper than Opus 4.8. GPU footprints reflect serving precision as well as parameter count: GPT-OSS-120B ships natively 4-bit quantized (MXFP4), engineered by OpenAI to fit a single 80GB GPU, while the PAIR model’s tested serving configuration is 2×H100. Engineering note · Harness ## Harness Choice vs Model Choice 7.8× more variance from the harness than from the model Zhang and coauthors ran three frontier models through three harness configurations, nine combinations in all. Changing the harness moved agent scores 7.8 times more than changing the model. PAIR Systems engineers custom harnesses for GoodMem customers: context assembly, tool use, task decomposition, and memory policy. Variance across the grid: 18.48 percentage points squared from harness configuration, 2.37 from model choice. The 7.8× ratio is what this benchmark measured, not a general constant. [Read “Stop Comparing LLM Agents Without Disclosing the Harness”](https://arxiv.org/abs/2605.23950) From architecture to production ## Start with memory. Optimize the system above it. Deploy GoodMem as the context layer, or work with PAIR Systems on the retrieval models, LLMs, and harnesses above it. [Start free](https://cloud.goodmem.ai/login?loc=agent-architecture) [Talk to PAIR Systems](https://goodmem.ai/contact-sales)