TREC AutoJudge Infrastructure Status

Status: Re-Run AutoJudges from Promp Cache

Reproducibility effectiveness by software and LLM. Score ranges show the minimum and maximum effectiveness observed within each evaluation. Scores matching the most frequent score in their row are green; deviations are red.

We intend to add more software systems to the status check, including a few selected systems from each team. The status checks are defined in the TREC AutoJudge infrastructure-status repository.

The health checks currently run on the kiddie-20260605-training dataset. We intend to add more datasets to the health checks later.

Software	LLM	2026-07-14 11:53	2026-07-14 13:48	2026-07-14 21:01	2026-07-15 08:13	2026-07-15 21:02	2026-07-16 08:15	2026-07-16 19:17	2026-07-16 21:00	2026-07-17 08:11	2026-07-17 20:57	2026-07-18 07:52	2026-07-18 20:51	2026-07-19 08:21	2026-07-19 20:51	2026-07-20 09:19	2026-07-20 21:13	2026-07-21 08:31	2026-07-21 21:11	2026-07-22 08:31	2026-07-22 21:09	2026-07-23 08:32	2026-07-23 21:06	2026-07-24 08:29	2026-07-24 21:05	2026-07-25 08:07	2026-07-25 20:54	2026-07-26 08:28	2026-07-26 20:59	2026-07-27 09:54	2026-07-27 21:11	2026-07-28 08:35
webis/tinyjudge	openai/gpt-oss-20b	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167	Kendall: 0.333TauAP-B: 0.167
webis/tinyjudge	Qwen/Qwen2.5-7B-Instruct	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833	Kendall: 0.913TauAP-B: 0.833
trema-unh/prefnugget-queryonly	openai/gpt-oss-20b	—	—	—	—	—	—	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667	Kendall: 0.667TauAP-B: 0.667
trema-unh/prefnugget-queryonly	Qwen/Qwen2.5-7B-Instruct	—	—	—	—	—	—	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000	Kendall: 0.913–1.000TauAP-B: 0.833–1.000
trema-unh/prefnugget-queryonly-best	openai/gpt-oss-20b	—	—	—	—	—	—	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000
trema-unh/prefnugget-queryonly-best	Qwen/Qwen2.5-7B-Instruct	—	—	—	—	—	—	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000
trema-unh/prefnugget-grounded-best	openai/gpt-oss-20b	—	—	—	—	—	—	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000	Kendall: 0.707–1.000TauAP-B: 0.333–1.000
trema-unh/prefnugget-grounded-best	Qwen/Qwen2.5-7B-Instruct	—	—	—	—	—	—	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000	Kendall: 0.913–1.000TauAP-B: 0.667–1.000
trema-unh/prefnugget-best-decide-plum	openai/gpt-oss-20b	—	—	—	—	—	—	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000
trema-unh/prefnugget-best-decide-plum	Qwen/Qwen2.5-7B-Instruct	—	—	—	—	—	—	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000
trema-unh/langchain-pref-best-decide-plum	openai/gpt-oss-20b	—	—	—	—	—	—	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000	Kendall: 0.548–1.000TauAP-B: 0.417–1.000
trema-unh/langchain-pref-best-decide-plum	Qwen/Qwen2.5-7B-Instruct	—	—	—	—	—	—	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000
trema-unh/prefnugget-queryonly-best-updated	openai/gpt-oss-20b	—	—	—	—	—	—	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000	Kendall: 0.667–1.000TauAP-B: 0.667–1.000
trema-unh/prefnugget-queryonly-best-updated	Qwen/Qwen2.5-7B-Instruct	—	—	—	—	—	—	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000	Kendall: 1.000TauAP-B: 1.000

Status: Nugget Banks

Automated health checks for nugget banks are in progress.

Status: Prompt Caches

Automated health checks for prompt caches are in progress.