Nuggets

A judge's create_nuggets() returns NuggetBanks — per-topic containers of questions and claims. NuggetDocEntry/TopicNuggetDocs carry the document-grounded variant.

NuggetBanks

autojudge_base.nugget_data.NuggetBanks pydantic-model

Bases: BaseModel

Container for multiple NuggetBanks, keyed by query_id.

Access banks directly via the banks field: banks.banks["topic-1"] banks.banks.get("topic-1") for qid, bank in banks.banks.items(): ...

Fields:

• format_version (str)
• banks (Dict[str, NuggetBank])

format_version pydantic-field

format_version: str = 'v3'

banks pydantic-field

banks: Dict[str, NuggetBank] = {}

get_bank_model classmethod

get_bank_model() -> Type[NuggetBank]

Return the bank model class for this container type.

Source code in src/autojudge_base/nugget_data/nugget_banks.py

@classmethod
def get_bank_model(cls) -> Type[NuggetBank]:
    """Return the bank model class for this container type."""
    return NuggetBank

from_banks_list classmethod

from_banks_list(banks: List[NuggetBank], overwrite: bool = False) -> NuggetBanks

Create from list of banks.

Parameters:

Name	Type	Description	Default
banks	List[NuggetBank]	List of NuggetBank instances	required
overwrite	bool	If False (default), raise error on duplicate query_id	False

Raises:

Type	Description
ValueError	If bank has no query_id, or duplicate query_id without overwrite

Source code in src/autojudge_base/nugget_data/nugget_banks.py

@classmethod
def from_banks_list(
    cls, banks: List[NuggetBank], overwrite: bool = False
) -> "NuggetBanks":
    """
    Create from list of banks.

    Args:
        banks: List of NuggetBank instances
        overwrite: If False (default), raise error on duplicate query_id

    Raises:
        ValueError: If bank has no query_id, or duplicate query_id without overwrite
    """
    result: Dict[str, NuggetBank] = {}
    for bank in banks:
        qid = bank.query_id
        if qid is None:
            raise ValueError("NuggetBank must have a query_id")
        if qid in result and not overwrite:
            raise ValueError(f"Duplicate query_id: {qid}")
        result[qid] = bank
    return cls(banks=result)

verify

verify(expected_topic_ids: Sequence[str], warn: bool = False) -> None

Verify nugget banks against expected topic IDs.

Source code in src/autojudge_base/nugget_data/nugget_banks.py

def verify(self, expected_topic_ids: Sequence[str], warn: bool = False) -> None:
    """Verify nugget banks against expected topic IDs."""
    from .verification import NuggetBanksVerification  # Local import avoids cycle
    NuggetBanksVerification(self, expected_topic_ids=expected_topic_ids, warn=warn).all()

autojudge_base.nugget_data.NuggetBanksProtocol

Bases: Protocol

Protocol for multi-topic nugget bank containers.

Implementations: NuggetBanks, NuggetizerNuggetBanks

banks instance-attribute

banks: Dict[str, NuggetBankProtocol]

get_bank_model classmethod

get_bank_model() -> Type[NuggetBankProtocol]

Return the bank model class for this container type (required for generic I/O).

Source code in src/autojudge_base/nugget_data/protocols.py

@classmethod
def get_bank_model(cls) -> Type[NuggetBankProtocol]:
    """Return the bank model class for this container type (required for generic I/O)."""
    ...

from_banks_list classmethod

from_banks_list(banks: List[NuggetBankProtocol], overwrite: bool = False) -> NuggetBanksProtocol

Create container from list of banks.

Source code in src/autojudge_base/nugget_data/protocols.py

@classmethod
def from_banks_list(
    cls, banks: List[NuggetBankProtocol], overwrite: bool = False
) -> "NuggetBanksProtocol":
    """Create container from list of banks."""
    ...

verify

verify(expected_topic_ids: Sequence[str], warn: bool = False) -> None

Verify nugget banks against expected topic IDs.

Source code in src/autojudge_base/nugget_data/protocols.py

def verify(self, expected_topic_ids: Sequence[str], warn: bool = False) -> None:
    """Verify nugget banks against expected topic IDs."""
    ...

Nugget documents

autojudge_base.nugget_doc_models.NuggetDocEntry pydantic-model

Bases: BaseModel

A nugget question with its relevant document IDs.

Fields:

• question (str)
• doc_ids (List[str])
• aggregator (str)
• answer_type (str)

question pydantic-field

question: str

doc_ids pydantic-field

doc_ids: List[str]

aggregator pydantic-field

aggregator: str = 'OR'

answer_type pydantic-field

answer_type: str = 'OPEN_ENDED_ANSWER'

to_collaborator_value

to_collaborator_value() -> list

Serialize to collaborator format: ["OR", {"OPEN_ENDED_ANSWER": [...]}]

Source code in src/autojudge_base/nugget_doc_models.py

def to_collaborator_value(self) -> list:
    """Serialize to collaborator format: ["OR", {"OPEN_ENDED_ANSWER": [...]}]"""
    return [self.aggregator, {self.answer_type: self.doc_ids}]

autojudge_base.nugget_doc_models.TopicNuggetDocs pydantic-model

Bases: BaseModel

All nugget-document mappings for a single topic.

Fields:

• topic_id (str)
• entries (List[NuggetDocEntry])

topic_id pydantic-field

topic_id: str

entries pydantic-field

entries: List[NuggetDocEntry]

to_collaborator_dict

to_collaborator_dict() -> dict

Serialize to collaborator format: {question: ["OR", {...}], ...}

Source code in src/autojudge_base/nugget_doc_models.py

def to_collaborator_dict(self) -> dict:
    """Serialize to collaborator format: {question: ["OR", {...}], ...}"""
    return {e.question: e.to_collaborator_value() for e in self.entries}

autojudge_base.nugget_doc_models.write_nugget_docs_collaborator

write_nugget_docs_collaborator(topics: Dict[str, TopicNuggetDocs], output_dir: Path) -> None

Write one nuggets_{topic_id}.json per topic in collaborator format.

Source code in src/autojudge_base/nugget_doc_models.py

def write_nugget_docs_collaborator(
    topics: Dict[str, TopicNuggetDocs],
    output_dir: Path,
) -> None:
    """Write one nuggets_{topic_id}.json per topic in collaborator format."""
    output_dir.mkdir(parents=True, exist_ok=True)
    for topic_id, topic in topics.items():
        path = output_dir / f"nuggets_{topic_id}.json"
        path.write_text(json.dumps(topic.to_collaborator_dict(), indent=4))
    print(f"Wrote {len(topics)} collaborator nugget files to {output_dir}")