Skip to content

Evaluation

Evaluation pipeline for trained sparse dictionaries.

EvalConfig pydantic-model

Bases: BaseConfig

Fields:

total_eval_tokens pydantic-field 

total_eval_tokens: int = 1000000

Total number of tokens to evaluate on

Evaluator 

Evaluator(cfg: EvalConfig)
Source code in src/lm_saes/evaluator.py 
61
62
def __init__(self, cfg: EvalConfig):
    self.cfg = cfg