PyTorch Model Loading

Just as countergen.api_to_generative_model() and countergen.get_generative_model_evaluator() allow you to create a ModelEvaluator (see here if you haven’t read about it yet), countergenedit provides two similar functions which work for PyTorch models, while providing optimization which only work on model you run locally (for 1-token outputs, having access to the full logit vector allows you to run the model only once no matter how many outputs you expect).

You can also evaluate classification models by using the following function:

Finally, if you just want to evaluate a pipeline from Hugginface’s transformers library, you can use

Note: this last function won’t let you access the weights of your model, making editing fail with the tools provided by this library.