# Trainer doesn't call compute\_metrics during evaluation
**URL:**
**Author:**  [@filbeofITK](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FfilbeofITK)%5C
**Post date:** [February 13, 2024, 4:24pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-doesnt-call-compute-metrics-during-evaluation%2F73027%2F1 "2024-02-13T16:24:16Z")
Hello!
I have a custom model that I train and also would like to test within the HF environment. However it seems that even tho I pass a custom compute\_metrics function to my trainer it doesnât call it once.
This is my code:
def plot\_covariance\_matrix(model\_output, config):
print(âHello World!â)
# Calculate covariance matrices
cov\_matrix\_og = np.corrcoef(model\_output.target, rowvar=True)
cov\_matrix\_reconst = np.corrcoef(model\_output.output, rowvar=True)
```
# Plot covariance matrix of the data
fig, axes = plt.subplots(1, 2, figsize=(12, 6))
sns.heatmap(cov_matrix_og, annot=True, cmap="vlag", ax=axes[0], xticklabels=features, yticklabels=features)
axes[0].set_title('Covariance of the original data.')
sns.heatmap(cov_matrix_reconst, annot=True, cmap="vlag", ax=axes[1], xticklabels=features, yticklabels=features)
axes[1].set_title('Covariance of the reconstructed data.')
fig.suptitle('Covariance matrices.')
plt.tight_layout()
print('Before saving figure path:', os.path.abspath('.'))
fig.savefig(os.path.join(config['savepath'], config['result_filename']), format='png')
```
def compute\_metrics\_baseline(model\_output):
plot\_covariance\_matrix(model\_output, config={
âresult\_filenameâ: âbaseline\_result.pngâ,
âsavepathâ: os.path.dirname(os.path.abspath( **file** ))
})
return {}
testing\_args\_baseline = TrainingArguments(output\_dir=âembeddingmodel\_test\_checkpointsâ, logging\_dir=â./baseline\_logâ,
remove\_unused\_columns=False, evaluation\_strategy=âepochâ,
per\_device\_eval\_batch\_size=BATCH\_SIZE)
baseline\_tester = Trainer(
model=embeddingModel,
args=testing\_args\_baseline,
eval\_dataset=test\_dataset,
data\_collator=baseline\_collator,
compute\_metrics=compute\_metrics\_baseline
)
print(âTesting baseline model.â)
baseline\_tester.evaluate()
As you can see there are some print statements already in there, because first I thought my figure is saved on a different location I want it toâŚ
But not even the first print statement is reached, since I cannot se anything.
I checked this topic: [Trainer never invokes compute\_metrics](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-never-invokes-compute-metrics%2F11440) and many others but I still canât find reason for this.
At this point Iâm considering ditching evaluate as is, because hand coding this would have taken far less time than trying to debug it, so this post is my last resort âš
---
**Author:**  [@helloelwin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhelloelwin)%5C
**Post date:** [June 21, 2024, 12:50am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-doesnt-call-compute-metrics-during-evaluation%2F73027%2F2 "2024-06-21T00:50:54Z")
Hi I met the same problem. Have you been able to solve it?
---
**Author:**  [@filbeofITK](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FfilbeofITK)%5C
**Post date:** [June 24, 2024, 2:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-doesnt-call-compute-metrics-during-evaluation%2F73027%2F3 "2024-06-24T14:49:31Z")
No sadly I had to write my own evaluation methods.