As tom suggested in the comment above, deleting the files:
fontList.cache
fontList.py3k.cache
tex.cache
solve the problem.
In my case the files were under:
`~/.matplotlib`
EDITED
A couple of days ago the message appeared again, I deleted the files in the locations mention above without any success. I found that as suggested here by T Mudau there’s an extra location with text cache files is: ~/.cache/fontconfig