The PermGen is garbage collected like the other parts of the heap.
The thing to note here is that the PermGen contains meta-data of the classes and the objects i.e. pointers into the rest of the heap where the objects are allocated. The PermGen also contains Class-loaders which have to be manually destroyed at the end of their use else they stay in memory and also keep holding references to their objects on the heap. The “Presenting the Permanent Generation” article by Jon Masamitsu on the Sun / Oracle blog site might help you.