There’s no formally defined difference between listeners and handlers. Some people would probably argue that they are interchangeable. To me however, they have slightly different meaning.
A listener is an object that subscribes for events from a source. Cf. the observer pattern. Usually you can have many listeners subscribing for each type of event, and they are added through addXyzListener methods.
Example: The MouseListener in the Java API.
A handler is an object that is responsible for handling certain events. A typical scenario would be to provide a handler for a specific event/task as an argument to a constructor, or set the handler through a setXyzHandler method. In other words, you usually have one handler for each type of event.
Example: The MemoryHandler in the Java API.