[1.14] WeakHashMap with entities' UUID: keys sometimes not clearing

I'm trying to create a system to register custom Entity data on-the-fly for already existing Entities (including vanilla Entities); for every data object i keep a normal HashMap that maps Class<Entity> -> DataParameter, and i keep a global WeakHashMap that maps UUID -> WeakReference<EntityDataManager> (as I noticed the Entity instance may change for one entity).

However I noticed sometimes (not always) the map does not clear out, even if I change world and run System.gc(), while sometimes it does