1.19.4 Method addModifier from ItemAttributeModifierEvent do not update multimap attribute

I wanted to add an attribute modifier to an item everytime it gets an enchantment and updating its value to the level of the enchantment
I was trying to use the addModififier method from the ItemAttributeModifierEvent it seems to work without removing the rest of the attributes however whevener i try to get the multimap containing the modifiers, its just not there
for some weird reason the modifier is added to the item but if you try to get the multimap via getModifiers method, its not there. Which ends up adding the same modifier over and over everytime the event is fired
is this intended to work this way? is there any other way to add a modifier to an item consistently?
to reproduce this:
just add max health modifier to a chesplate and use the methods mentioned here.