[1.15.2][Solved] Problem with diacritic

[fcelon]

Hi,

when language file in my mod contains characters with diacritic like š č ř or ž, those characters render as ? in game. However, the same characters render just fine in vanilla texts and in chat.

cs_cz.json:

{
	"entity.wt_improvements.decoy":"Dvojník",
	"item.wt_improvements.totem_of_decoy":"Totem zdvojení",
	"block.wt_improvements.trade_station":"Obchodní stanice",
	"item.wt_improvements.trade_station":"Obchodní stanice",
	"item.minecraft.potion.effect.mighty_night_vision": "Silný lektvar vidění ve tmě",
	"item.minecraft.potion.effect.mighty_invisibility": "Silný lektvar neviditelnosti",
	"item.minecraft.potion.effect.mighty_leaping": "Silný lektvar skákání",
	"item.minecraft.potion.effect.mighty_fire_resistance": "Silný lektvar ohnivzdornosti",
	"item.minecraft.potion.effect.mighty_swiftness": "Silný lektvar svižnosti",
	"item.minecraft.potion.effect.mighty_turtle_master": "Silný lektvar želvího mistra",
	"item.minecraft.potion.effect.mighty_water_breathing": "Silný lektvar dýchání pod vodou",
	"item.minecraft.potion.effect.mighty_healing": "Silný lektvar léčení",
	"item.minecraft.potion.effect.mighty_regeneration": "Silný lektvar regenerace",
	"item.minecraft.potion.effect.mighty_strength": "Silný lektvar síly",
	"item.minecraft.potion.effect.mighty_slow_falling": "Silný lektvar pomalého padání",
	"item.minecraft.potion.effect.mighty_fortitude": "Silný lektvar odolnosti",
	"item.minecraft.potion.effect.mighty_danger_sight": "Silný lektvar zviditelnění nebezpečí",
	"item.minecraft.potion.effect.mighty_stability": "Silný lektvar stability",
	"item.minecraft.potion.effect.mighty_relief": "Silný lektvar úlevy",
	"item.minecraft.potion.effect.mighty_restfulness": "Silný lektvar odpočinku",
	"item.minecraft.potion.effect.mighty_repellence": "Silný lektvar odpudivosti",
	"item.minecraft.potion.effect.mighty_vibing": "Silný lektvar přitažlivosti",
	"item.minecraft.potion.effect.mighty_luck": "Silný lektvar štěstí",
	"item.minecraft.splash_potion.effect.mighty_night_vision": "Silný vrhací lektvar vidění ve tmě",
	"item.minecraft.splash_potion.effect.mighty_invisibility": "Silný vrhací lektvar neviditelnosti",
	"item.minecraft.splash_potion.effect.mighty_leaping": "Silný vrhací lektvar skákání",
	"item.minecraft.splash_potion.effect.mighty_fire_resistance": "Silný vrhací lektvar ohnivzdornosti",
	"item.minecraft.splash_potion.effect.mighty_swiftness": "Silný vrhací lektvar svižnosti",
	"item.minecraft.splash_potion.effect.mighty_turtle_master": "Silný vrhací lektvar želvího mistra",
	"item.minecraft.splash_potion.effect.mighty_water_breathing": "Silný vrhací lektvar dýchání pod vodou",
	"item.minecraft.splash_potion.effect.mighty_healing": "Silný vrhací lektvar léčení",
	"item.minecraft.splash_potion.effect.mighty_regeneration": "Silný vrhací lektvar regenerace",
	"item.minecraft.splash_potion.effect.mighty_strength": "Silný vrhací lektvar síly",
	"item.minecraft.splash_potion.effect.mighty_slow_falling": "Silný vrhací lektvar pomalého padání",
	"item.minecraft.splash_potion.effect.mighty_fortitude": "Silný vrhací lektvar odolnosti",
	"item.minecraft.splash_potion.effect.mighty_danger_sight": "Silný vrhací lektvar zviditelnění nebezpečí",
	"item.minecraft.splash_potion.effect.mighty_stability": "Silný vrhací lektvar stability",
	"item.minecraft.splash_potion.effect.mighty_relief": "Silný vrhací lektvar úlevy",
	"item.minecraft.splash_potion.effect.mighty_restfulness": "Silný vrhací lektvar odpočinku",
	"item.minecraft.splash_potion.effect.mighty_repellence": "Silný vrhací lektvar odpudivosti",
	"item.minecraft.splash_potion.effect.mighty_vibing": "Silný vrhací lektvar přitažlivosti",
	"item.minecraft.splash_potion.effect.mighty_luck": "Silný vrhací lektvar štěstí",
	"item.minecraft.lingering_potion.effect.mighty_night_vision": "Silný doznívající lektvar vidění ve tmě",
	"item.minecraft.lingering_potion.effect.mighty_invisibility": "Silný doznívající lektvar neviditelnosti",
	"item.minecraft.lingering_potion.effect.mighty_leaping": "Silný doznívající lektvar skákání",
	"item.minecraft.lingering_potion.effect.mighty_fire_resistance": "Silný doznívající lektvar ohnivzdornosti",
	"item.minecraft.lingering_potion.effect.mighty_swiftness": "Silný doznívající lektvar svižnosti",
	"item.minecraft.lingering_potion.effect.mighty_turtle_master": "Silný doznívající lektvar želvího mistra",
	"item.minecraft.lingering_potion.effect.mighty_water_breathing": "Silný doznívající lektvar dýchání pod vodou",
	"item.minecraft.lingering_potion.effect.mighty_healing": "Silný doznívající lektvar léčení",
	"item.minecraft.lingering_potion.effect.mighty_regeneration": "Silný doznívající lektvar regenerace",
	"item.minecraft.lingering_potion.effect.mighty_strength": "Silný doznívající lektvar síly",
	"item.minecraft.lingering_potion.effect.mighty_slow_falling": "Silný doznívající lektvar pomalého padání",
	"item.minecraft.lingering_potion.effect.mighty_fortitude": "Silný doznívající lektvar odolnosti",
	"item.minecraft.lingering_potion.effect.mighty_danger_sight": "Silný doznívající lektvar zviditelnění nebezpečí",
	"item.minecraft.lingering_potion.effect.mighty_stability": "Silný doznívající lektvar stability",
	"item.minecraft.lingering_potion.effect.mighty_relief": "Silný doznívající lektvar úlevy",
	"item.minecraft.lingering_potion.effect.mighty_restfulness": "Silný doznívající lektvar odpočinku",
	"item.minecraft.lingering_potion.effect.mighty_repellence": "Silný doznívající lektvar odpudivosti",
	"item.minecraft.lingering_potion.effect.mighty_vibing": "Silný doznívající lektvar přitažlivosti",
	"item.minecraft.lingering_potion.effect.mighty_luck": "Silný doznívající lektvar štěstí",
	"item.minecraft.tipped_arrow.effect.mighty_night_vision": "Silný šíp vidění ve tmě",
	"item.minecraft.tipped_arrow.effect.mighty_invisibility": "Silný šíp neviditelnosti",
	"item.minecraft.tipped_arrow.effect.mighty_leaping": "Silný šíp skákání",
	"item.minecraft.tipped_arrow.effect.mighty_fire_resistance": "Silný šíp ohnivzdornosti",
	"item.minecraft.tipped_arrow.effect.mighty_swiftness": "Silný šíp svižnosti",
	"item.minecraft.tipped_arrow.effect.mighty_turtle_master": "Silný šíp želvího mistra",
	"item.minecraft.tipped_arrow.effect.mighty_water_breathing": "Silný šíp dýchání pod vodou",
	"item.minecraft.tipped_arrow.effect.mighty_healing": "Silný šíp léčení",
	"item.minecraft.tipped_arrow.effect.mighty_regeneration": "Silný šíp regenerace",
	"item.minecraft.tipped_arrow.effect.mighty_strength": "Silný šíp síly",
	"item.minecraft.tipped_arrow.effect.mighty_slow_falling": "Silný šíp pomalého padání",
	"item.minecraft.tipped_arrow.effect.mighty_fortitude": "Silný šíp odolnosti",
	"item.minecraft.tipped_arrow.effect.mighty_danger_sight": "Silný šíp zviditelnění nebezpečí",
	"item.minecraft.tipped_arrow.effect.mighty_stability": "Silný šíp stability",
	"item.minecraft.tipped_arrow.effect.mighty_relief": "Silný šíp úlevy",
	"item.minecraft.tipped_arrow.effect.mighty_restfulness": "Silný šíp odpočinku",
	"item.minecraft.tipped_arrow.effect.mighty_repellence": "Silný šíp odpudivosti",
	"item.minecraft.tipped_arrow.effect.mighty_vibing": "Silný šíp přitažlivosti",
	"item.minecraft.tipped_arrow.effect.mighty_luck": "Silný šíp štěstí",
	"subtitles.wt_improvements.block.trade_station.ding": "Zvonek stanice cinká"
}

result:

Does anyone know can I solve this problem?

Edited September 16, 2020 by fcelon

[Beethoven92]

mmm i don't have knowledge on the matter, but i took a look inside the vanilla cs_cz.json and i see they are using a lot of formatting codes which look like that: \u00e1..i believe those are doing the job to get the special characters you need. Take a look into the vanilla file

[fcelon]

Thakns you. I have been unable to locate vanilla cs_cz.json in my computer, but replacing problematic characters with codes seems to fix the issue.

[Beethoven92]

Deleted link to minecraft assets

Edited September 16, 2020 by Beethoven92

[Curle]

Note the Minecraft End-User License Agreement:

Quote

so you are not allowed to distribute the Game (or any part of it) to anyone else. [...]
Also, don't just rip art resources and pass them around, that's no fun.

 

That link breaks the Minecraft EULA/TOS, and is most likely very illegal. Do not recommend this in the future, or there will be worse repercussions.

The actual fix for this problem would likely be to change the encoding of the language json file. UTF-8 BLE does not encode these characters well, but the game is equipped to handle more sophisticated encodings. UTF-16 should work.

[Beethoven92]

2 minutes ago, Curle said:

Note the Minecraft End-User License Agreement:

 

That link breaks the Minecraft EULA/TOS, and is most likely very illegal. Do not recommend this in the future, or there will be worse repercussions.

The actual fix for this problem would likely be to change the encoding of the language json file. UTF-8 BLE does not encode these characters well, but the game is equipped to handle more sophisticated encodings. UTF-16 should work.

Oh well, i was not aware of that...i mean, those files are literally on a public github page that anyone can consult...its not like i ripped those myself and then distributed them..i just linked an existing resource on the web. But i understand what the problem could be, so i apologize if i broke the rules. I will remember next time. Thank you for letting me know that.

[Curle]

That repository breaks several copyright and EULA laws and has been reported numerous times.

Github's inaction does not make it okay.

[Beethoven92]

3 minutes ago, Curle said:

That repository breaks several copyright and EULA laws and has been reported numerous times.

Github's inaction does not make it okay.

I guess i have to thanks again because i was not aware even of that..

[Curle]

Quote

I don't know what "UTF-8 BLE" is.

Apologies, i meant UTF-8 BE, the default encoding of most modern text editors.

There are well known problems in the standard Java libraries with reading / decoding these from a file on disk.

 

You're right about all the rest though

[Alpvax]

I'm assuming big-endian. But TBH, I thought the byte order was part of the UTF-8 spec. I've never seen it called UTF-8 BE.