I'm encountering a weird issue when attempting to stream packets with a Forge SimpleNetworkWrapper. The use case in question involves streaming audio to each individual player that has their "pocket radio" turned on, and sends a constant stream of packets (the audio quality being relatively low to decrease the network usage, yet at the same time add realism to a "pocket radio").
"Are you insane???" Yes, I am, but the system works well enough... be it for one issue I've encountered.
The system consists of a thread sending running every 200 ms, that sends audio data split over several packets (used to be a single large packet in my attempt to fix the issue I'm about to describe) and a handler on the client side to actually get the audio data and feed it to a SourceDataLine (which does the actual playback).
The weird issue I've noticed is that if you have the pocket radio on for long enough, it'll eventually start causing some network lag - entity movements start stuttering and there's a delay when placing/breaking blocks or trying to open containers. The lag usually starts after a few minutes and gets increasingly worse. Addressing it is sometimes as simple as turning off the pocket radio for a few seconds and turning it back on again, but it seems that relogging will also clear any lag that might've started to happen. My only guess is some sort of adaptive QoS or priority system, but I couldn't find any hint to this being the case (and there seems to be no way whatsoever to assign a priority to the packets: if such a system existed, I would set the audio packets to have a low priority). The lag is not caused by server slowdown: any players not using the radio will not be affected, but they will of course see players with radios eventually start to stutter.
Packet file (PhotonicRadioPacket.java)
package email.com.gmail.cosmoconsole.forge.photoniccraft;
import email.com.gmail.cosmoconsole.forge.photoniccraft.client.PhotonicClientProxy;
import io.netty.buffer.ByteBuf;
import net.minecraft.client.Minecraft;
import net.minecraft.client.audio.SoundHandler;
import net.minecraft.client.audio.SoundManager;
import net.minecraftforge.fml.common.ObfuscationReflectionHelper;
import net.minecraftforge.fml.common.network.simpleimpl.IMessage;
import net.minecraftforge.fml.common.network.simpleimpl.IMessageHandler;
import net.minecraftforge.fml.common.network.simpleimpl.MessageContext;
import paulscode.sound.SoundSystem;
public class PhotonicRadioPacket implements IMessage {
public static final int PACKET_SPLIT = 5;
public static final int PACKET_SIZE = 2205 / PACKET_SPLIT;
public byte[] audio;
private int length;
private int start;
private int end;
public double strength;
public PhotonicRadioPacket() {
this.strength = 0;
this.length = 0;
this.audio = new byte [0];
this.start = this.end = 0;
}
public PhotonicRadioPacket(double strength, byte[] audio, int start, int end) {
this.strength = strength;
this.length = end - start;
this.start = start;
this.end = end;
this.audio = audio;
}
@Override
public void fromBytes(ByteBuf buf) {
strength = buf.readDouble();
length = buf.readShort();
start = 0;
end = length;
this.audio = new byte[length];
buf.readBytes(this.audio);
}
@Override
public void toBytes(ByteBuf buf) {
buf.writeDouble(strength);
buf.writeShort(length);
buf.writeBytes(audio, start, end - start);
}
public static class Handler implements IMessageHandler<PhotonicRadioPacket, IMessage> {
@Override
public IMessage onMessage(PhotonicRadioPacket message, MessageContext ctx) {
if (ModPhotonicCraft.sourceDataLine == null) return null;
double strength = message.strength;
if (strength < 0) {
PhotonicAPI.shouldNotBePlaying();
PhotonicAPI.stopPlayingRadio();
return null;
}
byte[] audio = message.audio;
if (audio.length != PACKET_SIZE) {
return null;
}
byte[] noise = new byte[PACKET_SIZE];
byte[] finalaudio = new byte[PACKET_SIZE*2];
float scv = 1.0F;
try {
scv = ((float)Minecraft.getMinecraft().gameSettings.getSoundLevel(ModPhotonicCraft.radioCategory));
} catch (Exception ex) {}
// extremely ugly and there's probably a better way, but works for now
double volumeToPlay = ((SoundSystem)ObfuscationReflectionHelper.getPrivateValue(SoundManager.class,
(SoundManager)ObfuscationReflectionHelper.getPrivateValue(SoundHandler.class, Minecraft.getMinecraft().getSoundHandler(), PhotonicReflectionHelper.sndManager),
PhotonicReflectionHelper.sndSystem)).getMasterVolume()
* scv;
if (volumeToPlay <= 0) return null;
PhotonicClientProxy.setVolume(volumeToPlay);
volumeToPlay = 1.0;
double istrength = 1 - strength;
PhotonicAPI.silentNoiseGenerate(noise);
for (int i = 0; i < PACKET_SIZE; i++) {
if (strength > 0) {
finalaudio[2*i] = (byte) scaleVolume(audio[i], noise[i], strength, volumeToPlay);
} else {
finalaudio[2*i] = (byte) scaleVolume((byte)0, noise[i], 0.0, volumeToPlay);
}
finalaudio[2*i+1] = (byte) (finalaudio[2*i] ^ 0x80);
}
long now = System.currentTimeMillis();
if ((now - ModPhotonicCraft.lastReceive) > 1500L) {
return null;
}
long start = System.currentTimeMillis();
ModPhotonicCraft.sourceDataLine.write(finalaudio, 0, 2*PACKET_SIZE);
return null;
}
}
private static double scaleVolume(byte d, byte n, double s, double v) {
return (((double)unsign(qualityRuin(d,v)) - 128) * s + ((double)unsign(n) - 128) * (1 - s)) * v + 128;
}
private static byte qualityRuin(byte val, double q) {
if (q >= 0.9) return val;
if (q >= 0.75) { return (byte) ((val & 0xFE) | ((val & 0x80) >> 7)); }
if (q >= 0.6) { return (byte) ((val & 0xFC) | ((val & 0xC0) >> 6)); }
if (q >= 0.45) { return (byte) ((val & 0xF8) | ((val & 0xE0) >> 5)); }
if (q >= 0.3) { return (byte) ((val & 0xF0) | ((val & 0xF0) >> 4)); }
if (q >= 0.2) { return (byte) ((val & 0xE0) | ((val & 0xF0) >> 3)); }
if (q >= 0.1) { return (byte) ((val & 0xC0) | ((val & 0xF0) >> 2)); }
return 0;
}
public static double unsign(byte b) {
if (b < 0)
return b + 256;
return b;
}
}
The thread that takes care of sending data via the network handler (ModPhotonicCraft.network)
serverTimer.scheduleAtFixedRate(new TimerTask() {
@Override
public void run() {
if (PhotonicAPI.midSend.get()) return;
PhotonicAPI.midSend.set(true);
try {
MinecraftServer srvr = FMLCommonHandler.instance().getMinecraftServerInstance();
radioHandled.clear();
recvData.clear();
recvPower.clear();
recvLoc.clear();
PhotonicAPI.cannotModify.set(true);
sendLock.lock();
try {
dataSentLastTick.clear();
for (int channel: sendCache.keySet()) {
recvData.put(channel, sendCache.get(channel));
dataSentLastTick.put(channel, sendCache.get(channel));
recvPower.put(channel, sendPower.get(channel));
recvLoc.put(channel, sendLoc.get(channel));
}
sendCache.clear();
sendPower.clear();
sendLoc.clear();
} finally {
sendLock.unlock();
}
PhotonicAPI.cannotModify.set(false);
long now = System.currentTimeMillis();
byte[] b = new byte[2205];
Arrays.fill(b, (byte)0);
ArrayList<UUID> removeKeys = new ArrayList<UUID>();
byte[] d = new byte[2205];
int m = 0;
for (UUID u: radioPlayers.keySet()) {
if (m >= ModPhotonicCraft.maximumRadio) {
break;
}
if ((now - radioPlayers.get(u)) >= 1000L) {
removeKeys.add(u);
} else {
int ch = radioChannels.get(u);
double ampl = 0.0;
EntityPlayer plr = getPlayerFromUUID(srvr, u);
if (plr instanceof EntityPlayerMP) {
EntityPlayerMP p = (EntityPlayerMP)plr;
if (recvData.containsKey(ch)) {
d = recvData.get(ch);
ampl = calculateAmplitude(recvLoc.get(ch).distanceSq(PhotonicLocation.fromPlayer(p)), recvPower.get(ch));
} else
Arrays.fill(d, (byte)0);
byte[] ba = ampl <= 0.0 ? b : PhotonicAPI.cloneByteArray(d);
for (int i = 0; i < PhotonicRadioPacket.PACKET_SPLIT; ++i) {
ModPhotonicCraft.network.sendTo(new PhotonicRadioPacket(ampl, ba, i * PhotonicRadioPacket.PACKET_SIZE, (i + 1) * PhotonicRadioPacket.PACKET_SIZE), p);
}
} else if (plr != null) {
if (recvData.containsKey(ch)) {
d = recvData.get(ch);
ampl = calculateAmplitude(recvLoc.get(ch).distanceSq(PhotonicLocation.fromPlayer(plr)), recvPower.get(ch));
} else
Arrays.fill(d, (byte)0);
byte[] ba = ampl <= 0.0 ? b : PhotonicAPI.cloneByteArray(d);
for (int i = 0; i < PhotonicRadioPacket.PACKET_SPLIT; ++i) {
ModPhotonicCraft.network.sendToAll(new PhotonicRadioPacket(ampl, ba, i * PhotonicRadioPacket.PACKET_SIZE, (i + 1) * PhotonicRadioPacket.PACKET_SIZE));
}
}
}
}
for (UUID u: removeKeys)
radioPlayers.remove(u);
} catch (Exception ex) {
PhotonicAPI.debugMessage("Radio server fatal error");
ex.printStackTrace();
} finally {
PhotonicAPI.flipState.set(!PhotonicAPI.flipState.get());
PhotonicAPI.cannotModify.set(false);
PhotonicAPI.midSend.set(false);
}
}
}, 10L, 200L);
There are several possible solutions to this, but I'd have to be certain of the underlying cause to try anything. Are there any tips, or perhaps, is there a system that is better designed for direct streaming of data (maybe even in Netty itself)?
update 1: "In addition, I've also noticed that there is immense connection lag should a player join with the radio already turned on, which also seems to point at an issue that can be fixed with something like QoS or packet priority."
update 2: To clarify, the radio stream remains completely smooth when everything else starts to lag (even in the case of what happens in update 1).
