Jump to content

[1.12.2] [UNSOLVED] Multi-threading to write/copy files


Bektor

Recommended Posts

Hi,

 

I've got a problem that I want to save some copied files with my Minecraft Mod, but I want this to be handled on an extra thread.

The problem is that I'm getting up to 5 temp files which contain my files instead of 1 zip file.

 

    public synchronized void start(Path path) {
        // Set the path here as it might have changed before the thread starts the backup process
        this.directory = path.resolve("tests.zip");
        
        this.thread = new Thread(this, "World Backup Thread");
        this.thread.setPriority(7);
        this.thread.start();
    }
    
    public synchronized void stop() {
        try {
            this.thread.join();
        } catch (InterruptedException e) {
            ServerUtilities.LOGGER.fatal(e);
        }
    }
    
    @Override
    public void run() {
        if(!this.isDone)
            this.createZipFile();
        
        this.isDone = true;  // at ThreadBackups.run(ThreadBackups.java:56)
    }
    
    private void createZipFile() {
        ImmutableMap immutableMap = ImmutableMap.of("create", String.valueOf(Files.notExists(this.directory)));
        
        FileSystemProvider zipProvider = FileSystemProvider.installedProviders().stream()
                .filter(provider -> provider.getScheme().equals("jar")).findFirst().get();
        
        try(FileSystem fs = zipProvider.newFileSystem(this.directory, immutableMap)) {
            Path root = fs.getPath("/");
          // this.world = path to the worldPath on which saving was disabled beforehand using world.disableLevelSaving = true;
            Files.walk(this.worldPath).forEach((Path sourcePath) -> {
                try {
                    Path destination = root;
                    for(Path p : this.worldPath.relativize(sourcePath))
                        destination = destination.resolve(p.toString());
                    
                    if(!Files.isDirectory(destination) || !Files.isDirectory(sourcePath))
                        Files.copy(sourcePath, destination);
                } catch(IOException e) {
                    e.printStackTrace();
                }
            });
        } catch(IOException e) {
            e.printStackTrace();
        } // ThreadBackups.createZipFile(ThreadBackups.java:81)
    }

 

// within TickEvent.ServerTickEvent
[...]
// disables all level saving and then calls the thread to start!
		Backups.INSTANCE.start(Backups.INSTANCE.server, "");
            
// returns the isDone value from the thread to which this class holds no reference as the Backups class created it
            if(Backups.INSTANCE.isDone()) {
                Backups.INSTANCE.stop(); // stops the whole thread by only calling the stop method in the thread class
                timeTillBackup = ModConfig.Backup.timeTillBackup;
            }
[...]

It also causes and AccessDeniedException for the zip file to be created as it can't find the zip file, thus causing a NoSuchFileException after the first exception.

Backups is not the class shown here! Backups holds an instance to the thread and calls the start/stop methods in the thread class which are posted here like the thread itself.

The start method in Backups disables level saving and calls afterwards the start method of the thread to start the backup process. The stop method of Backups does nothing but to call the

stop of the thread itself while isDone returns the boolean isDone from the thread.

 

This means the thread itself does nothing more as to create the files. All other logic is handles in other classes.

 

Exceptions:

Spoiler

[backups.ThreadBackups:createZipFile:82]: java.nio.file.AccessDeniedException: .\backups\tests.zip
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[backups.ThreadBackups:createZipFile:82]: 	at java.nio.file.Files.delete(Files.java:1126)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.sync(ZipFileSystem.java:1294)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:277)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.createZipFile(ThreadBackups.java:81)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.run(ThreadBackups.java:56)
[backups.ThreadBackups:createZipFile:82]: 	at java.lang.Thread.run(Thread.java:748)
[backups.ThreadBackups:createZipFile:82]: java.nio.file.NoSuchFileException: .\backups\tests.zip
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[backups.ThreadBackups:createZipFile:82]: 	at java.nio.file.Files.delete(Files.java:1126)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.sync(ZipFileSystem.java:1294)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:277)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.createZipFile(ThreadBackups.java:81)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.run(ThreadBackups.java:56)
[backups.ThreadBackups:createZipFile:82]: 	at java.lang.Thread.run(Thread.java:748)

 

 

I should note that I also tested if it is possible to write ZIP files within a thread and that just works fine, thought in Minecraft it doesn't.

Spoiler

public class ZipTest implements Runnable {
	
	private Thread thread;
	public boolean done = false;
	
	public void start() {
		this.thread = new Thread(this);
		thread.start();
	}
	
	public void stop() {
		try {
			thread.join();
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
	}
	
	public void run() {
		Path dest = Paths.get("test.zip");
		Path src = Paths.get("Test");
		
		Map<String, String> env = Collections.singletonMap("create", "true");
		
		FileSystemProvider zipProvider = FileSystemProvider.installedProviders().stream()
				.filter(provider -> provider.getScheme().equals("jar")).findFirst().get();
		
		try(FileSystem fs = zipProvider.newFileSystem(dest, env)) {
			Path root = fs.getPath("/");
			Files.walk(src).forEach((Path sourcePath) -> {
				try {
					Path destination = root;
					for(Path p : src.relativize(sourcePath))
						destination = destination.resolve(p.toString());
					
					System.out.println(destination.toString());
					if(!Files.isDirectory(destination) || !Files.isDirectory(sourcePath))
						Files.copy(sourcePath, destination);
				} catch (IOException e) {
					e.printStackTrace();
				}
			});
		} catch (IOException e) {
			e.printStackTrace();
		}
		done = true;
	}
	
	public static void main(String[] args) {
		ZipTest test = new ZipTest();
		test.start();
		
		if(test.done)
			test.stop();
	}
}

 

 

Thx in advance.

Bektor

Developer of Primeval Forest.

Link to comment
Share on other sites

I had the same problem, I created ZipFileSystems and then accest them via getFIlesSystem, but sometimes getFileSystem returned null so my code created a new one resulting in 2 open ZipFileSystem to the same file, when then all get closed both throw an error, but they leave themselfes in the FileSystem list so you cant create a new one since there is one but this is marked as closed, thus unusable. My fix was a java update to the latest version, this removed all the strange cases where the file system randomly broke and I made a map to store all FIleSystems myself.

catch(Exception e)

{

 

}

Yay, Pokémon exception handling, gotta catch 'em all (and then do nothing with 'em).

Link to comment
Share on other sites

//Static variable
public static ExecutorService saveExecutor = Executors.newFixedThreadPool(1);

//Example method
public void saveToFile(){
	saveExecutor.submit(() -> {
		//Your code here
	});
}

 

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Announcements



  • Recently Browsing

    • No registered users viewing this page.
  • Posts

    • As the title says i keep on crashing on forge 1.20.1 even without any mods downloaded, i have the latest drivers (nvidia) and vanilla minecraft works perfectly fine for me logs: https://pastebin.com/5UR01yG9
    • Hello everyone, I'm making this post to seek help for my modded block, It's a special block called FrozenBlock supposed to take the place of an old block, then after a set amount of ticks, it's supposed to revert its Block State, Entity, data... to the old block like this :  The problem I have is that the system breaks when handling multi blocks (I tried some fix but none of them worked) :  The bug I have identified is that the function "setOldBlockFields" in the item's "setFrozenBlock" function gets called once for the 1st block of multiblock getting frozen (as it should), but gets called a second time BEFORE creating the first FrozenBlock with the data of the 1st block, hence giving the same data to the two FrozenBlock :   Old Block Fields set BlockState : Block{minecraft:black_bed}[facing=east,occupied=false,part=head] BlockEntity : net.minecraft.world.level.block.entity.BedBlockEntity@73681674 BlockEntityData : id:"minecraft:bed",x:3,y:-60,z:-6} Old Block Fields set BlockState : Block{minecraft:black_bed}[facing=east,occupied=false,part=foot] BlockEntity : net.minecraft.world.level.block.entity.BedBlockEntity@6d1aa3da BlockEntityData : {id:"minecraft:bed",x:2,y:-60,z:-6} Frozen Block Entity set BlockState : Block{minecraft:black_bed}[facing=east,occupied=false,part=foot] BlockPos{x=3, y=-60, z=-6} BlockEntity : net.minecraft.world.level.block.entity.BedBlockEntity@6d1aa3da BlockEntityData : {id:"minecraft:bed",x:2,y:-60,z:-6} Frozen Block Entity set BlockState : Block{minecraft:black_bed}[facing=east,occupied=false,part=foot] BlockPos{x=2, y=-60, z=-6} BlockEntity : net.minecraft.world.level.block.entity.BedBlockEntity@6d1aa3da BlockEntityData : {id:"minecraft:bed",x:2,y:-60,z:-6} here is the code inside my custom "freeze" item :    @Override     public @NotNull InteractionResult useOn(@NotNull UseOnContext pContext) {         if (!pContext.getLevel().isClientSide() && pContext.getHand() == InteractionHand.MAIN_HAND) {             BlockPos blockPos = pContext.getClickedPos();             BlockPos secondBlockPos = getMultiblockPos(blockPos, pContext.getLevel().getBlockState(blockPos));             if (secondBlockPos != null) {                 createFrozenBlock(pContext, secondBlockPos);             }             createFrozenBlock(pContext, blockPos);             return InteractionResult.SUCCESS;         }         return super.useOn(pContext);     }     public static void createFrozenBlock(UseOnContext pContext, BlockPos blockPos) {         BlockState oldState = pContext.getLevel().getBlockState(blockPos);         BlockEntity oldBlockEntity = oldState.hasBlockEntity() ? pContext.getLevel().getBlockEntity(blockPos) : null;         CompoundTag oldBlockEntityData = oldState.hasBlockEntity() ? oldBlockEntity.serializeNBT() : null;         if (oldBlockEntity != null) {             pContext.getLevel().removeBlockEntity(blockPos);         }         BlockState FrozenBlock = setFrozenBlock(oldState, oldBlockEntity, oldBlockEntityData);         pContext.getLevel().setBlockAndUpdate(blockPos, FrozenBlock);     }     public static BlockState setFrozenBlock(BlockState blockState, @Nullable BlockEntity blockEntity, @Nullable CompoundTag blockEntityData) {         BlockState FrozenBlock = BlockRegister.FROZEN_BLOCK.get().defaultBlockState();         ((FrozenBlock) FrozenBlock.getBlock()).setOldBlockFields(blockState, blockEntity, blockEntityData);         return FrozenBlock;     }  
    • It is an issue with quark - update it to this build: https://www.curseforge.com/minecraft/mc-mods/quark/files/3642325
    • Remove Instant Massive Structures Mod from your server     Add new crash-reports with sites like https://paste.ee/  
  • Topics

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.