[1.12.2] [UNSOLVED] Multi-threading to write/copy files

[Bektor]

Hi,

 

I've got a problem that I want to save some copied files with my Minecraft Mod, but I want this to be handled on an extra thread.

The problem is that I'm getting up to 5 temp files which contain my files instead of 1 zip file.

 

    public synchronized void start(Path path) {
        // Set the path here as it might have changed before the thread starts the backup process
        this.directory = path.resolve("tests.zip");
        
        this.thread = new Thread(this, "World Backup Thread");
        this.thread.setPriority(7);
        this.thread.start();
    }
    
    public synchronized void stop() {
        try {
            this.thread.join();
        } catch (InterruptedException e) {
            ServerUtilities.LOGGER.fatal(e);
        }
    }
    
    @Override
    public void run() {
        if(!this.isDone)
            this.createZipFile();
        
        this.isDone = true;  // at ThreadBackups.run(ThreadBackups.java:56)
    }
    
    private void createZipFile() {
        ImmutableMap immutableMap = ImmutableMap.of("create", String.valueOf(Files.notExists(this.directory)));
        
        FileSystemProvider zipProvider = FileSystemProvider.installedProviders().stream()
                .filter(provider -> provider.getScheme().equals("jar")).findFirst().get();
        
        try(FileSystem fs = zipProvider.newFileSystem(this.directory, immutableMap)) {
            Path root = fs.getPath("/");
          // this.world = path to the worldPath on which saving was disabled beforehand using world.disableLevelSaving = true;
            Files.walk(this.worldPath).forEach((Path sourcePath) -> {
                try {
                    Path destination = root;
                    for(Path p : this.worldPath.relativize(sourcePath))
                        destination = destination.resolve(p.toString());
                    
                    if(!Files.isDirectory(destination) || !Files.isDirectory(sourcePath))
                        Files.copy(sourcePath, destination);
                } catch(IOException e) {
                    e.printStackTrace();
                }
            });
        } catch(IOException e) {
            e.printStackTrace();
        } // ThreadBackups.createZipFile(ThreadBackups.java:81)
    }

 

// within TickEvent.ServerTickEvent
[...]
// disables all level saving and then calls the thread to start!
		Backups.INSTANCE.start(Backups.INSTANCE.server, "");
            
// returns the isDone value from the thread to which this class holds no reference as the Backups class created it
            if(Backups.INSTANCE.isDone()) {
                Backups.INSTANCE.stop(); // stops the whole thread by only calling the stop method in the thread class
                timeTillBackup = ModConfig.Backup.timeTillBackup;
            }
[...]

It also causes and AccessDeniedException for the zip file to be created as it can't find the zip file, thus causing a NoSuchFileException after the first exception.

Backups is not the class shown here! Backups holds an instance to the thread and calls the start/stop methods in the thread class which are posted here like the thread itself.

The start method in Backups disables level saving and calls afterwards the start method of the thread to start the backup process. The stop method of Backups does nothing but to call the

stop of the thread itself while isDone returns the boolean isDone from the thread.

 

This means the thread itself does nothing more as to create the files. All other logic is handles in other classes.

 

Exceptions:

Spoiler

[backups.ThreadBackups:createZipFile:82]: java.nio.file.AccessDeniedException: .\backups\tests.zip
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[backups.ThreadBackups:createZipFile:82]: 	at java.nio.file.Files.delete(Files.java:1126)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.sync(ZipFileSystem.java:1294)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:277)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.createZipFile(ThreadBackups.java:81)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.run(ThreadBackups.java:56)
[backups.ThreadBackups:createZipFile:82]: 	at java.lang.Thread.run(Thread.java:748)
[backups.ThreadBackups:createZipFile:82]: java.nio.file.NoSuchFileException: .\backups\tests.zip
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[backups.ThreadBackups:createZipFile:82]: 	at java.nio.file.Files.delete(Files.java:1126)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.sync(ZipFileSystem.java:1294)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:277)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.createZipFile(ThreadBackups.java:81)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.run(ThreadBackups.java:56)
[backups.ThreadBackups:createZipFile:82]: 	at java.lang.Thread.run(Thread.java:748)

 

 

I should note that I also tested if it is possible to write ZIP files within a thread and that just works fine, thought in Minecraft it doesn't.

Spoiler

public class ZipTest implements Runnable {
	
	private Thread thread;
	public boolean done = false;
	
	public void start() {
		this.thread = new Thread(this);
		thread.start();
	}
	
	public void stop() {
		try {
			thread.join();
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
	}
	
	public void run() {
		Path dest = Paths.get("test.zip");
		Path src = Paths.get("Test");
		
		Map<String, String> env = Collections.singletonMap("create", "true");
		
		FileSystemProvider zipProvider = FileSystemProvider.installedProviders().stream()
				.filter(provider -> provider.getScheme().equals("jar")).findFirst().get();
		
		try(FileSystem fs = zipProvider.newFileSystem(dest, env)) {
			Path root = fs.getPath("/");
			Files.walk(src).forEach((Path sourcePath) -> {
				try {
					Path destination = root;
					for(Path p : src.relativize(sourcePath))
						destination = destination.resolve(p.toString());
					
					System.out.println(destination.toString());
					if(!Files.isDirectory(destination) || !Files.isDirectory(sourcePath))
						Files.copy(sourcePath, destination);
				} catch (IOException e) {
					e.printStackTrace();
				}
			});
		} catch (IOException e) {
			e.printStackTrace();
		}
		done = true;
	}
	
	public static void main(String[] args) {
		ZipTest test = new ZipTest();
		test.start();
		
		if(test.done)
			test.stop();
	}
}

 

 

Thx in advance.

Bektor

[MCenderdragon]

I had the same problem, I created ZipFileSystems and then accest them via getFIlesSystem, but sometimes getFileSystem returned null so my code created a new one resulting in 2 open ZipFileSystem to the same file, when then all get closed both throw an error, but they leave themselfes in the FileSystem list so you cant create a new one since there is one but this is marked as closed, thus unusable. My fix was a java update to the latest version, this removed all the strange cases where the file system randomly broke and I made a map to store all FIleSystems myself.

[diesieben07]

I urge you to not write threading code manually.

Use an ExecutorService and CompletableFuture, this will eliminate 90% of your problems.

[Bektor]

46 minutes ago, diesieben07 said:

I urge you to not write threading code manually.

 

This makes me wonder why those classes are still around in Java. I've used them since I started with Java in Java 7 times.

[diesieben07]

37 minutes ago, Bektor said:

This makes me wonder why those classes are still around in Java. I've used them since I started with Java in Java 7 times.

Because they are low-level building blocks that are used by the more high-level classes like ExecutorService. It's the same as asking why we still have arrays if there is ArrayList.

[Soukyan]

//Static variable
public static ExecutorService saveExecutor = Executors.newFixedThreadPool(1);

//Example method
public void saveToFile(){
	saveExecutor.submit(() -> {
		//Your code here
	});
}