Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

[1.12.2] [UNSOLVED] Multi-threading to write/copy files


Bektor
 Share

Recommended Posts

Hi,

 

I've got a problem that I want to save some copied files with my Minecraft Mod, but I want this to be handled on an extra thread.

The problem is that I'm getting up to 5 temp files which contain my files instead of 1 zip file.

 

    public synchronized void start(Path path) {
        // Set the path here as it might have changed before the thread starts the backup process
        this.directory = path.resolve("tests.zip");
        
        this.thread = new Thread(this, "World Backup Thread");
        this.thread.setPriority(7);
        this.thread.start();
    }
    
    public synchronized void stop() {
        try {
            this.thread.join();
        } catch (InterruptedException e) {
            ServerUtilities.LOGGER.fatal(e);
        }
    }
    
    @Override
    public void run() {
        if(!this.isDone)
            this.createZipFile();
        
        this.isDone = true;  // at ThreadBackups.run(ThreadBackups.java:56)
    }
    
    private void createZipFile() {
        ImmutableMap immutableMap = ImmutableMap.of("create", String.valueOf(Files.notExists(this.directory)));
        
        FileSystemProvider zipProvider = FileSystemProvider.installedProviders().stream()
                .filter(provider -> provider.getScheme().equals("jar")).findFirst().get();
        
        try(FileSystem fs = zipProvider.newFileSystem(this.directory, immutableMap)) {
            Path root = fs.getPath("/");
          // this.world = path to the worldPath on which saving was disabled beforehand using world.disableLevelSaving = true;
            Files.walk(this.worldPath).forEach((Path sourcePath) -> {
                try {
                    Path destination = root;
                    for(Path p : this.worldPath.relativize(sourcePath))
                        destination = destination.resolve(p.toString());
                    
                    if(!Files.isDirectory(destination) || !Files.isDirectory(sourcePath))
                        Files.copy(sourcePath, destination);
                } catch(IOException e) {
                    e.printStackTrace();
                }
            });
        } catch(IOException e) {
            e.printStackTrace();
        } // ThreadBackups.createZipFile(ThreadBackups.java:81)
    }

 

// within TickEvent.ServerTickEvent
[...]
// disables all level saving and then calls the thread to start!
		Backups.INSTANCE.start(Backups.INSTANCE.server, "");
            
// returns the isDone value from the thread to which this class holds no reference as the Backups class created it
            if(Backups.INSTANCE.isDone()) {
                Backups.INSTANCE.stop(); // stops the whole thread by only calling the stop method in the thread class
                timeTillBackup = ModConfig.Backup.timeTillBackup;
            }
[...]

It also causes and AccessDeniedException for the zip file to be created as it can't find the zip file, thus causing a NoSuchFileException after the first exception.

Backups is not the class shown here! Backups holds an instance to the thread and calls the start/stop methods in the thread class which are posted here like the thread itself.

The start method in Backups disables level saving and calls afterwards the start method of the thread to start the backup process. The stop method of Backups does nothing but to call the

stop of the thread itself while isDone returns the boolean isDone from the thread.

 

This means the thread itself does nothing more as to create the files. All other logic is handles in other classes.

 

Exceptions:

Spoiler

[backups.ThreadBackups:createZipFile:82]: java.nio.file.AccessDeniedException: .\backups\tests.zip
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[backups.ThreadBackups:createZipFile:82]: 	at java.nio.file.Files.delete(Files.java:1126)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.sync(ZipFileSystem.java:1294)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:277)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.createZipFile(ThreadBackups.java:81)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.run(ThreadBackups.java:56)
[backups.ThreadBackups:createZipFile:82]: 	at java.lang.Thread.run(Thread.java:748)
[backups.ThreadBackups:createZipFile:82]: java.nio.file.NoSuchFileException: .\backups\tests.zip
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[backups.ThreadBackups:createZipFile:82]: 	at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[backups.ThreadBackups:createZipFile:82]: 	at java.nio.file.Files.delete(Files.java:1126)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.sync(ZipFileSystem.java:1294)
[backups.ThreadBackups:createZipFile:82]: 	at com.sun.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:277)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.createZipFile(ThreadBackups.java:81)
[backups.ThreadBackups:createZipFile:82]: 	at backups.ThreadBackups.run(ThreadBackups.java:56)
[backups.ThreadBackups:createZipFile:82]: 	at java.lang.Thread.run(Thread.java:748)

 

 

I should note that I also tested if it is possible to write ZIP files within a thread and that just works fine, thought in Minecraft it doesn't.

Spoiler

public class ZipTest implements Runnable {
	
	private Thread thread;
	public boolean done = false;
	
	public void start() {
		this.thread = new Thread(this);
		thread.start();
	}
	
	public void stop() {
		try {
			thread.join();
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
	}
	
	public void run() {
		Path dest = Paths.get("test.zip");
		Path src = Paths.get("Test");
		
		Map<String, String> env = Collections.singletonMap("create", "true");
		
		FileSystemProvider zipProvider = FileSystemProvider.installedProviders().stream()
				.filter(provider -> provider.getScheme().equals("jar")).findFirst().get();
		
		try(FileSystem fs = zipProvider.newFileSystem(dest, env)) {
			Path root = fs.getPath("/");
			Files.walk(src).forEach((Path sourcePath) -> {
				try {
					Path destination = root;
					for(Path p : src.relativize(sourcePath))
						destination = destination.resolve(p.toString());
					
					System.out.println(destination.toString());
					if(!Files.isDirectory(destination) || !Files.isDirectory(sourcePath))
						Files.copy(sourcePath, destination);
				} catch (IOException e) {
					e.printStackTrace();
				}
			});
		} catch (IOException e) {
			e.printStackTrace();
		}
		done = true;
	}
	
	public static void main(String[] args) {
		ZipTest test = new ZipTest();
		test.start();
		
		if(test.done)
			test.stop();
	}
}

 

 

Thx in advance.

Bektor

Developer of Primeval Forest.

Link to comment
Share on other sites

I had the same problem, I created ZipFileSystems and then accest them via getFIlesSystem, but sometimes getFileSystem returned null so my code created a new one resulting in 2 open ZipFileSystem to the same file, when then all get closed both throw an error, but they leave themselfes in the FileSystem list so you cant create a new one since there is one but this is marked as closed, thus unusable. My fix was a java update to the latest version, this removed all the strange cases where the file system randomly broke and I made a map to store all FIleSystems myself.

catch(Exception e)

{

 

}

Yay, Pokémon exception handling, gotta catch 'em all (and then do nothing with 'em).

Link to comment
Share on other sites

37 minutes ago, Bektor said:

This makes me wonder why those classes are still around in Java. I've used them since I started with Java in Java 7 times.

Because they are low-level building blocks that are used by the more high-level classes like ExecutorService. It's the same as asking why we still have arrays if there is ArrayList.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share



  • Recently Browsing

    No registered users viewing this page.

  • Posts

    • Hmm, I might just do some sort of jimmy rig to get around this. Do something like make a block model in blockbench that is slightly large than a normal block (something like 0.01 on each side to prevent z-fighting), draw the textures and just have the blocks act as a multipart. (I really hope this can end up transparent and not just blackout the texture underneath) Might be messy, but I really just want to get this done to move onto something else (been working on this one feature for maybe a month because IRL stuff). Thanks for the suggestions anyways mate (I can probably use them for some other crazy idea later).
    • I'm now facing an issue when serializing and deserializing, I better know how NBT works but when using Gson I can only serialize to a file which looks like this:  {"mainInventory":[{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}}],"armorSlots":[{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}},{"tagMap":{"id":{"data":"minecraft:air"},"Count":{"data":1}}}]} But when it comes to deserialization, Gson doesn't work... I won't upload the full log but important bits I cannot understand: [11:32:13.645] [Server thread/ERROR] [EventBus/EVENTBUS]: Exception caught during firing event: Unable to invoke no-args constructor for interface net.minecraft.nbt.INBT. Register an InstanceCreator with Gson for this type may fix this problem. Index: 1 Listeners: 0: NORMAL 1: ASM: class com.example.examplemod.Event.PlayerEvents onPlayerChangeDimension(Lnet/minecraftforge/event/entity/player/PlayerEvent$PlayerChangedDimensionEvent;)V java.lang.RuntimeException: Unable to invoke no-args constructor for interface net.minecraft.nbt.INBT. Register an InstanceCreator with Gson for this type may fix this problem. Caused by: java.lang.UnsupportedOperationException: Interface can't be instantiated! Interface name: net.minecraft.nbt.INBT Thank you in advance, and sorry for the long waiting time.
    • In this Class:  package net.the_goldbeards.lootdebugs.Server; import net.minecraft.util.ResourceLocation; import net.minecraftforge.fml.network.NetworkRegistry; import net.the_goldbeards.lootdebugs.LootDebugsMain; public class SimpleChannel { private static final String PROTICOL_VERSION = "1"; public static final SimpleChannel CHANNEL = NetworkRegistry.newSimpleChannel( new ResourceLocation("corruption","main") ,() -> PROTICOL_VERSION , PROTICOL_VERSION::equals , PROTICOL_VERSION::equals); }  
  • Topics

  • Who's Online (See full list)

×
×
  • Create New...

Important Information

By using this site, you agree to our Privacy Policy.