WriteFile occassionaly truncates files entirely

[klauern]

I am trying to write a simple sed -i script using this, but it seems that when I run it on the same path that I am reading it from, occasionally the file is completely truncated instead of written with the piped changes.

func Replace(path, from, to string) error {
	pipe := script.File(path)
	file, err := pipe.Replace(from, to).WriteFile(path)
	if err != nil {
		fmt.Println(path, err)
		return fmt.Errorf("error writing file %s: %w", path, err)
	}
    return nil
}

I can make this work by pulling the write operation out of the pipe:

func Replace(path, from, to string) error {
	pipe := script.File(path)
	file, err := pipe.Replace(from, to).String()
	if err != nil {
		fmt.Println(path, err)
		return fmt.Errorf("error writing file %s: %w", path, err)
	}
	return os.WriteFile(path, []byte(file), 0644)
}

[klauern]

What's frustrating is that this isn't the case all the time, so it does work for a good percentage, so there is some kind of weird race condition that is causing it not to work. I am not running any of these calls in parallel, so I am pretty sure it's how the file is handled in bitfield/script itself.

[bitfield]

I'm not sure it's safe to write to a file at the same time as reading from it—or rather, it's safe, but might give you puzzling results, as you've experienced. The reads and writes will be buffered, which means the bits don't necessarily show up on disk when you expect them to.

[klauern]

I was thinking that, too. I thought maybe the Wait() command would let me do that, but since it doesn't return a *Pipe, that seemed like a dead-end.

[andrew-werdna]

I've encountered the same issue as well. Filters run concurrently as stated in the readme, and it doesn't look like there is an option to enforce consecutive execution. I was trying to do the exact same thing as you @klauern and I solved it the same way too.

[bitfield]

Another option that avoids reading the whole file into memory would be to pipe it to a temporary file, and then once the pipe has completed, rename the temporary file to the original file.