Is this indicative of silent data corruption?

[iroc409]

About two years or so ago I switched over from a barebones FreeBSD file server with ZFS to Synology (for ease of replication--2 Synology units--and small form factor). At that time I switched from centralized data to local data for better file performance.

I use Advanced Business Backup for the Synology to do a rsync pull from my main desktop (OpenSuSE). I do a lot of photography, so I have a whole sort of archive folder system with my historical photography in it. I've had a lot of issues with ABB, and Synology is still trying to work it out, but part of going through the logs I've noticed it is backing up new versions of files from these photography archives that are essentially static data.

It seems like a head scratcher but with the other issues I basically ignored it. I just did an offsite backup, which consists of a direct rsync from the desktop to an external drive (it's ext4 or xfs can't remember). I was doing a checksum-based update and it moved one of the photo files, because it didn't pass checksum.

I've never run into this issue in a very long time, due to mostly running centralized file storage on ZFS in a proper server. If the CRC check is failing, that must mean the file is changing. I opened the file and everything looks fine, but this seems like the leading edge of a larger issue--and there are tons of files that it's transferred over the last couple of years. Should I start hunting some sort of other issue?

I have been trying to back up the entire ABB data set to an external drive, but the Synology thinks the data set is 417TB (it's about 3.8TB of data), so it fills up the drive before I can make a backup (?!).

 

[SamirD]

Ah yes, bit rot.

I too have a large archive of my photography (about 700,000 photos and videos), and I use a '3 set original' strategy with the original file written to 3 drives and then compared before the original is deleted off the memory card. Then, about every year or so, I will compare all 3 sets of drives with each other. Any file that has changed when there's two more than haven't is the 'bit rot' one and is overwritten with a good original.

This has basically been my type of goto for data corruption prevention, and this was before zfs was prevalent. I still haven't started using zfs, so I'm still comparing originals and their copies regularly to hunt for corruption.

And what you're describing definitely sounds like some sort of data corruption which is actually being propegated by the synology backup program and destroying your originals. I hope you have a third backup somewhere that is untouched so you can restore the mess that has been made.

And it's because of stories and experiences like this that I avoid any type of 'smart' or 'automatic' backup system--when they do something unpredictable, it usually results in data loss. I only go as far as using robocopy, but still trust xcopy more than robocopy and really should be using xxcopy instead of robocopy because it has made messes for me too.

 

[uOpt]

If an rsync without -c (no checksumming) shows unexpected re-transmit of old, static files then it is not likely that it is silent corruption.

Silent corruption of the contents of a file doesn't update the timestamp, hence rsync doesn't notice the change.

Your external drive error, which is presumably on USB, is more likely to be a USB error than RAM corruption or somesuch.

 

[SamirD]

Silent corruption of the contents of a file doesn't update the timestamp, hence rsync doesn't notice the change.

I forgot about this--yeah a corrupt file will generally not copy because the timestamp would be the same and you'd only notice the corruption on a file compare.