I am working on a project with a goal of storing 40 petabytes in the near term. The project has already been mostly specced out, but I want to see if anyone has any innovative or radically differing ideas. I should add that I am not a technical project leader, and this project was just introduced to me, so I don't have many of the details. Just want to get a feel what what is out there.
Note: Project leader is very cost sensitive. Real-time access is not necessary for 99.99% of the data at any given time. In theory they do eventually want up to 2 PB of data on HDDs. TTL of data must be at least 100 years, with 1000 years preferred.
Please recommend solutions. I can google a lot and will be working with a technical team. However, please specify at least the following details:
1. Primary archival medium and vendor: (e.g. tape and Oracle)
2. Real-time cache medium: (e.g. HDD and Western Digital in SuperMicro bays)
3. File system: (e.g. SAM-FS, Hadoop, ZFS)
4. Security considerations: (e.g. xxxxxx)
5. Other details:
Note: Project leader is very cost sensitive. Real-time access is not necessary for 99.99% of the data at any given time. In theory they do eventually want up to 2 PB of data on HDDs. TTL of data must be at least 100 years, with 1000 years preferred.
Please recommend solutions. I can google a lot and will be working with a technical team. However, please specify at least the following details:
1. Primary archival medium and vendor: (e.g. tape and Oracle)
2. Real-time cache medium: (e.g. HDD and Western Digital in SuperMicro bays)
3. File system: (e.g. SAM-FS, Hadoop, ZFS)
4. Security considerations: (e.g. xxxxxx)
5. Other details:
Last edited: