i have a 3TB drive almost full of tv shows, and it is painfully slow sometimes. once i use it its fine, navigation doesnt hang up just browsing. but if i leave it for about 30 seconds or so then try to double click a folder, it doesnt respond for a good 5 seconds or more. at first i assumed that it was just a slow drive but i switched from my hitachi to my seagate and its still the same problem. i think its just too many folders. is there something i can do to permanently index the drive or something so it doesnt have to think so long each time? thanks.
![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
access delay on 3TB with LOTS of folders
- Thread starter ekuest
- Start date
Depends on how many folders and files (sizes) there actually are, but in general you start to see negative results with hard drives (even SSD's from what I've heard) when you get near filling it all up. For a mechanical drive that starts to come @ 50% full, plus it's a massive amount of filled space the heads have to move over and process to find the specific file you're trying to access.
Literally becomings finding a needle in a haystack.
Literally becomings finding a needle in a haystack.
Old Hippie
Supreme [H]ardness
- Joined
- Oct 31, 2005
- Messages
- 6,013
QFT.
This is one of the drives on your HTPC, right? Do you get the problem when you access the drive locally or remotely or both? How many files are in the folder? IIRC NTFS can have a problem at about 17K files, but that was patched ages back.
Have you checked the event logs?
Have you checked the event logs?
How many files to you have in an individual folder?
Or how many folders within a given parent folder?
putting thousands (or even hundreds) of file entries int he same directory will swamp just about any operating system...Windows more so than Unix, but even linux has problems once you get into the thousands.
Or how many folders within a given parent folder?
putting thousands (or even hundreds) of file entries int he same directory will swamp just about any operating system...Windows more so than Unix, but even linux has problems once you get into the thousands.
thanks for the replies guys! ill try to answer all the questions from memory, im currently at work.
1) OS is windows 7 x86, i think professional.
2) current status is ~120GB free of 2.82TB.
3) paths are A:\TV Shows\SHOW\Season\file, with probably 100+ shows and however many seasons of each of folders inside, then whatever numbers of episodes per season in those. some are 1 season, some are 20+. some are 3 episodes/season, some are 24. all told, im sure there are many hundreds of directories and thousands of files. :/
4) problem is the same locally and remotely. i click on the shortcut to my TV Shows folder, and it takes about 5-10 seconds before it loads. fine after that.
5) i have not checked event logs. i dont think its an error, just the computer being slow from too many files in too many folders.
6) i dont think any single folder has more than about 50 files/folders in it. most are around 1-10 since most shows dont last longer than 10 seasons.
i used to have the same problem using the search feature in windows for any file, taking a couple minutes even. it popped up a suggestion that i index the drive or something and now i can search pretty fast. i'm looking for something similar that i can do to my TV drive so that it's a little more responsive. if there's nothing i can do then i wont worry about it, it's really just a very minor annoyance.
1) OS is windows 7 x86, i think professional.
2) current status is ~120GB free of 2.82TB.
3) paths are A:\TV Shows\SHOW\Season\file, with probably 100+ shows and however many seasons of each of folders inside, then whatever numbers of episodes per season in those. some are 1 season, some are 20+. some are 3 episodes/season, some are 24. all told, im sure there are many hundreds of directories and thousands of files. :/
4) problem is the same locally and remotely. i click on the shortcut to my TV Shows folder, and it takes about 5-10 seconds before it loads. fine after that.
5) i have not checked event logs. i dont think its an error, just the computer being slow from too many files in too many folders.
6) i dont think any single folder has more than about 50 files/folders in it. most are around 1-10 since most shows dont last longer than 10 seasons.
i used to have the same problem using the search feature in windows for any file, taking a couple minutes even. it popped up a suggestion that i index the drive or something and now i can search pretty fast. i'm looking for something similar that i can do to my TV drive so that it's a little more responsive. if there's nothing i can do then i wont worry about it, it's really just a very minor annoyance.
Windows doesn't usually choke until you break 4096 files/folders in the directory, but a number of file scanners/malware detectors/anti virus apps magnify the problem as well. If you temporarily disable any of these apps running on your machine, does the problem at least subside a little?
i dont have anything except mse on my computer. :/ peerblock/my movies/utorrent/firefox/steam are pretty much always running in the background. i wonder if my movies might have something to do with it? i do have most of the tv collection (95%+) indexed in my movies. i will try turning off the tray tool and see if that helps at all.
houkouonchi
RIP
- Joined
- Sep 14, 2008
- Messages
- 1,622
Its probably fragmented btree's on the directories. Indexing the drive might help (if its disabled). On linux you can do echo 0 > /proc/sys/vm/vfs_cache_pressure to disable the clearing of inode/dentry cache for disk cache which means file system metadata stays mostly cached and speeds things up grately but no such thing on windows AFAIK.
well whatever the problem is, it's persisted even after i switched from a 3TB hitachi 5400rpm to a 3TB seagate 7200rpm and copied all the files. i didnt even clone the drive in case that might also copy any directory or whatever problems, just drageed and dropped the whole folder. i will check to see if indexing is on too, but i never turned it off so i assume it is.
hotcrandel
Gawd
- Joined
- Feb 26, 2010
- Messages
- 781
I'd suggest
1. check if its running defrag
2. check if the drive is set to spin down
3. check the SMART information.
1. check if its running defrag
2. check if the drive is set to spin down
3. check the SMART information.
ho do you check if its set to spin down? BIOS? it might be my movies inconclusive so far. def not defragging as its been going on for almost a year. havent checked SMART either. i have mon off and my gf doesnt so i will probably get to some diagnostics then at the earliest.
from what you're describing (providing HD is healthy, no SMART flags), it sounds like the drive is spun down/idle/sleep mode when first trying to access the HD.
type 'power options' in start search.
click link 'change plan settings' beside the plan that is currently selected (balanced/high performance).
then click 'change advanced power settings'
in 'power options' window, drill down the 'Hard Disk' tree (hard disk->turn off hard disk after->setting)
there you can set the time frame till when the disk spins down in minutes.
if you 3TB drive is only storing large video files, fragmentation shouldn't be a huge issue tho a regularly scheduled defrag should be enabled.
open up defrag. if you have an SSD, windows should have removed defrag from list (never run).
the other platter disks should have something listed under 'last run'. if they haven't, setup a weekly run under 'configure schedule..' button.
obviously as the drive become full, disk performance degrades, probably look at getting another big disk to spread your files across. an ideal solution if you are rapidly filling your disk, maybe look at setting up a seperate NAS box.
not directly related, a good tool at getting a excellent visual representation of your disk contents is 'WinDirStat'. lets you quickly see what gobbling up disk space.
type 'power options' in start search.
click link 'change plan settings' beside the plan that is currently selected (balanced/high performance).
then click 'change advanced power settings'
in 'power options' window, drill down the 'Hard Disk' tree (hard disk->turn off hard disk after->setting)
there you can set the time frame till when the disk spins down in minutes.
if you 3TB drive is only storing large video files, fragmentation shouldn't be a huge issue tho a regularly scheduled defrag should be enabled.
open up defrag. if you have an SSD, windows should have removed defrag from list (never run).
the other platter disks should have something listed under 'last run'. if they haven't, setup a weekly run under 'configure schedule..' button.
obviously as the drive become full, disk performance degrades, probably look at getting another big disk to spread your files across. an ideal solution if you are rapidly filling your disk, maybe look at setting up a seperate NAS box.
not directly related, a good tool at getting a excellent visual representation of your disk contents is 'WinDirStat'. lets you quickly see what gobbling up disk space.
Try changing the folder optimization from general to documents (properties->customize->"Optimize this folder for:") and disabling indexing for the entire folder if it's enabled.
Another possibility that others have already brought up is the power save...but if it still happens with different drives then I'm not sure that's the cause.
Another possibility that others have already brought up is the power save...but if it still happens with different drives then I'm not sure that's the cause.
from what you're describing (providing HD is healthy, no SMART flags), it sounds like the drive is spun down/idle/sleep mode when first trying to access the HD.
type 'power options' in start search.
click link 'change plan settings' beside the plan that is currently selected (balanced/high performance).
then click 'change advanced power settings'
in 'power options' window, drill down the 'Hard Disk' tree (hard disk->turn off hard disk after->setting)
there you can set the time frame till when the disk spins down in minutes.
if you 3TB drive is only storing large video files, fragmentation shouldn't be a huge issue tho a regularly scheduled defrag should be enabled.
open up defrag. if you have an SSD, windows should have removed defrag from list (never run).
the other platter disks should have something listed under 'last run'. if they haven't, setup a weekly run under 'configure schedule..' button.
obviously as the drive become full, disk performance degrades, probably look at getting another big disk to spread your files across. an ideal solution if you are rapidly filling your disk, maybe look at setting up a seperate NAS box.
not directly related, a good tool at getting a excellent visual representation of your disk contents is 'WinDirStat'. lets you quickly see what gobbling up disk space.
i suspected that could be a problem too, but i just checked and its set to spin down after 20 minutes. i have the problem within 2 or 3 minutes of the last time i accessed the drive. i also just tried my movies drive (40GB free, tv is 174GB free) and it lags too when its first opened. so i do think its just getting too full. i will try defragging tonight.
tried the folder optimization thing and that didnt help either. :/
Chrisroman
Gawd
- Joined
- May 29, 2011
- Messages
- 692
Have your run perfmon? That may help shed some light.
Since it's not spindown, fragmentation, or the individual drive itself. It's possible one or more of the video files have some sort of corruption. Deleting those files and re-ripping them. Also disabling thumbnail view for now if you are using that by chance and using details view will help.
Like the other guys here I can vouche that of the handful of 3TB drives I have and from diff manufacturers, they are all fast when it comes to general access, including thumbnailing, even when they are completely full. So it may be a problem with one or more of the files. How large are each of the files on average?
One thing I have definitely noticed though is on very large music databases there has always been a slight delay when switching tracks in playlists that happen to be on different parts of the disk. Anywhere from 1-3 seconds.
They are also painfully slow with large quantities of simultaneous small file editing. Good thing I don't use them for that though.
This is all in relation to the 5400-5900 series. I've found large platter "green" series to be good for most things, movies, archives, photos, etcs. And 7200's for large music databases.
Like the other guys here I can vouche that of the handful of 3TB drives I have and from diff manufacturers, they are all fast when it comes to general access, including thumbnailing, even when they are completely full. So it may be a problem with one or more of the files. How large are each of the files on average?
One thing I have definitely noticed though is on very large music databases there has always been a slight delay when switching tracks in playlists that happen to be on different parts of the disk. Anywhere from 1-3 seconds.
They are also painfully slow with large quantities of simultaneous small file editing. Good thing I don't use them for that though.
This is all in relation to the 5400-5900 series. I've found large platter "green" series to be good for most things, movies, archives, photos, etcs. And 7200's for large music databases.
Last edited:
Don't rule it out just yet...
Windows is extremely aggressive with its power saving behaviour. If you simply access a directory while a disk is sleeping, Windows will spin your disk up to read the file table and then spin back down immediately no matter how many minutes you have set. So the next time you try and access a folder on the disk Windows will spin it up and down again even if its just a few minutes later.
Try setting your HDs to "never" spin down in Windows power options, and see if the problem goes away.
You can also download Smartmontools and run "smartctl.exe -i -n standby c:\" to see if a drive is currently spun down or not.
oh ok thanks. i didnt know that. yeah i will try setting it to never and see what happens. although does that mean that the computer will never sleep? i do like the power saving options as this computer is not in use for most of the day and is only on from 3pm-10pm at the most.