Defragmenting Drives with Large Cluster Size

M

mikesong

New member
Ok, I have 2 hard drives both formatted NTFS on a Windows 2000 system...the audio HD has 64k clusters. When I try to defragment it using the stock Windoes defragmenter it says it's unable to defrag NTFS drives w/ large cluster sizes. Is this caused by the large cluster sizes or the NTFS? Does anyone know of any other defrag programs that doesn't have this limitation? If I had it formatted using FAT32 and 64K (assuming that's possible), would the Windows defragmenter be able to work on it? Or how about with 32k on either file system? I'd be willing to reformat the audio hard drive to FAT32 if that would allow me to use large cluster sizes and still defragment it, but are there any drawbacks to running a DAW w/ NTFS on the OS drive and FAT32 on the audio drive?...i.e. any incompatibilities, I like to be able to move files back and forth between drives for backup purposes.
Thanks, Mike
 
although it hasn't crashed on me once and i've used it for dozens of hours i'm not quite ready to pronounce it healthy because i haven't used the audio engine much yet. i'm gonna be doing that right now and once i can say 100% that it's good i'll post on the other thread on how i fixed it. that said...

anyone got any answers for my defragmentation quagmire?
 
How about going around the outside?

It's an audio drive, so why not back up what you want to keep and just format it the way you want. Much faster than doing the defrag anyway.
 
i'm not sure what you mean - are you saying just backup and reformat everytime i need to defrag? if i just have one defragmented file it's much easier and quicker to be able to defrag.

i'd still like to know if there's a file system i can format it to that will allow me to defrag? (does anyone have a FAT32 drive w/ 64k clusters on a Win2K system?)
 
I haven't defragged any of my 20 or 40GB Audio Drives in years. No problems here.

What exact problem are you having with fragmented files?
 
i'm not sure if fat32 will defrag but you can try by downloading partition magic pro

this allows you to change the file system without losing data

Tukkis
 
well, i haven't really had any observed performance problems with fragmented files, but when i "analyze" the audio hard drive (40 Gig) using the defrag progam, several of my files show up as having 2 or more fragments. is this not a problem on a 7200 RPM? me just being paranoid and wanting the best performance, i'd assume unfragmented files are the best way to ensure sample-accurate playback and get the midi and audio sync'd, etc. however i have seen numerous posts by slackmaster2k, emeric, et. al, stating that they don't defragment and i've never understood why. if it's just the fact that it's time-consuming and a pain-in-the-arse, i don't mind because i can do it overnight.
 
If it's not causing any problems I wouldn't worry about it. NTFS doesn't need to be defragged as religiously as Fat32. I'm afraid I can't answer you specific question but you might find some info here:

www.tomshardware.com
 
Well, 2 fragments is not really much, so you won't notice it. My experience with ntfs is that you better leave it alone, in some instances it got worse after defrag. If it gets really slow, then the backup-restore scenario works well for me, just from drive a to drive b back to a. Lots also depend on how the audio application works. CEP makes a temp file that it uses while working, so if the stored files are fragmented, it does not matter, it only takes a little more to build the temp.
 
Why the hell do you have 64k cluster sizes? Now days a hd with 64k cluster sizes should only be seen on 500+ gb hard drives (if there is such a thing). Using 64k clusters is very inefficient use of your hard drive especially if you are going for optimal performance.

From your post, it sounds like you are now sure what it means to defrag your comp, here we go: When you write to your hd, it writes files one after another from the inside out (i think..., it's not important) as the platter spins, putting the data into induvidual clusters, if the file is 100k, it uses 2 clusters(when using these 64k clusters). Now if a month later you go and delete this 100k file after saving more data, there is this gap between all your saved data. The next time you save a file (say it's 150k), it is written there until those 2 clusters (128k) is taken up, then it jumps to the next available cluster on the hard drive. With multiple files splitted up into many pieces, it can be cluttered, and at this point, your hard drive is fragmented. When you defrag, your computer is taking these spaced out chunks of your file and re-writes them so that they are all in clusters that are lined up one after another. When your hard drive is defragged, it doesn't have to skip all over the place finding all the individual chunks of your file, thus more efficient. (I hope that isn't confusing, if it is, tell me I'll explain more)

The reason your computer is having problems or simply will not defrag your hard drive is because it would be very processor intensive for your computer to be moving multiple 64kb clusters around your hard drive instead of just 2kb clusters. I think you're kind of confussed on the concept of clusters, trust me, the smaller, the better (for once, some people got to hear what they wanted).

Also, when using 64k clusters, and you save a 129k file, it HAS to use 3 clusters. And that's 63k of hard disk space wasted, it cannot be used until that file is deleted, and that cluster is freed. For this reason it would be more wise to use 2k (FAT 32) or 4k (FAT 16) clusters. I am also using a 40 gig hd split into 3 partitions that are all each FAT 32. Using smaller clusters will more efficiently use the disk space you have.

QUOTE:
"but are there any drawbacks to running a DAW w/ NTFS on the OS drive and FAT32 on the audio drive?..."

There shouldn't be any drawback to storing your OS on the NTFS drive and FAT 32 on your music drive. But using the FAT 32 (2k clusters) in place of the 64k clusters will be a godsend.

QUOTE:
"i'd still like to know if there's a file system i can format it to that will allow me to defrag? (does anyone have a FAT32 drive w/ 64k clusters on a Win2K system?)"

The FAT# (File Allocation Table) doesn't describe your drive exactly, it describes the size of the clusters. FAT32 has 2k clusters.

QUOTE:
"several of my files show up as having 2 or more fragments. is this not a problem on a 7200 RPM?"

The speed of your hard drive should play no part as to whether a file is fragmented.

QUOTE:
"i'd assume unfragmented files are the best way to ensure sample-accurate playback "

Correct. It would help prevent uninterrupted playback.

I would recommend that the next time you back up your music drive, you reformat it, using FAT16 (4k clusters) for one partition or FAT32 (2k clusters) for more than one. Also, defrag once a week using, Windows' generic defragger, if this is a drive you edit a lot. And with this you should get optimal performance (even though you experience no problems yet). I hope this lengthly post answered all of your questions and doubts, if it didn't I would be glad to explain more clearly.
 
64k clusters are not abnormal when using audio. Most of the files I work on are 40MB up to 1.4GB. So losing 63kB is peanuts. As for the record, you can use larger clusters with fat32, 2k is standard, but nothing to stop you. Why would it be more cpu intensive to defrag? It is in fact easier as there are less data entries in the fat to handle, and the amount of data to transfer stays the same.

You seem to be a bit confused on the concept of partitions. There is no need to format a disk in fat16 if you have 1 partition, or fat32 if more partitions. Each partition needs to be formatted seperately in whatever you want. You can use on the same disk fat16, fat32 and ntfs (in different partitions of course).

The influence depends on the application. If it reads directly from the file, then yes. If it first makes a copy to a temp, then no (unless the temp file is fragmented of course).
 
True, when recording a 1.4 Gig file, there can be an advantage to having 64k clusters. The reason it's processor intensive and probably the reason he can't defrag is because it would be much easier to be swapping multiple 2k clusters rather than 64k clusters. I could almost guarantee that if he went down to at least 16k clusters, he would be able to defrag with no problem.

As for the topic on partitions, I not not confussed at all. It seems to be a general rule in the world of computing (for the sake of efficiency) that the larger the partition/hard drive, the larger the size of the clusters you use. I would not use 2k clusters for a 40 gig partition. But I would if the partition were 10 gig on that 40 gig drive.

Taking into account that some files that are being used are 1.4 gig, I would probably use 8 or 16k partitions.
 
Noop, it says "cannot defrag with clusters larger than 4k" The problem is that the standard windoze defrag is a downsized application of a commercial defrag tool. Give it a try with the demo of Diskeeper, see what that one says.
 
I believe that optimal cluster size is dependant more on the type of application you are referring to. I am a DBA and for an Oracle install it is recommended to use 32k cluster sizes due to the way Oracle reads and writes to the disk.
 
you were saying that fat32 2k is optimal...

i thought NTFS was?

i know nothing about fat32/ntfs differences...just that for audio - ntfs = better...

am i right? ha.
 
NTFS is actually slightly slower than FAT32 in audio performance. I'm not totally sure why this is, but I know NTFS also supports a number of features that may be useful to you that FAT32 does not support. In any case, the dropoff in performance is pretty small, and gets even smaller when you consider how fast computers are getting nowadays. Maybe we're talking about the difference between 90 and 92 tracks... something on that order.
 
>on the concept of clusters, trust me, the smaller, the better

For audio files this is stuff and nonsense!

What's 64KB out of 100MB? Right- that ain't doodley squat!

The performance edge from using smaller clusters accrues when writing a lot of small(er) files, because it's obviously wasteful to store a 10 byte file by using up a 64KB cluster.
 
NTFS vs. FAT32

If I recall correctly, the reason NTFS is slower is because the NT filesystem manages more metadata per file, things like permissions for user, group, and everyone. Most how-tos I've read advise against using NTFS for audio. The most common recommendation is FAT32. FAT16 is limited to 2GB partitions. I don't know about you, but I just bought an 80GB drive for audio, and I'm damned if I'm going to try to manage 40 2GB partitions! There aren't even enough letters (C through Z, minus any CDRs or other devices) to manage those partitions.

There are no issues running different filesystems on different partitions.

I have used Partition Magic, and it is a wonderful program. However, it would not allow me to resize the clusters of my FAT32 file systems to 64k. It was simply marked as "unavailable". Perhaps my partitions were not large enough. I did convert all my audio partitions to 32k. 64k is supposed to result in better performance for audio, since the CPU is less bothered with creating or skipping around to new clusters.

I am a very firm believer in defragging. Keeping your files defragged simply helps speed. I won't believe anyone who tells me that defragging hurts performance. I defrag every night I do audio work. I use the standard Win2K defragger, which is not the best program, since it's a stripped down version of DiskKeeper. You can't automate/schedule it, you have to kick it off manually, and only do one partition at a time (I found some hacks on the web for automating it with scripts, tried that once, didn't quite nail it). I also understand it doesn't defrag folders, or some such nonsense. Someday I might buy a better defragger. A defragger should be standard issue, I can't believe Microsoft didn't include a full-fledged one with NT when they have one for 98.

If you have an old Win98 install disk lying around, you might consider installing a dual boot. Then you could use the schedulable Win98 defrag utility, which will result in a nice, complete defrag. This is inconvenient, tho, and setting up dual boot is not for the faint of heart. Also, Win98 can't read NTFS. If you're multiple OSes, be aware of what OSes can access what file systems.

With all those DirectX and VST plugins we run, you need to save every CPU cycle you can, especially on older computers. Using large cluster sizes and defragging helps.
ElSilva
 
According to a prorec.com article on optimizing audio drives in Win2000, "Any time you format a disk or partition with FAT32, be sure to use the /z:64 switch (format [drive]: /z:64) to get 32K cluster sizes. This is critical to getting the best performance from the disk." I think my vague recollection of reading this before led me to use 64k because that is the number in the switch instead of the 32k they recommend. However, it would make sense to me that 64k would be better as you said ElSilva, but I'd go with 32k if you say you are able to defrag with it under Win2K. However, I just figured out I don't even have the option of FAT32 because it can only handle 32GB drives in Win2K and mine is 40GB.

So my question now is does anyone use Win2K with NTFS drives w/ 32k clusters, and are you able to defrag w/ the standard defragmenter? I consider it mandatory that I defrag because it is common for me to record several gigs of data per day spread over several hundred files, some of which get deleted, and I'm sure it would be a big mess if I didn't defrag.
 
You must log in or register to reply here.
Back
Top