Page 1 of 3 123 LastLast
Results 1 to 10 of 26

Thread: OT: Cluster Size in the 21st Century

  1. #1

    Default OT: Cluster Size in the 21st Century

    Are we still formatting our audio HDDs in 64K clusters? Or is our throughput sufficient these days that we don't worry about it, anymore?

    Sound off.
    Dave "it aint the heat, it's the humidity" Labrecque
    Becket, Massachusetts

  2. #2
    Join Date
    Oct 2009
    Location
    Maple Ridge, BC Canada
    Posts
    1,846

    Default Re: OT: Cluster Size in the 21st Century

    Dave,

    I have seen heavy-use systems that leave "all as it is" - that is, at 512. So, does it really matter?

    From my research - which I do admit is somewhat dated...I found that 64K is preferred for data drives. There was no real benefit for apps drives.

    Now, if you are using on a single partition, the default is 512K clusters...but with some HHD tools you can alter the cluster sizing without data loss.

    So, my recommendation is: 512K on boot drive, and 64K on data drive(s).

    Hope this helps...but I am waiting to see what Philip has to say.

  3. #3

    Default Re: OT: Cluster Size in the 21st Century

    Quote Originally Posted by mr_es335 View Post
    Dave,

    I have seen heavy-use systems that leave "all as it is" - that is, at 512. So, does it really matter?

    From my research - which I do admit is somewhat dated...I found that 64K is preferred for data drives. There was no real benefit for apps drives.

    Now, if you are using on a single partition, the default is 512K clusters...but with some HHD tools you can alter the cluster sizing without data loss.

    So, my recommendation is: 512K on boot drive, and 64K on data drive(s).

    Hope this helps...but I am waiting to see what Philip has to say.


    Dell, you got me thinking. 512 didn't sound familiar to me. (I assume you mean 512 bytes and not KB.) I poked around (a little) and found this:

    https://support.microsoft.com/en-us/...fat--and-exfat

    They have default cluster sizes per file system and volume size listed. The only 512 I see is for a very small volumes, FWIW. Looks like for the realms we deal with (a few TB max on NTFS), they say the default is 4K.

    That said, it appears to be an older web page, since the most recent OS they list is Windows 7.

    I checked my partitions for which I went with the default value when formatting, and they're both at 4K.

    Maybe you meant sector size, Dell? My boot partition shows that as being 512 bytes. Or do you just like really small clusters on your boot partition?
    Dave "it aint the heat, it's the humidity" Labrecque
    Becket, Massachusetts

  4. #4
    Join Date
    May 2004
    Location
    Melbourne, Australia
    Posts
    347

    Default Re: OT: Cluster Size in the 21st Century

    Dave;

    The way I understand it is: For Audio files, where maximun throughput is desirable use the largest cluster size available (without checking 64kb, could be larger with terrabyte volumes, not sure).
    The disadvantage is that on average you will lose half the cluster size as wasted space, so if you have something that uses many very small files i.e. a couple of KB, you tend to waste (many files) x 32kb per file
    instead of (many files) x 256b.
    As disk space is cheap and audio files are large, it makes sence to go for the largest cluster size and therefore the maximum transfer speed from the hard drive on a DAW computer.

    Hope that clarifies it!

    Cheers!
    RobertV

    www.shinustudios.com

  5. #5

    Default Re: OT: Cluster Size in the 21st Century

    Quote Originally Posted by RobertV View Post
    Dave;

    The way I understand it is: For Audio files, where maximun throughput is desirable use the largest cluster size available (without checking 64kb, could be larger with terrabyte volumes, not sure).
    The disadvantage is that on average you will lose half the cluster size as wasted space, so if you have something that uses many very small files i.e. a couple of KB, you tend to waste (many files) x 32kb per file
    instead of (many files) x 256b.
    As disk space is cheap and audio files are large, it makes sence to go for the largest cluster size and therefore the maximum transfer speed from the hard drive on a DAW computer.

    Hope that clarifies it!

    Cheers!
    Hi Robert,

    Yes, I know the rationale. I just wonder if it's still worth it in this age of way-fast throughput, regardless of cluster size. Are we fast enough these days that cluster size isn't a practical factor anymore? Does it matter to us in our workflow in 2017? That's what I'm curious to know. I know the "physics" is still valid.
    Dave "it aint the heat, it's the humidity" Labrecque
    Becket, Massachusetts

  6. #6
    Join Date
    Jul 2006
    Location
    SF Bay Area
    Posts
    944

    Default Re: OT: Cluster Size in the 21st Century

    Quote Originally Posted by Dave Labrecque View Post
    Hi Robert,

    Yes, I know the rationale. I just wonder if it's still worth it in this age of way-fast throughput, regardless of cluster size. Are we fast enough these days that cluster size isn't a practical factor anymore? Does it matter to us in our workflow in 2017? That's what I'm curious to know. I know the "physics" is still valid.
    You don't have to concern yourself with cluster size and it hasn't really mattered for a long time.
    ---------------------------------------
    Philip G.

  7. #7
    Join Date
    Oct 2009
    Location
    Maple Ridge, BC Canada
    Posts
    1,846

    Default Re: OT: Cluster Size in the 21st Century

    Dave,

    From an acquaintance of mine who knows a lot more about hardware than I ever did:

    "In terms of space efficiency, smaller allocation unit sizes perform better. The average space wasted per file will be half the chosen allocation unit sizes. So 4K wastes 2K per file and 64K wastes 32K. However, modern drives are massive and a little wasted space is not worth fussing over and this should not be a determining factor - unless you are on a small SSD.
    Compare 4K vs 64K average case waste (32K-2K = 30K), for 10,000 files that only comes out to 300,000KB or around 300MB.

    Instead think about how the OS uses space. For example, you have a 3K file which needs to grow 2K. With a 4K allocation unit size, the data needs to be split over two blocks - and they may not be together so you get fragmentation. With a 64K allocation unit size, there are a lot fewer blocks to keep track of and less fragmentation. 16x the block size means 1/16th the number of blocks to keep track of.

    For a media disk where you photos, music and videos are stored, every file is at least 1MB I recommend the largest allocation unit size. For a windows boot partition I recommend using the Windows default - which is 4K for any NTFS drive smaller than 16TB."

    Hope you find information this of interest? However, I think Philip is on the right track.

  8. #8

    Default Re: OT: Cluster Size in the 21st Century

    Quote Originally Posted by mr_es335 View Post
    Dave,

    From an acquaintance of mine who knows a lot more about hardware than I ever did:

    "In terms of space efficiency, smaller allocation unit sizes perform better. The average space wasted per file will be half the chosen allocation unit sizes. So 4K wastes 2K per file and 64K wastes 32K. However, modern drives are massive and a little wasted space is not worth fussing over and this should not be a determining factor - unless you are on a small SSD.
    Compare 4K vs 64K average case waste (32K-2K = 30K), for 10,000 files that only comes out to 300,000KB or around 300MB.

    Instead think about how the OS uses space. For example, you have a 3K file which needs to grow 2K. With a 4K allocation unit size, the data needs to be split over two blocks - and they may not be together so you get fragmentation. With a 64K allocation unit size, there are a lot fewer blocks to keep track of and less fragmentation. 16x the block size means 1/16th the number of blocks to keep track of.

    For a media disk where you photos, music and videos are stored, every file is at least 1MB I recommend the largest allocation unit size. For a windows boot partition I recommend using the Windows default - which is 4K for any NTFS drive smaller than 16TB."

    Hope you find information this of interest? However, I think Philip is on the right track.
    The fragmentation angle is interesting. Hadn't considered that. Philip -- how 'bout that? Does it mean defragging more often when keeping with the default cluster size "under" bigger files?
    Dave "it aint the heat, it's the humidity" Labrecque
    Becket, Massachusetts

  9. #9

    Default Re: OT: Cluster Size in the 21st Century

    Quote Originally Posted by cgrafx View Post
    You don't have to concern yourself with cluster size and it hasn't really mattered for a long time.
    Aha! Glad I asked.
    Dave "it aint the heat, it's the humidity" Labrecque
    Becket, Massachusetts

  10. #10
    Join Date
    Oct 2009
    Location
    Maple Ridge, BC Canada
    Posts
    1,846

    Default Re: OT: Cluster Size in the 21st Century

    Dave,

    Notice that I stated, "However, I think Philip is on the right track."

    Also, from what I have read, you should not defrag SSD's.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •