Well THERE'S my problem: Playback issues for large multi-track projects

PTravel

Senior Senior Member
I've got a couple of projects that are well over 100 tracks each with lots of VSTs and would choke Audition 3.0, even with the largest ASIO buffer setting, on my quad core 8 gig RAID 5 editing computer. I have a quad core hyperthreaded laptop with 16 gig and an SSD, so I tried the projects on that, and they worked fine. My first thought was that the extra memory and faster CPU made the difference. However, last night, I was working with one of the projects that was on USB 3.0 external drive and I noticed that it, too, choked a bit (though not as bad as on my editing computer). Apparently, drive speed is a critical concern in working with really, really large projects. My editing computer has a SSD for its primary disk -- in the future, I'll copy the really large projects over to it; I suspect it will manage them far better.
 
Correct. Hard drive speed limits the number of tracks you can run. The bottle neck is the spindle speed itself. I'm sure the SSD will really open up the track count.

But jeez, if you're only starting to see problems at 100 tracks, that's pretty damn good.
 
do you have separate hard drives for audio and for your operating system? this makes a huge difference. i have managed to run about 150 tracks at 48k 24bit. so try adding a drive for audio as a usb drive does take up cpu usage for usb buss.
 
do you have separate hard drives for audio and for your operating system?
Yep. I thought I mentioned this. The OS and programs are on an SSD and I have a 3 TB RAID 5 internal SATA array for my data.

this makes a huge difference. i have managed to run about 150 tracks at 48k 24bit. so try adding a drive for audio as a usb drive does take up cpu usage for usb buss.
I record and mix at 96K 32bit (and won't change so no one should try to argue me out of it :)). The amount of cycles lost to a hyperthreaded quad core i7 CPU is negligible, particularly when using a program like Audition 3.0, which does not support multiple cores. The internal SATA bus for the SSD maxes out at roughly the same speed as the USB 3.0 bus for the external drive. That the 100 plus tracks runs fine on the SSD but with occasional glitches on the USB 3.0 drive pretty much proves that both spindle speed and actuator speed are responsible for the difference.
 
Yep. I thought I mentioned this. The OS and programs are on an SSD and I have a 3 TB RAID 5 internal SATA array for my data.

I record and mix at 96K 32bit (and won't change so no one should try to argue me out of it :)). The amount of cycles lost to a hyperthreaded quad core i7 CPU is negligible, particularly when using a program like Audition 3.0, which does not support multiple cores. The internal SATA bus for the SSD maxes out at roughly the same speed as the USB 3.0 bus for the external drive. That the 100 plus tracks runs fine on the SSD but with occasional glitches on the USB 3.0 drive pretty much proves that both spindle speed and actuator speed are responsible for the difference.

C'mon.... go 192... you know you want to! :)

I'm absolutely not trying to talk you out of it, but I'm interesting in the reason, particularly the 32 FP (I assume) thing... wondered about that myself..
 
C'mon.... go 192... you know you want to! :)
I would if my interface would support it. :)

I'm absolutely not trying to talk you out of it, but I'm interesting in the reason, particularly the 32 FP (I assume) thing... wondered about that myself..
Okay, this is just my thinking and I'm not an engineer or recording professional. Two words: rounding errors.

2 + 2 + 2 + 2 + 2 = 10
2.1 + 2.3 + 2.2 + 2.1 + 2.3 = 11

For some of what I'm working on, I have to do tons of corrections of various kinds -- Celemony, adjusting length, and lots of VSTs. In my view, the more accurate the original data, the less likely it is to be distorted or create unintended artifacts due to rounding errors.
 
I think your problem is not Audition's fault. (...and btw, version 3.0 DOES support multiple cores.) I would be inclined to place the blame on the controllers for the drives, not within the drive themselves. Let me explain.

USB ports do not have a dedicated I/O controller like your other drives. All data coming and going to the USB drive is routed through your main processor which is already swamped with vst calculations. Data flow to the USB drive probably never reaches max speed except for when the main processor is not busy with other tasks. (Good luck waiting for that to happen.)

Raid 5 has poor performance because of the overhead associated with parity calculations. Raid 0 would have been a better choice for speed. Better still, avoiding Raid altogether and using a single 1TB or 2 TB drive for data. Big Sata 3 drives are cheap now-a-days and more reliable too. Other than redundancy, there is not much reason to use Raid anymore. Raid controllers must do a lot of extra work slowing the read/write operations.

Here is something you can try. Adobe Audition uses temporary files to do its background work. Higher track count causes additional drive activity due to these temp files constantly being written, read, and deleted. You can go into Audition Preferences and set the "Primary Temp File Directory" to be on the SSD drive. Then set the "Secondary Temp File Directory" to be on a separate drive, (your storage drive is fine for this). This spreads the workload between two drives which should speed things up.

Ultimately...
Your Audition program files should be on the boot drive (the SSD).
Your session files should be saved onto and restored from your largest storage drive.
Your temp file operations should make use of the fastest drive as well as a second drive.
Set all internal operations to be done at 32 bit.

Most of us believe that recording at 96K sample rate is not necessary because we can't hear the difference from 41 or 48k, but that is your prerogative. If you hear a noticeable difference and have a machine that can handle it, by all means, knock yourself out.

Hope this helps.
 
Last edited:
RAID 0 does not give redundancy, it is supposed to increase performance, but this is usually for small data writes (1K or less used in database transactions), and to increase storage capacity. Lose 1, loose all. Raid 2 gives the mirror and is used for redundancy/spillover in case of system failure.

Using RAID for non-business computers/servers really makes no sense.

Using a 1TB HD for recording is also a mistake. The increased capacity also increases the read/seek/write times (it is bigger). Other than SDD, a 500 GB or less, 7200 RPM, SATA HD would probably give the best performance.

500 GB for projects should be plenty of space, push everything off to a storage server (I use a NAS, not USB) with an image of the OS disk stored in case of failure would be plenty for any home/professional recording studio.

RAID is bad, Don't do RAID, um'K (really, it is meant for servers). Some geeks think they get a performance boast, but I don't see how.
 
Thanks, RawDepth and DM60 -- some interesting ideas.

A couple of points unique to my situation:

1. Why RAID 5 rather than a single smaller drive: There are a few reasons. First, I'm writing a musical and I tend to work on multiple songs from the show in a given mixing session. For example, last week I did some pick-up recording at my writing partner's, then edited the pick-ups into 3 separate songs, rendered to a mix and placed the mixes into the master. Total space consumed by the show right now is just under a terabyte, and I'm only half way through the first act. To make matters worse, I started this project on a Fostex A8LR. I have, on occasion, had to go back to the original tapes for something, so I keep the digitized audio from them on my project drive. It's just not practical to work with a smaller drive and move each song over to it from my LAN NASes (I have two 3 terabyte RAID 5 NASes that are mirrored), at least not on this project.

I'm also paranoid about data loss. I back-up my project disk nightly to the primary NAS and it is automatically mirrored to the secondary NAS. However, I have, on occasion, had a drive fail during a mixing session with catastrophic results, at least for that day. I'm far more comfortable with a redundant disk system that, in a worst case scenario, allows for hot swaps with no data loss in the event of a failure.

I'm not sure why someone would think RAID 5 is for business scenarios, only. The very features that make it desirable for business use -- redundancy and hot-swapping -- make it desirable for my uses.

2. Why I use 96K: I'm well aware that the difference between 44.1K, 48K and 96K can't really be heard. However, I believe (based on my actual experience of listening to the results on my system) that, when lots of correction, effects and other alterations are applied, rounding errors accumulate resulting in a degraded sound containing unwanted artifacts. I don't know enough engineering to know whether I'm right or not, but I have the storage space so I see no reason not use 96K. If my interface supported it, I'd use 192K. No one has ever suffered from having too much data (except with respect to bogging down the system), whereas having too little can, in some circumstances, be problematic.

3. Audition 3.0 and multiple cores: This is really strange, because I watch core usage when I'm mixing and it always looks like a single core is bearing the brunt of the processing -- other core usage appears, at least to me, to be confined to OS overhead and other housekeeping chores. This might be because my editing machine has an older quad core or, perhaps, there is a setting buried somewhere in Audition for specifying the number of cores to use. I'll look.

4. Temp files: I like the idea of setting the primary temp files to the SSD. That makes a lot of sense and I'll try it.

5. 32-bit: I already run that way.

Thanks for the suggestions.
 
PT, you are not a typical user or a typical situation. You are closing in on situations that I see corporate users facing. I made that statement to cover the other 99%'ers. I do see your point on using a RAID 1, RAID 5 is a newer standard of RAID 0 (from what I can gather) and it allows to still be able to read if one drive fails, but requires at least three drives. Still seems like you loose something, but I didn't investigate much more to be honest.

My understanding of the RAID 5 standard is your gain is on read, you take a hit on write, maybe not an issue for you. You are working with a lot of data, so your complete situation is going to be difficult and an easy workflow difficult. I can see your point on the mirroring (RAID 1), you might be adding more burden to the system using the RAID 5 without any real benefits. (As stated, and you may want to look further, the performance improvement comes in smaller data read/writes, audio are large data chunks).

If you kept the OS not mirrored, did a snapshot image of its configuration and stored it (I have to assume this is pretty stable), then used your working drives and mirrored those only, with some sort of backups (I am not sure that would be required), that would give yo a pretty decent system without over burdening your system.
 
PT, you are not a typical user or a typical situation. You are closing in on situations that I see corporate users facing. I made that statement to cover the other 99%'ers. I do see your point on using a RAID 1, RAID 5 is a newer standard of RAID 0 (from what I can gather) and it allows to still be able to read if one drive fails, but requires at least three drives. Still seems like you loose something, but I didn't investigate much more to be honest.
My system uses 4 drives, both internal for the editing computer and for the two LAN NASes.

In the pre-video game days, both computer manufacturers and Microsoft made the mistake of assuming that home systems should be simpler and less powerful than business systems. Exactly the opposite is true. The typical business user does word-processing, spreadsheets, reads PDFs and sends lots of email. That doesn't require a powerful machine, though, of course, it does mandate data security. The typical home user, on the other hand, is editing photos (which continue to increase in megapixels), watching videos and listening to music, editing video and music, and doing far more tasks that are CPU and GPU intensive.

Our data security needs are also significant. For example, I have decades of photographs stored on my LAN -- I shoot digital now, but I also have a high-quality negative scanner and, over the past 5 years ago, have made a significant dent in my film. I would be devastated if I lost those -- they're irreplaceable.

When home computing was still new, I had acquired one of the very early IBM PC-XTs, a major step up from the Commodore 64 that I had been using before it. The XT came with a 20 megabyte drive (that's not a typo :)). I thought that was all the data storage I'd ever need, and happily dumped everything onto it, without much concern for backup. I had one of the first "standardized" MIDI interfaces, a Roland MPU-401 (anyone remember those?) and, if I recall correctly, was working with Cakewalk, the Fostex A8LR (I had worked my way up from an early Tascam 4-track cassette-based Portastudio), and a box that synced the Fostex to the computer. Well, one night I was defragging the hard drive, a process which took 24 hours (!!!) and something went wrong -- the hard drive crashed, all the data was lost and the script and all the Cakewalk MIDI files for a musical (the same one I'm working on now) were lost. THAT was when I realized the importance of data redundancy, and immediately bought a tape backup for the hard drive. I've been using some form of heavy-duty backup ever since and, happily, haven't lost a byte of data. That musical set dormant for nearly 20 years, until I acquired an m-Audio Fast Track Ultra and was able to digitize all the tapes from the Fostex (which, amazingly, still worked after sitting on the shelf for so long) and, with modern DAWs, was able to manipulate the recorded audio and add new material with little difficulty. One of the interesting problems was that the reel-to-reel tapes, which were not stored properly, had stretched just enough to pull the audio noticeably off pitch, so every track had to be corrected. Also, the limitations of only 8 analog tracks, one of which was dedicated to the sync track, meant that I had to bounce a lot of tracks to get full orchestrations; I've spent considerable time playing with filters and the like to separate out (more or less) the individual instrument parts. And, of course, both the limitations of the Fostex and cramming 8 tracks onto 1/4" tape meant processing each and every track to remove hiss, as well as added noise from using the crappy mikes that I had at the time.

I do my composition on a music computer, also a quad core, but save the data over the LAN to the NASes. It does slow things a little bit, but it's not much of an issue with MIDI. I also render out to the NAS, rather than the local music computer when I'm ready to mix. Mixing is done on my belts-and-suspenders editing computer, which I also use for other tasks.

Mirroring the editing computer isn't feasible -- though I have a gigabit LAN, I do a lot of photo, video and music editing, and moving that much data to the NASes on a mirrored basis would just bog down the computer. I have a service computer (an HP thin client -- very useful little beasts, and I have a bunch of them) that handles back-up chores on a nightly basis when I'm (usually) not working on the editing computer. Just for safety's sake, it also sends a subset of the backup data (though, unfortunately, not media data) via VPN to another thin client with a 3 terabyte USB drive that's at my office. I also have another USB 3.0 3-terabyte drive that I use to mirror the NASes and, once a month or so, I'll take it to the office and sync the non-nightly back up files to the off-site server I have in my office.

This system, though probably overkill, has served me well and I've already had several instances in which, for one reason or another, critical data went missing -- restoring it from one of the many redundant back-up options I have was a piece of cake.

The RAID 5 system in my editing computer runs of a dedicated card, rather than the MB's SATA connections. Accordingly, there is no additional overhead on the CPU. I do it this way because, should my MB ever get fried, I can simply plug the card into a replacement computer with no data loss whatsoever.

The OS and programs are on the SSD and not backed up. It's easy enough to reinstall the OS if necessary, and I have to do it every year or so anyway as Windows becomes unstable over time. My only concern with the OS and programs is speed -- boot fast, load fast and let me get to work.

Though I'm probably not a typical user, I think my computing and data backup needs are pretty typical -- though most people don't have a system like mine, they probably should think about acquiring something with similar functionality. We all live digital lives now, and data security should be paramount.

My understanding of the RAID 5 standard is your gain is on read, you take a hit on write, maybe not an issue for you. You are working with a lot of data, so your complete situation is going to be difficult and an easy workflow difficult. I can see your point on the mirroring (RAID 1), you might be adding more burden to the system using the RAID 5 without any real benefits. (As stated, and you may want to look further, the performance improvement comes in smaller data read/writes, audio are large data chunks).

If you kept the OS not mirrored, did a snapshot image of its configuration and stored it (I have to assume this is pretty stable), then used your working drives and mirrored those only, with some sort of backups (I am not sure that would be required), that would give yo a pretty decent system without over burdening your system.[/QUOTE]
 
PT, yes, I remember the MPU-401, I still have my LAPC-1 and I used a program called Ballade. 20 MB HD that cost me $800. About 6 years ago I finally threw the computer away, it was hard throwing $3000, 1989 dollars away. I still have my old files that still have creation dates of 1989/90. That is why I am so familiar with with copying dead hard drives.

I do see a lot of redundancy in your system, but it if works for you, one can never really be too careful, it is just a matter of how much one wants to spend for insurance. When I am helping these companies build their systems, we always look at up time (disaster recover is a different topic), once one get's past %99.99 up time, the costs go up and the return less. I do the OS image as it is so much quicker and you get a system you had before the "insert bad crap here" happened. I just hate rebuilding the OS, its like moving, I just don't like setting up everything again. But, reinstalling is also like moving in that, you clean up a lot of bad history.

But to the business at hand, if your primary music computer is doing all of these things, then it is an overhead that will impact performance. You could, set up a small computer for you network computers, let that computer do all of the backs ups and routine house cleaning on a scheduled bases and let that run separately. But now that is getting into networking and servers. Linux has real good software that does that for very little to no money, but that really starts to become a pain as you are getting into network administration. Which might be the way you need to think about to secure you business. You may have reached a point where, paying for a service to so some of that might be worth it. Especially knowing they could back up to a place offsite and now you get a disaster recovery in the mix. But that is probably your next step to improve what you have.
 
PT, yes, I remember the MPU-401, I still have my LAPC-1 and I used a program called Ballade. 20 MB HD that cost me $800. About 6 years ago I finally threw the computer away, it was hard throwing $3000, 1989 dollars away. I still have my old files that still have creation dates of 1989/90. That is why I am so familiar with with copying dead hard drives.
Yep. Broke my heart to trash the XT, but my wife wouldn't let me start a museum. :)

I do see a lot of redundancy in your system, but it if works for you, one can never really be too careful, it is just a matter of how much one wants to spend for insurance. When I am helping these companies build their systems, we always look at up time (disaster recover is a different topic), once one get's past %99.99 up time, the costs go up and the return less. I do the OS image as it is so much quicker and you get a system you had before the "insert bad crap here" happened. I just hate rebuilding the OS, its like moving, I just don't like setting up everything again. But, reinstalling is also like moving in that, you clean up a lot of bad history.
I rely on Win 7's system restore. The problem isn't so much installing crap as the registry getting bloated and lots of unnecessary junk getting loaded.

But to the business at hand, if your primary music computer is doing all of these things, then it is an overhead that will impact performance. You could, set up a small computer for you network computers, let that computer do all of the backs ups and routine house cleaning on a scheduled bases and let that run separately. But now that is getting into networking and servers.
I didn't explain myself very well. My home has a gigabit LAN. My editing computer doesn't do anything but run application software as and when I need it. The two NASes are stand-alones that set on the network. I have a thin client on which I installed a full copy of XP -- it handles all the back-up and synchronization chores, as well as running an FTP server and updating dyndns with my non-static IP address. I've got another think client that sits and LAN and does nothing but be a phone server for a MagicJack line. My music computer (at which I compose) is also on the LAN, as is the media computer in the living room (along with a Roku, a SlingBox and my internet-enabled BluRay player) and a laptop in the bedroom for my wife. There's a Chromecast on the bedroom TV that sits on the LAN via wireless, as does two tablets and our two Android phones. I'll be adding an older laptop that will run FreePBX when I get some spare time to fool with it; right now there's also a Cisco SPA8800 for VOIP, as well as a dedicated VOIP phone, also on the LAN. I also have a CD printer, a photo printer and a color laser, also all on the LAN. Everything runs through a dd-wrt router.

Administration of all of this is very straightforward, given that all the backup and mirroring software runs under XP, as does the FTP server. The NASes take care of themselves, though I monitor them regularly to make sure all is well. dd-wrt is rock solid, though lately the router, a TP-Link, has been a little hinky -- I suspect the problem is heat-related. I've got a new one arriving Wednesday -- it cost all of $40.

On the office end, I have thin client that I also adapted to full XP. It acts as a file server for the remote 3 TB drive, and is also responsible for opening and maintaining the VPN connection to my home system. My firm's IT people are fine with me doing it that way.

Linux has real good software that does that for very little to no money, but that really starts to become a pain as you are getting into network administration.
Exactly. I fooled around a bit with Linux when I was a DirecTV subscriber and hacked my boxes so I could transfer DVR recordings to my computer system. I'm not real eager to learn another OS and, frankly, there's no need. The NASes run Linux, but I've had no reason to hack them -- they work just fine by themselves.

Which might be the way you need to think about to secure you business. You may have reached a point where, paying for a service to so some of that might be worth it.
I wouldn't do that for a few reasons. First and foremost is the cost, both for the hardware and the service. I also don't like anyone, and I mean anyone, having access to my systems and hardware. At work, I'm the only lawyer who uses all his own hardware (I have laptop, three monitors, three printers and a host of other gear, all my own personal property). My laptop isn't on the firm domain, though I can access the firm's servers and printers (we use Active Directory), and I don't allow the IT people to access my laptop for remote administration (which is moot, anyway, because there is nothing I'd let them administer). With respect to disaster recovery, there have been server crashes from time to time at my firm -- not surprisingly, I'm the only one who can keep working through them. :)

My home system is not a business system, nor is it suitable for a business. It is, and always has been, a home computer system and for that works very well.

Especially knowing they could back up to a place offsite and now you get a disaster recovery in the mix. But that is probably your next step to improve what you have.
I already have off-site backup, done nightly, for most of my data. The only thing preventing me from doing off-site backup for everything is the speed of my internet connection and the cost of backing up truly enormous blocks of data on a regular basis. However, even my huge files (audio and video) get backed up off-site monthly.

I would like my LAN to be faster, but solutions for greater than gigabit are too expensive -- I'd have to replace all of NICs, get a new switch (I have a 24-port unmanaged switch from Netgear that works just fine), and get a new router, all of which would be prohibitive. When I get around to, I'll buy 8 2-terabyte drives and upgrade my NASes to 6-terabytes each of RAID 5 -- these boxes won't support larger than 2-terabyte drives.

Aside from that, I'll be upgrading the MB on my editing computer at some point, probably with an i7 quad core, and 16 gig of RAM. Otherwise, I'm satisfied with my system.
 
Back
Top