Redundant Remote DAWs: Lessons Learned
If you like high stress, you'd have LOVED my Saturday. :)
Up till 2 AM the morning of, getting my ducks in a row for the Saturday night show: a top-shelf pianist and his jazz trio at a new venue outside of town. I'm no jazz buff, but apparently this guy's been around. I really didn't want to have to tell him I had a system crash during his show, which he was also capturing with a three-camera shoot to produce a demo montage. He also said he wanted to hang onto the tracks for a future CD if the show was particularly good. And if he liked what he heard from the demo mixes, maybe I could even mix his full-length CD. Yowza.
Failure was not an option.
As it turned out, failure would've been less likely if I'd not bothered with the backup DAW. Ouch.
Lessons learned:
1 I have way too much gear for a guy approaching 50 to be loading in, setting up, tearing down, and loading out.
2 With any new configuration, make sure it works, problem-free, before hitting REC. This is something I think I learned in second grade. Oh, well. The school of hard knocks has made it stick, finally.
3 Prayer works.
4 But it's still probably a good idea to check out a new configuration before hitting REC. Dummy. :rolleyes:
Here's a breakdown of the saga:
Showed up at the venue at 12:30 PM. Started unloading my RAV4.
1 PM: started moving stuff around -- stage stuff to the stage (mics/stands/cables), recording gear to the FOH position at the rear of the room.
1:30 PM: started putting the record gear together. This included a rented passive mic splitter. Many thanks to the FOH guy for helping me out by dealing with the repatching of his console inputs to the splitter, etc.
2:30 PM: Trouble. Clocking isn't working. My clocking configuration was ADA8000 #3 as the master (not being used for audio at this gig). I slaved the other two ADAs off of that, then the two Digifaces (one per computer) each slaved off of one of those (audio from ADAs was split to the Digifaces using a toslink splitter from each).
It didn't work. No lock lights on the RME's. What I discovered is that an ADA8000, when in slave mode, will not output usable clock in it's ADAT streams. Which I would've known if I'd followed number 4, above. So, this, then, would be Lesson 5. :rolleyes: D'oh! I hate real life.
3:30 PM: Add to that the fact that the new Belkin toslink cables I bought for the backup DAW don't fit properly; they're loose. They don't snap in, like every other toslink connector I've ever used. And I've been using them since 2nd grade. All I can do is stick 'em in and hope they stay. And tell the FOH guy, who's on the back side of my rack, not to play with all the cables during the show, if he can help it.
4:00 PM: Need a new approach. Here's the new idea -- Digiface 1 is master, Digiface 2 and both ADAs I'm using for the show are slaves. I do lose the total system isolation offered (in theory) by the first approach -- the master system crashing could now impact the clock stream; but what choice do I have? Seems to work. All the lock lights are on. So long as the toslink cables stay in. :o
4:30 PM: Geeze. The stage still isn't mic'd. Eat some quick lunch and seize the stage. Big help from the FOH guy, again. Throughout running cables and setting up mics, the guys are running through tunes. Man, I hope I can get this done in time to go tweak levels on my rig!
5:30 PM: The guys are gracious enough to give me a few minutes of their loudest stuff so I can set optimum levels on all channels.
6:00 PM: The piano tuner shows up and starts tuning.
6:30 PM: Folks are shuffling in.
6:50 PM: piano tuner finishes. Collected crowd applauds. He bows.
7:01 PM: I hit REC on both systems (SAWStudio on my late-model laptop, SAW Pro on my turn-of-the century AMD box running 256 MB of RAM). Recording 12 channels each. And the show begins. Everything's wonderful. I pat myself on the back for my ingenious rig and my extraordinary performance-under-pressure problem-solving abilities.
9:08 PM: Show ends. I do some playbacks in the cans. You know, just to hear some instant-gratification results of my genius.
9:09 PM: Nightmare of nightmares. I'm hearing fuzzy cracklies coming from my SAW Pro rig. AND... I'm hearing snips and snats coming from my SAWStudio rig. My life flashes before my eyes.
9:10 PM: Hoping against hope, I start checking the details of each track on each system. It looks like some of the tracks are good. Is it just possible that between both machines -- I can gather a single set of good tracks? This is NOT what I had in mind with the idea of a redundant backup DAW! :eek:
9:13 PM: Looks like I actually might be covered for Piano L/R tracks as well as stand-up bass. Great! Now, to just check the drum tracks, and then check all of 'em at several points along the time line, for added confidence -- OMIGOD, it's the artist, approaching me with that "Can I hear a little playback?" look on his face! I hate that look. Especially right now. Maybe if I throw up, he'll be distracted, and forget the whole silly playback idea.
"What'd ya think?"
"Great show!"
"Aw, shucks. Thanks. Hey, can I hear a little play back?"
"Of course you can. Here ya go..."
I hand him the cans plugged into the SAWStudio DAW. I think that "take" was more subtly "impaired" than the other. Let's hope his ears are still ringing from the performance. Maybe he won't notice because he doesn't have my well-trained ears. Maybe he'll --
"Sounds good. We'll talk in the next couple days, once the video is cut." He hands me the cans.
"Great. Looking forward to it."
9:14 PM: break out in (somewhat delayed) cold sweat. Begin (somewhat delayed) silent prayer. (Trauma's funny that way.)
9:15 PM: No time to do this now. Gotta get outta here. Tear down and load-out begins.
10:00 PM: pack up stage stuff.
11:00 PM: Everyone's gone but me. I'm literally told by the second-to-last guy out to just kick the rock out and let the door close and lock behind me when I make my last trip out to the car. Start the 40-minute trip home. Continue praying.
Till 3 PM the next day: due to sleep requirements and social obligations, continually experience the unsettled state of mind of not knowing whether or not I have a usable recording and what it might be like to have to explain the bad news to the artist, who's not only hired me, but also a three-camera video crew to shoot it. Live in this not-knowing place for 16 hours straight.
Sunday, 3 PM: Unload the RAV4 back at the studio. Set up the computers. Offer one more prayer to whatever entities will listen. And my list was long (redundancy again). Start a rigorous examination of tracks.
Within a few minutes, the truth reveals itself. Somehow (divine intervention? ya think?) for every track on one computer marred by a clocking issue, there was, indeed, a good track to be found on the other. And there was no rhyme nor reason to them, either. There were the varied manifestations of the drift as described above (consistent within each track, but not from track to track), and then there were the odd combinations of tracks: it seemed that the problem tracks couldn't be associated with a certain lightpipe channel. The same lightpipe channel of 8 tracks had some good and some bad tracks on the same computer. And not on the other computer (at least not the same combination of good/bad tracks). Weird.
So, I dodged a bullet. And lost, perhaps, ten to fifteen years of my life expectancy. Worth it? You decide.
What the hell was I doing building a backup DAW at 2 in the morning the day of the show, not leaving time to even consider testing my brilliant clocking/configuration idea?
All I can say is that I thought I knew how all this clocking stuff worked, and I was wrong. That ART SyncGen is starting to look really nice.
I remember probably 12 or 13 years ago talking to Rail on the phone and asking him about this mysterious clocking stuff. I think I was toying with using two CardD Pluses together at the time. He was telling me about word clock and how you really needed a common reference between different devices, etc. He talked about different ways to patch cards' clocks into each other, and ended by saying something like, "Yeah, eventually you end up just getting a house clock to drive everything. It's really the only ultimate solution to all this stuff."
I'm such a slow learner. :o
Re: Redundant Remote DAWs: Lessons Learned
I think I going to check my cables now! :eek:
Hope you get the CD gig.
Oh and yes Dave prayer does work ;)
Chris
Re: Redundant Remote DAWs: Lessons Learned
Quote:
Originally Posted by
Dave Labrecque
If you like high stress, you'd have LOVED my Saturday. :)
I love your way of telling the story, though I don't envy you for the experience(s) ;-)
Sometimes we have to learn the hard way. I did so several times during my days at radio. I'm with you on the prayers, cold sweat and your believe in miracles :-)
Literally, all the best,
Veit
Re: Redundant Remote DAWs: Lessons Learned
I did try to share with you a couple of salient points. ;) But there's nothing to beat personal experience. :eek:
Thanks for giving us the low-down. Maybe we can learn from your experience.
Well done for getting through it. Glad that you managed to get a good take overall. At least the praying worked but I don't think I can add a prayer section to any of my IT disaster-recovery plans.
Get some rest!
Dominic
Re: Redundant Remote DAWs: Lessons Learned
Holy guacamole Dave!! Time for a vacation, me thinks. I know that stress/down to the wire feeling. I had several occasions where I was not only the opening act, but doing the sound for the main act. I get pretty worked up on a good day when I have to play live.
And, I lost half of a show once because the fellow who I subcontracted the job to brought my ADAT into a hall when it was feezing outside. With condensation on the heads, the first tape was complete digital hash. And then there was the time when I was going to play guitar to some pre-recorded drums, bass, and guitar. I hit play on my Tascam DA-30 DAT and I heard munch munch munch. Fortunately, I had a good 'old cassette tape on hand as a backup and the show went on.
Re: Redundant Remote DAWs: Lessons Learned
Quote:
Originally Posted by
Dave Labrecque
1 I have way too much gear for a guy approaching 50 to be loading in, setting up, tearing down, and loading out.
in a RAV4 ????
[ROTFLOL]
Re: Redundant Remote DAWs: Lessons Learned
4:30 PM: Geeze. The stage still isn't mic'd. Eat some quick lunch and seize the stage. Big help from the FOH guy, again. Throughout running cables and setting up mics, the guys are running through tunes. Man, I hope I can get this done in time to go tweak levels on my rig!
Boy! Does that EVER work out?
I cringed at about every step in your story.... knowing what was coming even before the next sentence.
I'm so glad you salvaged the overall recording though!:)
Losing the piano tracks from a paino trio sucks.:eek: (ask me, I know.:rolleyes: )
Good luck with the mix! That'll make it all right again.
DanT
Re: Redundant Remote DAWs: Lessons Learned
Just curious, maybe I missed something....
since you had a mic splitter, whay were you trying to clock the two machines together?
I don't see anything pariticularly wrong with trying to do that, I'm just curious, since it increases the chances of a failure taking out both machines, wherein two machines running independently really IS redundant, as far as the recording portion goes.
Bill
Re: Redundant Remote DAWs: Lessons Learned
This reminds me of one of DAW's biggest drawbacks-no confidence monitoring.
You truly don't know what you have until it is too late to do much about it.
Just to make you feel better, Dave, here's my latest embarrassing story. Last week I committed a classic blunder. On an ISDN recording session with remote talent I managed to put the wrong channel into record and didn't catch it until 40 minutes into the session. I had good meter levels, it just happened to be an open mic in the control room reflecting the monitors to perfection instead of the codec output.
My butt was saved by a DAT backup at the other studio. But, the first try from the DAT had errors (they used a different machine to digitize). The playback from the original machine worked fine. I spent much of the day a special hue of crimson. Thankfully, the client was cool about it. But, I've got some demerits to work off.
Regards,
MM
Re: Redundant Remote DAWs: Lessons Learned
Dave,
I did enjoy your story telling. Especially after realizing you did have good results in the end. The story has to have a good ending, right?