As a professional photographer, hard drive space is a constant battle! Of course, we hear all the time how "cheap" HD space is, so what's the problem? Well, what most people "experts" don't acknowledge is the real cost of adding space to your working system. I'm talking about the hours of researching reliable drives, planning for future growth, installing and reconfiguring your current system and/or paying for someone to help to do all of that for you. Time is money and there's a lot more involved than cost per/GB when it comes to running things reliably and professionally. Second, you rarely hear talk of a serious back-up plan which should at a minimum consist of two complete copies of all your data (one being off-site). With file sizes as large as they are as well as the addition of HD video to most cameras this can start to add up quickly, even for "cheap" space!
If your system is getting full (say a single drive in an iMac for example), then you're stuck either upgrading that drive or adding external drives. If adding externals, buy drives with the fastest connection your computer will handle (eSata, iScsi, Firewire 800, Firewire 400, USB), but realize this solution is simply a band-aid. Externals are great for archiving or back-up to, and yes, I'd include the drobo in this group, but not so great when editing large photography or video files (5D Mark II or larger) (New drobo (iscsi) may be an exception, but I've heard mixed reviews on speeds). With the exception of eSata, Read/write and transfer speeds of externals becomes too difficult to work with. You need your drives working FAST, especially on data you access daily. At some point you'll need to upgrade to a box (Mac Pro). More on the back-up plan below.
If you already own a Mac Pro, still running out of space, there's another solution for you. The beauty of the Mac Pro is the ability to expand from 4 drives into 6 or even 8 by taking over the optical bays (Screenshot below). A few weeks ago, I found myself in this situation and opted to add two Samsung 2TB drives in the 2nd optical bay (below the DVD drive/burner) via a Pro Caddy 2 from Transintl for roughly $450. This is awesome! The future possibility for growth here is: 8 bays x 2TB drives = 16TB of onboard data. My current Mac Pro is set-up as follows:
-Early 2008 2.8Ghz 8 Core Mac Pro
-16GB RAM
-Hard Drive set-up:
Bay 1 - 320GB Factory Drive - Holds OSX and all Applications
[Drives 2-4 are set as a software RAID 0 (through the disk utility in your applications]
Bay 2 - 1TB Western Digital Black
Bay 3 - 1TB Western Digital Black
Bay 4 - 1TB Western Digital Black
-NEW: 4 Terabytes of space for $450
2nd Optical Bay- 2 x 2 Terabyte Samsung Drives via Pro Caddy 2 ---[currently set as a RAID 0]
[Note: There are cheaper 2TB drives on the market. They don't look as reliable as the Samsungs. Saving $40 on an HD that is of questionable reliability, which 2TB still are, even the Samsung, is about the most insane thing you could do as a photographer, in my opinion.]
In summary, the "Mounted" drives sitting on my desktop are as follows (remember drives 2-4 and the optical are set-up as RAID "0's" so they appear on your desktop as 1 drive):
Mounted Drive 1 - OS/Apps
Mounted Drive 2 - 3TB in RAID 0 - Current years work with 64GB partition for Photoshop scratch disk
Mounted Drive 3 - 4TB in RAID 0 - All data (Photos, Documents, Music, Motion/Video)
WARNING -- ***DO NOT SET-UP ANYTHING IN RAID 0, WITHOUT A REALLY, REALLY SOLID BACK-UP AND RESTORE PLAN (In my current set-up, If my "bay 2 drive" crashes, then I lose all photography (data) across 3 drives (bays 2-4). So in my mind, I have to ask myself - "When a drive crashes, can I get my photographs back?" AND "How long will it take to restore the data?"
BACK-UP PLAN
Probably the most important part of your data management plan as a photographer is to have a consistent back-up plan in place. When I say consistent, I really mean automated. If it's not automated, something will slip through the cracks.
Probably the most important part of your data management plan as a photographer is to have a consistent back-up plan in place. When I say consistent, I really mean automated. If it's not automated, something will slip through the cracks.
Mine's not foolproof, but it's solid and a good place to start...
Part A) Drive #1 (OS/APPS) backed up via Time Machine to a USB drive
Part B) Drives #2-6 backed up to Server via Gigabit ethernet. My main block of live data (i.e. Photographs, video, docs, music) spans 5 hard drives within my Mac Pro, equalling 8 Terabytes of space. My server consists of a 10 Drive, RAID 6 box which will handle all 8TB's of data. RAID 6 just means that two drives can completely fail and the data will still be in tact within the server. A drive will fail on you at some point. Plan on it. If you can't afford or don't need a server, get a Drobo or even a simple external Hard drive from Costco that has enough space to completely mirror your entire data set). Another option if your a Mac Pro user, is to simply use the remaining Bay's for back-up (remember you have up to 6 bays with the Pro Caddy 2). This is the cheapest and cleanest method if the space is available.
STEP 2: Back-up all of your data offsite.
You need to back-up your data to a place that will still be around if current location experiences a natural disaster such as a flood or fire. The external HD's get backed-up and stored securely off-site at least monthly and more often if possible.
You need to back-up your data to a place that will still be around if current location experiences a natural disaster such as a flood or fire. The external HD's get backed-up and stored securely off-site at least monthly and more often if possible.
STEP3: Back-up offsite again - Online.
I'm currently evaluating CrashPlan and Backblaze for online back-up (both around $50/year). I'll let you know soon which I prefer. So far I'm leaning toward CrashPlan for 2 reasons 1) You can send a 1TB upload initially via a Hard Drive and 2) If and when you have to restore the data Crashplan is a few hundred dollars, Backblaze is several thousand. Both programs run continuously in the background and update only the files that have changed/added on your computer.
I'm currently evaluating CrashPlan and Backblaze for online back-up (both around $50/year). I'll let you know soon which I prefer. So far I'm leaning toward CrashPlan for 2 reasons 1) You can send a 1TB upload initially via a Hard Drive and 2) If and when you have to restore the data Crashplan is a few hundred dollars, Backblaze is several thousand. Both programs run continuously in the background and update only the files that have changed/added on your computer.
STEP 4: AUTOMATE everything.
I use Chronosync ($30) to automate my main drive back-up (Drives 2-5) to my Server. I also use chronosync for my monthly external, offsite back-ups. The beauty of this program, is that you don't have to re-copy your entire library of photographs. For example, let's say I go into Lightroom and tweak my Saturation on a RAW file shot way back in 2005. When I sync my hard drives with Chronosync, it will notice the change to that file (XMP) and add the saturation change to the back-up.
I use Chronosync ($30) to automate my main drive back-up (Drives 2-5) to my Server. I also use chronosync for my monthly external, offsite back-ups. The beauty of this program, is that you don't have to re-copy your entire library of photographs. For example, let's say I go into Lightroom and tweak my Saturation on a RAW file shot way back in 2005. When I sync my hard drives with Chronosync, it will notice the change to that file (XMP) and add the saturation change to the back-up.
STEP 5: BACK-UP one more time.
IF you're extremely paranoid (like me), hook-up an additional external drive (or box) via eSata, FW800, or Ethernet directly to your Mac Pro (iMac or PC) to acts as another back-up source to either all of your data or your most important files/folders. A drobo would be an example of such a box.
IF you're extremely paranoid (like me), hook-up an additional external drive (or box) via eSata, FW800, or Ethernet directly to your Mac Pro (iMac or PC) to acts as another back-up source to either all of your data or your most important files/folders. A drobo would be an example of such a box.
*IF you're on a budget, take STEPS 1 and 3 at a MINIMUM. This is the least expensive, yet viable option.
I realize this could either be completely overwhelming to you OR too basic, depending on which camp you fall into on the techy side of things. Don't stress too much if it sounds overwhelming, just enough to decide to put a back-up plan in place! If you have questions about your particular system or where to start, please put them in the Comments section and let's generate a good discussion.
Also, please chime in with your experience and plan for both adding more space for data and backing it up. I would love to hear your thoughts!
-Kevin
--
5 comments:
Great advice here Kevin; thanks for sharing.
Having been down the road of 'adding more and more' external hard drives I've now opted for the Drobo solution. Reason? Reliability, speed and the storage space for the eSATA drives once full.
Backing up is certainly something that each and every one of us has to do and to develop a 'system' I'm sure you would agree is essential.
Once again, thanks for a great post, especially the advice on adding more drives internally. What with your advice and that of Matt Kloskowski (http://bit.ly/cNnJza) recently, the message is definitely getting out.
Cheers, Glyn
Yes. Great advice. Thanks for getting this information out there.
I am at the crux of needing a better 'solution'. The juggling of externals and working on laptop just isn't going to cut it this season.
I just bought an iMac with 2TB HD. I'll likely go with an external RAID onsite with backups offsite.
The CrashPlan looks like it might fit my needs nicely.
Thanks,
samperry.com
Thanks Glyn and Sam.
A quick update: CrashPlan has received roughly 300GB after about 8 days of working. It's estimating 60 days to completely back-up my data. This is a bit disappointing so far as I have a fast connection, but I'll keep it going for a bit longer. Any suggestions out there for Online back-up solutions when working with large blocks of data?
Superb discussion. I'm only a fledgling in respect to these matters, but want to get a dependable system in place before libraries swell. I'm on a MBP with a 2TB Time Capsule humming in the background. As recommended, all backups are automated. Gives me piece of mind to have at least an initial back-up barrier, but I'd feel even better if the TC could push daily back ups to an off-site server... the server service having multiple redundancy barriers, but again, it's still early days for me.
What do you do for back ups while on the road? I presume you work with an adequate HD on a laptop, but do you simply get an equivalent size external HD and hope both don't get tossed overboard/get stolen or have you sorted out something a bit more robust?
Cheers, - Jason
Hi Jason,
This past week I was on the road with four different photo shoots, so that's a timely question.
After each days work, I'll pop in my CF cards and use the LIghtroom importer to copy the files. I first copy the files to an external HD (Lacie 500GB Rugged) and edit and work from that hard drive. At this point I have two copies (CF cards and Lacie HD). Next, I'll review all the images, do a quick ranking, delete those I will never use). I then make another copy to my Macbook Pro HD. I now have two permanent copies (Lacie HD and Macbook HD). If it's a very important job, I'll make a third copy of the data to another external HD (Western Digital, 500GB Passport). Lastly, remember to diversify as much as possible. So, for example, the next day I go out to shoot, if I leave my Macbook Pro at the hotel, then at least one of the HD's is coming with me for the day to keep an "offsite" copy. If the hotel room gets broken into, I'm covered with my back-up. The other option is to FedEx a drive back home with your data. Redundancy is just as important when working on location. Lastly, I never delete the photos on my CF cards until I have at least two copies of all the data.
-Kevin
Post a Comment