My Photo Workflow

More info about this image

We take a lot of photos. Most of you who’ve been coming here for awhile know this. The problem with taking a lot of photos is where to put them. We each have a main computer, and each has a laptop (photos live on these when we travel) and both of us have slightly different workflows. Our photos in the past have lived on whatever was the main machine in the house, but last year, I got my tower and the problem of managing photos got a lot worse.

For years I’ve wanted to consolidate and then have a multi-tiered, redundant backup system so we didn’t lose anything. We had been using Heather’s machine as the base machine for all the photos by using attached drives:

090202-fig1-sm.png

Each of the external drives was partitioned into two separate volumes. So on drive 1 I had volume 1 and volume 2. Volume 1 would automatically backup/clone the internal hard drive in the main computer, with volume 2 being used for new photos. As that filled up, I’d add external drive 2 and partition it into volume 3 and volume 4. Volume 3 would mirror/back up volume 2. Volume 4 would be used for new photos. This was going on when I heard about the Drobo. The Drobo eliminated the need for multiple external hard drives. I’ll get to the reasoning shortly. The setup for both of us now looks like this:

090202-fig2-sm.png

We each have a machine with a Drobo attached and we move our photos off the laptops onto the Drobo. When we get home. I also purchased a portable drive that backs up the photos we take when traveling, just in case either laptop decides to die.

In short, the Drobo is a hard drive enclosure that handles up to four drives. The drives can be any size, any combination of drive. This is not like a normal multi-drive enclosure. The Drobo has some smarts about it. The best part is that you can add drives as you need more storage. I’ve formatted my Drobo to hold up to 16 terabytes of data. Right now, I’ve just got 4 1 terabyte drives, but over time, I’ll be able to expand the Drobo up to 16 terabytes as higher capacity drives become available.

I’m probably being overly cautious at this point, but I’m thinking we’ll buy a couple of more Drobos to back up the original Drobos periodically and stash the backup Drobos off-site, brining them in once a month for a backup. I’m still contemplating this move as it will mean some expense. Still, it would be nice to know that in a catastrophe, I at least have a second shot at recovering data.

Here’s my workflow for handling digital photo files

Start by shooting in RAW mode to get the most flexible file format (in terms of editing). If you have a newer camera and an older version of Photoshop, Adobe makes a DNG converter app that will convert your RAW files to DNG files. It’s a pain, but doable.

After the photos have been shot, we then move the photos off the card and onto a hard drive. That’s a laptop when traveling or one of our desktop machines. I do not immediately import the photos into an app at this point. The photos are just moved to a hard drive. Here’s my naming convention for the folder(s).

Year (folder) > month – year > – descriptive text > files from the card

On a Mac, it looks like this:

090202-fig3-sm1.gif

(I’m using column view in the Finder to display the folders like this)

You’ll note that I’ve got my catalog from Lightroom in year folder. I do that because I’m anal. Lightroom lets you use as many catalogs as you want. I use one catalog per year as a holder catalog. Once I’ve created that catalog in Lightroom and Lightroom relaunches, I then use “Import from Disk…” and select my folders to bring in to Lightroom.

If you have any questions, ask in the comments.

  • http://lifeasitcomes.com/ Kevin T Driver

    I know this isn’t what this post is about but …

    How about in terms of post-processing? For instance, all the etsy photos listed above seem to have had a somewhat similar treatment.

    • http://blurbomat.com/wordpress blurb

      I’ll do another post where I talk about using Lightroom for more than just simple tweaks. Patience! I may do a screencast… Not sure if that’s the way to go or not.

      • http://lifeasitcomes.com/ Kevin T Driver

        Didn’t mean to sound impatient…

        • http://blurbomat.com/wordpress blurb

          All good. :-)

  • meowsk

    Thank you for this post! I was always curious how you managed to keep all of those photos organized and am currently trying to figure out the best way to organize all of my photos. While I don’t think I will have the need for a Drobo I am thinking that my space won’t last long and I should definitely invest in another bigger hard drive. A couple of questions…

    When you edit a photo either to be printed or posted on your site do you save an original version and the edited version in the same folder that you uploaded the original files to?

    • meowsk

      Grrrr…. my finger accidentally slipped on the enter key before I was done typing…

      Or do the edited photos get renamed and saved in a folder specific to what they were edited for?

      Do you keep all of the photos from a particular photo session or go through and delete the outtakes once they are on a drive to save on space?

      Also, kind of off topic but you guys obviously seem to always be taking photos. I am trying to get in the habit of using my camera more to inspire my creativity. But often times my camera is either packed away or not with me when I wish I had it or it is too dark to go photo-ing when I get home from work. Any suggestions on making photography more part of everyday life? The obvious answer is to take my camera with me where ever I go and not be afraid to take it out. But do you have any tips to get me going on this journey?

      • http://blurbomat.com/wordpress blurb

        So far, I keep all the shots, but I’m learning to be more selective about what I shoot.

        Carrying around a camera is a good start. However, making time for taking shots is what I do. I’ll go on a walk (especially when traveling) and make a point of the walk being a photowalk, where I force myself to shoot. Some I share, some I don’t. But the practice is the best.

        One of the myths is that you need a $5,000 camera to take great shots. You can use a phonecam to get magic shots, to be sure. If you have a point and shoot, carry that with you all the time. Learn how to work it quickly. If you have a DSLR that’s a bit more clunky… definitely harder to do. When I worked in an office, taking shots was a lot harder. Mostly because Heather had the camera at home :-). I learned to watch the light and take the best shots I could with my janky cellphone. I’m still happy about a lot of those 2004/2005 shots.

        Hope this helps!

    • http://blurbomat.com/wordpress blurb

      I’ll export the file from Lightroom as a 16-bit tiff file and stash that in a “photos for sale” folder.

      If I’m just posting, I’ll leave the master file wherever it is and then export a JPG out from Lightroom (in the past, that’s been Aperture).

  • shelley

    How is your laptop talking to your Drobo? Is it a shared volume off your desktop or do you have drobo share?

    I have a drobo plugged into my mini. I have a volume on the drobo for my Time Machine backups. This volume is shared and I can connect and write/read files from it. The mini backs up fine to this volume, but I am unable to get my macbook pro to backup to this volume via Time Machine.

    So I was just curious what your setup was and how you do your backups of your laptops.

    • http://blurbomat.com/wordpress blurb

      Shared volume off the desktop.

      The laptops can copy files over wifi. Or we’ll take the portable drive and move everything there and connect that to one of the desktop machines.

  • http://faydean.typepad.com faydean

    Ok, wow…

    Well, first off do you use and auto-backup program. From my long ago days as a graphics production coordinator on Macs, I don’t even recall what we used, but we had to do daily backups since it was huge graphic files for publication. We have alot of media on our computers…one desktop, one laptop. So do you know what a comparable back up software would be for a PC?

    This laptop I’m using is a few years old and it’s harddrive is literally full due to photos. Dave bought me an iBook drive to move them to, but I’m totally paranoid about external drives dying so I haven’t moved hardly any files yet. I have been storing new photos on there (that I need to burn to disc like now).

    So that leads me to my next question, do you back up to DVD/CDR too? I mean, if you do, when you do it…you guys take alot of photos like I do, and huge files (I don’t shoot in RAW alot), so you’d have to do a burn session quite regularly. How long will you go before burning some discs? And when you do, do you do more than one copy? And do you keep any copies of the discs off site or in some kind of protective box, ie fireproof box? A friend of mine said that kind of thing was pointless for disc media because intense heat of a housefire would still damage the discs in the box.

    I guess bottom line, you have how many layers of backup again? Let me see…

    a hardrive, and the Drobo…is that it? Just the two?

    Oh, and lastly…Drobo just for Macs or PC compatible too?

    Thanks Jon! ALOT. I’ll show this to my hubby. He’ll be extremely interested in this Drobo!!

    • http://blurbomat.com/wordpress blurb

      I use SuperDuper! to clone volumes.

      Right now, I have my main hard drive that is backed up via Time Machine (comes with Mac OS 10.5).

      The beauty of the Drobo is that there are four drives that act like one. If one dies, you pull it and can replace it with another. The theory is that four drives are more reliable than one. In my life, I’ve only had two catastrophic crashes and the backup regimen saved most of the files. In one of the crashes, in 2005, the drive didn’t die per se, but I was able to back up a bunch of stuff onto DVD. The issue for me is that DVD and CDs are not the most reliable medium and they take awhile to write data. They are too small and too slow. But if you have data that is priceless, that should be backed up at least twice.

      Drobo is Mac & PC compatible. The USB is slower than the FireWire800, but your mileage may vary.

      Final note: the Drobo is an expensive option, but for us, the photos are worth it.

      • http://faydean.typepad.com faydean

        So you do set for an automatic backup daily, weekly, what?

        What do you backup? Just photos or music files too..I’m sure your blog site stuff. Btw, can I ask you a related question? For a blogger who isn’t their own designer (ie I use Typepad) what is the best way to backup/catalogue a blog page? I’ve had friends burn copies of screen shots, others actually print them out and catalog posts. Since I blog most as a family journal, I want it for the girls to have years from now…preferably digitally, but I’m not sure how to back that up. Or would you even bother. I mean Typepad should keep my blog for me, theoretically, forever, but I am a worrier at my core. Any suggestions? I’ve got a four year old blog to try and backup!

        Lastly, so if you backup a set of photos on DVD…do you have a brand you prefer? And then do you store those DVDs any special way, ie in a special photo dvd case or anything?

        Thanks SO much for all this help. Dave already looked at the Drobo…and the concerns over mass failure of the drives etc. He seems to like the theory behind the way it’s set up, he was just saying it was a little unclear on the actual amount of storage you got considering the drives are connected to mirror each other. But I think that’s more than enough storage, especially if you have a couple of iBooks in addition.

        • http://blurbomat.com/wordpress blurb

          Time Machine handles the backups on my main hard drive. Time Machine does an hourly, daily, weekly and monthly backup. It does this automatically. I set it and walk away. Time Machine backs up the entire drive.

          The best way to backup a blog would be to get FTP/SFTP access and copy all the files to your hard drive. I’d do this every week or so.

          Like I said, I don’t really back up to DVD, so I can’t help you there. I wouldn’t go cheap. You can’t replace photos!

          Re: Drobo storage. I have 4 drives all of them 1 Terabyte. My total storage is 2.67 Terabytes. But that’s ok, because if a drive fails, I’m ok. I can just replace that drive.

          When the backup system is complete, I’ll likely have three Drobos. One main one, one doing nightly clones (I’ll run SuperDuper every night) and an off-site one that I’ll SuperDuper and then rotate it in, giving the other SuperDuper’d Drobo a rest for awhile. Still working on the final solution. Hope this helps.

          • http://faydean.typepad.com faydean

            Jon…

            sorry dude, but I am not a web/tech person. I have NO clue what you’re talking about on the FTP/SFTP stuff, LOL.

            I mean…well…explain for a laywoman, pretty please sir. LOL

            And by copying all the files, do you mean that it would reproduce the actual blog page or just what I put onto it. Call me dorky, but I want to catalog the page just as it was on the web. Honestly I have no idea how to build a page or anything. I just log in to Typepad and voila, a blog page.

            If it’s too complicated to explain to someone with this much lack of knowledge, maybe a direction on where I could read up on such things perhaps.

  • http://surfmonkey89.myopenid.com/ Paul Lukinich

    Interesting that you moved from Aperture to Lightroom. Any insight into how you made that decision?

    I’m debating between the two as well. I LOVE Aperture’s cataloging & browsing, but that’s somewhat mitigated by the fact that I’m so damned anal about it anyway. On the other hand, while Lightroom isn’t quite as good in that area, it’s very “Photoshopy”, which makes it a little easier for me to learn.

    Add in the fact that Lightroom is cross platform and who knows what Apple will do with Aperture, and I’m 85% decided on Lightroom.

    Any suggestions you have would be greatly appreciated.

    paul in kirkland

    • http://blurbomat.com/wordpress blurb

      I still plan on using Aperture. It’s printing kicks ass over Lightroom if you want to make prints on a local printer. By local I mean one attached to your network or machine.

      Lightroom is faster and does a great job handling multiple catalogs. Aperture is more finicky.

      I love what they’ve done with the Adjustment Brush in Lightroom.

      • http://surfmonkey89.myopenid.com/ Paul Lukinich

        So you’re going to use both?

        Damn, you’re a glutton for punishment :)

  • nbrett

    I’ve also been considering a “off-site” storage for backups.

    What are your thoughts on “cloud” storage, specifically, the Google G-Drive I’m starting to hear about.

    Does this concept seem, like it does to me, a bad idea for things as precious as photos?

    • http://blurbomat.com/wordpress blurb

      Read my response comment below. It’s the one immediately below Geof F. Morris’ comment.

  • http://gfmorris.net/ Geof F. Morris

    Jon: Have you considered a Drobo-stored device with CrashPlan? This is where I’m headed right now, I think: the initial dump done over the local network, then carting the machine over to a friend’s house that I trust. I’m a skosh concerned given the problems that Fraser Speirs has had with his Drobo, but a replacement unit seems to have improved things.

    I love my Drobos.

    • http://blurbomat.com/wordpress blurb

      Geof,

      Thanks for linking to the Speirs post. I’ve left a comment there.

      I’m extremely leery of online backups. How long does it take to upload a terabyte or more of data? What happens if that company goes out of business? How secure are they really?

      The safest route would be to rent a safe deposit box or storage unit where one could stash the off-site drive(s). I’ve seen people with pelican cases and a Drobo with four backup drives in the Pelican. That seems a tad extreme until you think about things like floods & fires.

      In my case, I purchased a Drobo (the FireWire one) and it was wonky. I returned it (bought through Amazon) and the second unit has worked like a charm. I think they had a bad run of boxes.

      Second Drobo is performing like a champ. I see 25-40MB/second transfer rates. Much higher than I expected. Can’t wait for the 200MB/sec SSDs coming soon.

      • http://gfmorris.net/ Geof F. Morris

        Yeah, I’m nervous about online backups, too. That’s why I like CrashPlan’s personal bit—you can back up to other computers on your local network, or with friends, or whatever. If you want to test it out with a few files, I’m game and will provide you space on the Drobo to try. It is, of course, bandwidth-intensive, which is why I’m looking at doing the first major backup locally, then transporting the computer after that.

  • http://www.ransom-note-typography.com/ jon deal

    Good for you for having a backup plan. I’ve always said: “There are two kinds of computer users: people who have lost data and people who will. You get you decide which one you want to be.” :-] (I’m also incredibly paranoid about losing data.)

    You didn’t mention it, so I’m going to toss this out there. There is a difference between “back-ups” and “archiving.” Obviously a backup plan like the one you’ve got going is critical, but at some point you might want to look into long term, “permanent” storage. i.e., get the data off the Drobos and permanently off-site. (Or at the very least into a fire-safe, locking file cabinet or safe).

    As far as off-site storage goes, you mentioned the critical “flaw” in all “mozy.com/backblaze.com/homebrew Amazon S3″ set ups: Getting a metric TON of data up to the cloud. Even with broadband, you are looking at days or even WEEKS to get the data uploaded.

    You and I have talked a bit about this as I recall, but I still think you might be well served (sadly, pun probably intended) by an easy-peasy installation of Leopard Server on a machine in your house. Leopard Server has some pretty cool Time Machine stuff built in and you could centralize all your data on it. Depending on the speed of your internal network, it would be plenty fast enough. (Hard wire your towers to the server (GigE) and you run 802.11N, right?) Plus, it plays nicely with the Drobo. (or a chain of Drobos).

    Anyway, at least you can sleep soundly knowing that your critical “stuff” is backed up! Good for you!

    • http://faydean.typepad.com faydean

      Can I ask you about the fire-safe cabinet deal…

      I’d mentioned that earlier in a comment. I’ve been told by people that fire safe isn’t necessarily safe for digital media/hard drives/memory cards etc because of the heat the cabinet would still absorb.

      Are there special media fireproof cabinets?

      Thanks in advance for any info.

      • http://www.ransom-note-typography.com/ jon deal

        I can’t remember the details of the one we bought for our studio, but I do remember that it had some stuff about being tested for magnetic media. We archive to tape (blech), so I was concerned about things melting in there. As I recall, I got the one with the highest heat rating that still fit in the budget.

        It’s a heavy mother, too. It’s reenforced internally as well, I think. I guess that’s so when the giant beams of our basement fall on it, it won’t split open. :-]

        (Not terribly helpful, sorry!)

  • http://shmeagan.wordpress.com/ shmeagan.wordpress.com/

    I just wanted to leave a note to say thank you for writing this up. I’m still working on developing my own workflow, and storage is a huge component of that for me, mostly because I’m terrified of losing files. While the drobo’s a bit too rich for my blood at the moment, I will definitely keep it in mind for the future!

    Thanks again!

  • http://blog.ahles.nl/ Patrick

    Thanks Jon, for talking about this. It has reminded me that I too need to get the job done. A Drobo is not in the picture right now (I still have two external drives which will do for the moment) but I do need to make a plan as the number of pictures I take is growing and growing. Of course I still have to weed out the blurry ones and I definitely have to think of a solution of what to do with the originals and the ‘treated’ ones…

  • http://kristanhoffman.com/ Kristan

    Dare I ask… how much space (gigabytes, terabytes, whatever) of photos do y’all have??

    As one little person, I can’t even begin to imagine needing all this. But then again, I shoot in JPG format and delete the shots I don’t like.

    (I know y’all do large prints so that changes the equation a lot.)

  • http://passion.squirrelism.net/ natalija

    Your photo workflow is both disturbingly and wonderfully organized. I’m frightened of it, but I would definitely want it over my own organization, which is basically just, “Throw it all into a photographs folder with the photos dated through Lightroom dating and then organize it when I get to it.”

    Though, your folder scheme is certainly inspiring me to get to organizing my own.

    Thanks for the motivation. ^_^.

  • brebolivar

    So….if I don’t do a whole lot of photo editing, is it still a good idea to shoot in RAW? I have a nice digital SLR but don’t totally know how to use it, but perhaps someday I’ll be upset that I didn’t shoot in RAW?

    • http://blurbomat.com/wordpress blurb

      RAW gives you a lot of flexibility with your shots. It’s kind of hard to explain in a comment. I’ll probably do a post.

      This guy is against RAW for the most part:
      http://www.kenrockwell.com/tech/raw.htm

      Whereas this guy makes a strong case for RAW:
      http://michaelmistretta.com/2008/raw-vs-jpeg/

      I like being able to recover from mistakes, especially when shooting kids in sub-optimal settings and pets. RAW gives you more flexibility and more ability to tweak. Even RAW shots that are not taken with a super high end camera give you more to work with.

      • brebolivar

        thank you!
        i look forward to the possibility of a RAWvsJPEG post.

  • http://emailtoid.net/i/c09f1d8a/ef79f52a/ emailtoid.net/i/c09f1d8a/…

    Not familiar with Drobo, but I’m guessing you can accomplish essentially the same thing for far less money using older machines configured with FreeNAS installed on them. FreeNAS is a miniaturized version of FreeBSD linux that lets a box function as network attached storage. You can do pretty much all the same things you just mentioned, including having RAID configurations, throwing on additional external USB drives, etc., and accessing any of it via standard network protocols, inside or outside your network.

    Robert

    • http://blurbomat.com/wordpress blurb

      I’ve toyed with a NAS, but the Drobo adds a level of flexibility that a RAID alone does not. You really should look into the Drobo, if only for research.

  • http://surfmonkey89.myopenid.com/ Paul Lukinich

    Where did you get this killer nested comment system btw? Or did you roll your own?

    • http://blurbomat.com/wordpress blurb

      Thanks!

      WordPress 2.7 has comment replies, but it’s a bear to figure out. I’m using that and a plugin for OpenID visitors (like yourself).

      I coded the CSS so it makes it easier to see who is replying to you. :-)

      • http://surfmonkey89.myopenid.com/ Paul Lukinich

        Thanks for the reply, and great job.

        This is one of the few sites I frequently visit even though I also subscribe via Google Reader because the actual site offers more than the feed, ie great design, much bigger pics, cool community except for the rabid conservatives in some comment threads :)

  • stellahella

    Jon,

    I don’t have a question but just wanted to say thanks for the post. I’m just getting ready to upgrade to a digital SLR and have been trying to figure out a filing system and where/how to store such large files (I’m a Mac user too, but have a G4 iMac).

    And you’re spot on about taking great images with any camera. I just got accepted into a photography show with pics I shot on a digital point-n-shoot.

    Looking forward to your next photography-related post!