How to save your sanity when a drive fails

Backing up your image library and important business files is deathly serious. But, there’s more to Digital Asset Management than simply running Time Machine once in a while. Tim Grey investigates.

Climate change is entirely my fault. From my sketchy understanding of the science, it seems as though the average temperature of the earth has risen around two degrees due to the amount of carbon dioxide in the atmosphere, released by the burning of fossil fuels, which traps heat around the planet like a big stinky doona. It’s been on my mind, and now it’s time to fess up: I made all that carbon. At last count, there are six external hard-drives hot-glued to the bottom of my desk, a couple of switches, and a server. They’re running 24/7. And still, I’m not totally sure my photos are secure.

This bothers me. So, while I’m waiting for Elon Musk to solve the energy crisis, I thought I’d talk to five experts about how to better manage my piccies. These experts refer to my photos as ‘digital assets.’ Who knew?

© James Brickwood. Harry Triguboff, 2017, shot for The Australian Financial Review.
© James Brickwood. Harry Triguboff, 2017, shot for The Australian Financial Review.

Can’t I have a new lens instead?

No. And anyway, it won’t make you a better photographer. What will make you a better photographer is not ruining a job and costing everyone – yourself included – lots of time and money. “I shake my head when people say they’ve lost their files. At that point, I think that it’s just unprofessional,” admits veteran photographer, William Long. “I just find it incredibly frustrating to hear people who say it’s just all on their iMac. What will you do if shit hits the fan?”

It’s a good question. The industry abounds with horror stories of wedding photographers having to trump up the cost of restaging a wedding. But really, what would you do – particularly if you’re legally obliged to keep a copy (Hint: you might be).

The common response is to freak out. Perhaps you’ll try a retrieval service: but solid-state drives (like an SD card or your computer’s main drive) can’t be recovered once they’ve been rewritten. And old-fashioned stylus-and-platter numbers aren’t much better.

The horrible truth is that all drives fail. Not some drives: all drives, including your backups. According to data management strategy consultant, Thom Mackey: “It will happen. You’ll have your hard-drives fail. It might take a few years, or it might be random. Whether it’s through use, or whether it’s through manufacturing fault, it will fail,” he warns. “You just need to make sure you have a backup. There’s not much more to say about it.”

Stay paranoid

Hopefully, you’re horribly frightened. Good. Now we can get busy. Backup is as easy as 1-2-3. For those who don’t know the old adage, it translates to the following rule: Keep three backups of your data; two of which will be on different storage mediums (like, the cloud and a physical hard-drive); at least one of which will be offsite.

The place to begin is the beginning. Some photographers are paranoid enough to create backups the moment they shoot, configuring their camera to write to two separate cards simultaneously. But even Long thinks this is overkill. “I have never, never – touch wood – had a problem with any card,” he says. “Show me where someone’s fucked up and truly lost all their images.”

While it’s likely that one can prove Long incorrect on this count, it’s probably a rare occurrence. So, the point at which your paranoia should kick in is somewhere around ingestion – that is, getting your photos off the camera.

Melbourne photographer James Morgan has a belt-and-braces approach to file downloads. He sticks his cards in two separate readers, which simultaneously back up to two separate drives (and from there, trickle their way up to the cloud). “Those two card readers fire up a program called Downloader Pro. It renames all the files with dates and job-names,” he explains. “It copies them to two locations straight away – each card goes to a different place. That second download location, nothing ever happens to it – apart from it getting backed up.”

© William Long
© William Long

Say my name, say my name

Once the files have made their way from the card to your machine, it’s time to give them a once-over. While Morgan handles renaming and folder structure at the point at which they’re imported, sometimes a filename isn’t sufficient for identifying your images.

While Long uses a simple Year/Month/Day/Subject protocol when creating folders, it can be extremely helpful to add a little more information. Given the sheer volume of files produced by the contemporary photographer, any help you can provide ‘Future Me’ in locating the right file will undoubtedly be appreciated.

Beyond naming, this occurs in metadata, the minuscule packet of contextual info written into your file that records camera make and model, capture time, shutter speed, and so forth. Many photo management programs like Lightroom, Capture One, and Photo Mechanic make it easy to add extra information to this automatically generated envelope, such as client name, GPS location, copyright licenses, and even subject information.

Award-winning photojournalist and portrait photographer, James Brickwood is an excellent object lesson: he’s obliged to fiddle with his metadata before he submits his photos to The Sydney Morning Herald. “Everything we do is captioned,” he explains. “There’s one or two fields we have to have in, otherwise it takes a lot of work and double-handling. From a work point of view, we cannot find anything unless it’s got a caption in it.”

It allows Brickwood not only to locate his photos – it preserves them for future generations. “At the end of the day, what if I die?” he asks. “I want whoever has my archive to be able to find the images and make sense of it. It’s important. A lot of my work – besides the Herald stuff – I’m shooting for posterity.”

Gearing up

Here’s the bit where we talk gear (and you find an excuse to buy a QNAP, LaCie, or something else from one of the leading manufacturers). Yes, Network Attached Storage (NAS) is extremely convenient and very sexy – it’s essentially a stack of connected hard-drives stuffed in a box that’s accessible via Wi-Fi, and remotely via the Internet.

The box is not just a box, of course: it has a tiny computer inside that runs a little operating system allowing it to operate like a server. This allows photographers to access their files not only on the studio or home network, but while they’re on the road as well. The beauty of the system is that your archive can be set to copy to the backup drive, which then makes a copy of itself, and then makes a copy of itself, and so forth. The other major advantage of a NAS system is that it’s scalable and can grow as your data storage requirements increase. Consequently, adding larger drives should prove relatively straightforward.

The beauty of a NAS system is that, from a user’s point of view, it’s relatively simple. Once set up, your archive mostly takes care of itself, and by instituting a RAID configuration, it can further secure your precious material by using additional drives to add a layer of data redundancy so that if one of the drives fail, the data should hopefully remain unscathed on another of the drives. Aesthetically, a NAS also looks a hell of a lot better, and more professional, than a studio full of plastic boxes and cables. That said, nothing in this life is certain – except for the fact that hard drives fail. And NAS is no different, so you’ll want to consider replacing drives every few years.

Mackey says that he would buy the cheapest storage so that he could have the space. “Generally speaking, storage space is a commodity,” he says. “The difference in failure rate would be a fraction of a percentage. Nothing’s going to have a zero-failure rate, so therefore you have to have a backup anyway.” Mackey’s point here is that even if you have a NAS, you’ll need another backup of your archive on a different device. It’s unlikely you’ll see five drives fail at once, but you can propagate unwanted changes relatively easily.

Morgan, for instance, runs his archive using only external drives. “I just wanted a system that used external drives as backups, because they’re the cheapest and the most accessible,” he says. “And, a NAS is usually limited by Gigabit Ethernet, and even 10 Gigabit Ethernet is just not that fast compared to internal drives, or even USB 3,” Morgan says.

Obviously, the speed of a periodic backup isn’t that important. But, if you’ve got a substantial archive, there’s still the distant possibility that you might want to look at it. This is as much a matter of information hygiene as it is technology: we know that if the TVs on, you’ll look at it. This works in reverse, too – if something’s hidden from view, humans have a tendency to forget about it. So, it’s worth considering all the variables – speed, simplicity, reliability, and budget – when settling on a storage medium.

© Kristoffer Paulsen. Yassmin Abdel-Magied, photographed for Good Weekend.
© Kristoffer Paulsen. Yassmin Abdel-Magied, photographed for Good Weekend.

Get out of the house

The beauty of having simple external drives (ideally in a case or housing) is that, well, they’re external. Brickwood, for instance, keeps a third copy of his files on him at all times (which, again, is probably at the extreme end of the spectrum). “I’ve got A and B copies at home, and there’s a C copy in my van,” he says. “They’re all 3.5-inch drives, and they’re sitting in specially-made plastic boxes with foam in them.”

Long, on the other hand, has his third copy in the back shed. “I have a waterproof and fireproof safe, and that’s to save me from the worst possible scenario,” he says. “Nowadays, I could upload to Backblaze, but frankly I’ve got too much stuff.”

Beam me up

Backblaze, for the uninitiated, is an online storage service sometimes referred to as ‘The Cloud’. Google Drive, Dropbox, OneDrive, et al are all species of this mysterious vapour. Some of them are better than others – but not by much, really. “The whole market at this point is relatively mature,” says Mackey. “There’s not much difference between [what’s on offer].”

Nevertheless, online storage can be divided into two types of services. Products such as Google Drive broadly operate like network attached storage, while Backblaze periodically creates an archive that can be used to restore your computer in the event of an emergency.

Melbourne photographer, Kristoffer Paulsen is a fan of CrashPlan, which backs up his whole computer periodically. But, being technically a millennial, he’s only semi-monogamous when it comes to online storage.

Unfortunately, backing up to the cloud might be feasible overseas, but here in Australia, anyone with a modest archive is going to struggle with transfer times. Getting my own files to Backblaze, for instance, was going to take 364 days on an NBN connection. And here lies the problem…

The tragic tale of the NBN

Basically, Federal Governments of both persuasions flopped the landing of the National Broadband Network. While this ‘nation building’ exercise was initially designed to bring Australian Internet up to global standards, we’ve ended up with a two-tier service that barely meets (and mostly misses) current needs.

For photographers, the abject failure of the NBN has meant that managing your archive in the cloud remains a pipe-dream. Many areas – both metropolitan and regional – are still without service, and those who do have a connection know that speeds are inadequate for transferring large amounts of data.

“Turnbull fucked it up,” says James Brickwood, echoing a sentiment with which most photographers agree. “Australian Internet is really slow. The amount of time wasted sitting there waiting for things to upload. The productivity lost is ridiculous.”

While the fibre-to-the-node debacle is widely known, what’s less commonly discussed is that fact that higher-speed services that use the NBN’s full potential currently exist – but they’re so wildly expensive that only major corporations could afford them. “They’re throttling uploads because they want to be able to sell expensive corporate plans. They’ve all leaned on the NBN to make upload speeds a bit shit,” says Morgan plainly.

© James Morgan. David Bromley at his Prahran studio, photographed for Alquimie Magazine (edition 2), 2013.
© James Morgan. David Bromley at his Prahran studio, photographed for Alquimie Magazine (edition 2), 2013.

The future of your files

Still, we live in hope. One day, we might have an Internet service that’s fit for purpose. When that day comes, the spaghetti network of boxes and cables on the bottom of my desk will become obsolete. But online storage isn’t the only possibility of a connected future.

Meanwhile, it’s certainly reasonable to imagine that files could be streamed directly to the cloud in camera while standing on a hillside by Machu Picchu. “If my phone can take pictures, why can’t my camera make phone calls?” asks Long. “By that I mean – instead of plugging some kind of system in to Bluetooth or Wi-Fi, why can’t I just put a SIM-card into my camera?” It’s a good question. Surely the technology exists to start streaming files direct from your camera.

Mackey, meanwhile, is excited about the opportunities machine learning will offer photographers. Very decent image recognition already exists, and it’ll soon be even better. Right now, finding images depends on mostly manual tagging and filing, but in future, a computer should be able to organise and extract everything you need. “My rule of thumb is that anything that you can see is in the photo, the computer will be able to tell you is in the photo – if not now, then very soon,” Mackey says. “It’s plausible to be able to train a model to find aesthetically better photos. I’m a sceptic, but image recognition is the one area that machine learning is useful. So, it’s of real interest to photography.”

While this might sound a little abstract at present, imagine the possibilities of having your database universally connected, and searchable by anyone using human-language terms such as, ‘photos of Nico eating a hamburger on a horse in 1974’.

Sadly, that’s all in the future for the minute. In the meantime, you’ll have to survive on your own wits if you want to keep your photographs. So, if you’ve already got a backup drive, it’s time to get another one. 

Contacts

James Brickwood        

William Long    

James Morgan  

Kristoffer Paulsen      

 

cph-newsletter
Get more stories like this delivered
free to your inbox. Sign up here.