Say I have a plain text file of 500 dates formatted as MM/DD/YY and I need to change them to YYYY-MM-DD. There are a couple of options. I can do it all by hand, wearing out the backspace and arrow keys, and opening myself to the possibilities of typos. Or, I can find an automated way to do it. Say I’m slow, and it takes me three hours of fighting to find a good text editor and figure out how to use regular expressions to make the changes all at once. It would’ve been quicker just to do it all by hand. So which method is better?

Obviously, using regular expressions was much more efficient, but the overhead was much higher. There is a comparatively steep learning curve, and it takes a lot more time to figure out and implement than mere manual labor. But it’s an investment, and the investment is all up front rather than being spread over years. Some day I’ll need to do something similar again, and leveraging the experience I gained here will make it that much quicker.

Pursuing efficiency even when the road is bumpier and filled with pitfalls is a hard resolution to make.

Personally, I’d prefer to look for a creative and automated solution to a problem, even if it takes ten times more time and effort than doing it the normal way. The normal way is boring. Being engaged in the mind is always the better choice over being efficient in inefficient processes.

Take my How to give file names to your photos article for example. A pro-progress yet short-sighted person might ask, “Why would you waste time on something so trivial? Just give descriptive file names to your photos like a normal person.” But that’s not the point. The point is to find the better way to do things and follow through, even if the upfront investment is higher. Inventing is always more fun, even if the gains are minimal, for the coolness factor alone. I’d rather be creative but take six hours than to work efficiently but boringly in three.

This website is an example of high efficiency plus high investment. When I post a photo here, all I do is upload a file, link to it, write out a description and some keywords, and possibly a source image, which takes all of ten minutes. Then, an army of automated processes take over to start updating tag pages, category listings, RSS feeds, counters, galleries, posts lists, and more, site-wide and beyond. A URI is created based on my title, automatically (spaces turn to hyphens, etc.). The post counter in my sidebar updates. Three thumbnails with links are dynamically created with links: a small one for the header, a large one for the post, and a medium one for the gallery. The gallery is automated updated and every older photo is pushed back. The small thumbnail appears in the header randomly. The large one pops up right on the screen when you click, and all the code is handled on the server side. Photos pop up at the top of the home page, and at the bottom of the index.

My server automatically mirrors the post to LiveJournal and Facebook, serving as a running backup and giving options to my readers. An RSS feed is updated, and a third party called Feedburner sends out an email for my posts to 150 subscribers each day. Sites I link to are pinged, Google, Yahoo, and others automatically start crawling, and my new post starts popping up around the thripp.com network: on the main page, in the global feed, in the latest posts, and in every site’s sidebar. The local search engine is updated behind the scenes. Three of my other posts are displayed below the new photo through automated keyword matching. A printable, formatted version is created and linked to. The timestamp is saved. Archives by date are created. Social bookmarking links are added. A comments system and accompanying RSS feed pops up. When the second visitor stops by, a static HTML copy and gzipped (compressed) version has been created, and one or the other is automatically served up depending on what my visitor’s browser supports. It’s like magic.

The time investment for all this has been quite large. Despite doing almost no original coding and the extensive open-source community around WordPress MU, it’s taken my hundreds of hours to plan and design everything, to work out compatibility issues between plugins, to experiment, to debug, to get the software to do what I want instead of what it wants to do. I had to get a basic understanding of HTML, CSS, PHP, Apache, MySQL, SEO, etc. just to know where to go. Far harder than plugging away on deviantART. But that is someone else’s website, where they make the profits. You might think the efficient way is to forgo the investment and keep posting there. Then when I get banned on frivolous charges or get fed up with the service, I can move to Flickr or MySpace or SmugMug, confuse all my followers, and start the process anew. But to the leader’s mind, that is obviously the wrong way to go, despite the low barriers to entry.

The literal overhead for all this is very high. I even had to switch hosts recently, because the old one wasn’t cutting it with the weight of the processes. There are hundreds of files and scripts on my server, all handling different pieces of a dyanmic website. This is heavy and slow. But the efficiency outweighs it. It’s not efficiency in terms of computer time (insignificant), but efficiency in terms of human time (very significant). The most efficient way for the server is if I post a static HTML file with some text and one image. But this is no good for me, and it’s no good for my visitors. Computer time is always cheap, but your time is quite important. If a much faster or more complex computer can do what you do, even if it costs five times more, it’s worth it. This is efficiency triumphing over investment.

Efficiency is always the higher road, because it’s human. You don’t see cats coming up with more efficient ways to hunt birds. They spend hundreds of years on the same tried, true, and tired practices. They’re being quite “efficient,” but it’s done in a most uneconomical way, so much so that they may as well be wasting 90% of their time, if they were only using creative solutions in the remaining 10%, like weapons or camouflage or traps.

As an employee, you’ll often be stuck with methods that are battle hardened, but highly inefficient. Perhaps it is the library that has all its employees add bibliographic records by hand, because the administrators are too afraid to change systems. Or it could be the office store that focuses on pushing valueless warranties at the expense of its customers, the government agency that stifles thriftiness by requiring you to meet wasteful spending quotas, or the college that is afraid to shift to digital photography or online classes. Inefficiency is the way of the bureaucracy, just as it’s the way of animals and dull people. It’s easy to take the long, uncreative way; there’s almost no risk of loss because it’s a sure-fire success. That doesn’t make it worthwhile, though, because its accomplishments are meager and unspirited at best. To get somewhere, you have to invest time and money, even if it may yield no benefits.

Some organizations are more daring and better to work for. Google is known for that. But all of them are going to turn blindly against investment occasionally. The only way to avoid it is to work for yourself, and even then, doing the best thing rather than the safe thing is hard. Anything else is being penny wise but dollar foolish. Like my sociology teacher used to say: bureaucracies are hardy and resilient. So are cockroaches (except in that photo). Do you want to be like a cockroach? It isn’t worth it, no matter how much vulnerability you are shielded from.

You can use investing to garner efficiency in many areas of your life. Why buy a pack of cigarettes when you can save by buying the whole carton? If you’re a smoker, you don’t expect to suddenly quit, do you? Why publish your book through an on-demand press that charges $5 a copy, when you can get it for under $1 by doing a run of 5000 up front, and paying all at once? That’s what my Dad did. It’s expensive to start, but is going to save you so much in the long run. Unless you totally fail. My Dad hasn’t been successful. But if you don’t take the risk of committing yourself, you’re all that more likely to avoid success. You’ll be stifling yourself every step along the way, because your subconscious mind wants to prove itself right for not being willing to invest.

Investment is not a panacea; you must use it on things you are very committed to; things which have a chance of succeeding. I wouldn’t say buying $10,000 in lottery tickets is a good thing. But if you’re doing one thing well, you are efficiently losing money. It’s a much more efficient way than heading to the grocery store each week to buy just one lottery ticket. All those wasted trips…You could’ve just thrown away your money all at once and have been done with it! And that is the best choice, but only if you’ll be wasting the money anyway.

High investment narrows your focus. Instead of working willy-nilly, you pick a specific path and work at it incessantly. Being a jack of all trades is great, but gaining cursory knowledge of ten subjects can take as much time as becoming an expert in one. I’m trying to become an expert with my photography, at all costs. I hardly play the piano anymore, because it just wasn’t working out when I wanted to focus my attention elsewhere. I’m invested in being unemployed, because I want to make money off my passion without having to work (my work isn’t work—it’s fun). But if I stay on the fence and refuse to invest time, money, efforts, and resources, I’m guaranteed not to succeed. If I focus too heavily on maintaining dead hobbies rather than building my strength, I’m also squandering efficiency. The path is always changing, and old goals need to be abandoned unfinished to make way for the new. But this in fact is the highest investment of all: investment not for your comfort, but for your growth.