Monthly Archives: October 2006

The Problem of Email

The Problem of Email.

By Allison Randal

I have a problem, and its name is “email”. Many people have the same problem. Not many have it quite as badly as I do. When I say “my inbox is out of control”, people respond “Yeah, mine too. I spent 5 hours this weekend and knocked it down from 3,000 messages to 50 messages and I feel so much better.” I have over 20,000 messages spread out over 5+ inboxes. This is after I declared defeat 5 months ago, dumped everything into an archive, and started fresh. This is after I unsubscribed from all but the critical mailing lists (Perl lists and internal company mailing lists). This is after spending 3-5 hours every day working on email, and sometimes spending all day on it.

You have to understand, email is the primary means of communication in both my work for O'Reilly and my open source work. When I interview a company and write-up the hot new technology I found, the result and following discussion is email. When I respond to author questions, edit chapters, look for new authors, or develop book proposals it's all via email. When I checkin code or design documents to Parrot, it comes back to me as email. When I review other people's checkins, design proposals, bugs, and feature requests it's all email.

Then there's the semi-personal communications from my bank or credit card company, receipts from online purchases, from my long distance service, from my ISP and colocation accounts. Personal messages are a relatively small percentage of the email I get, and often get buried in the deluge.

Part of it is also spam, though not as much as you might expect. I spend about 30 minutes a day tagging uncaught spam. I spend a half-day (or more) every month or so rescuing falsely tagged spam from the spam folders and deleting the actual spam.

I'm on a quest to eliminate, or at least alleviate, the pain of email. Aside from spam, I can't reduce the flow of email, but with better tools I could be faster and more effective at managing the email I've got.

  • I want spam filtering that works, rather than just skims a few messages off the top. (My O'Reilly account only gets a couple of spam messages a week, so that's my gold standard of spam fighting.)
  • I want accurate complete-text searching across all my inboxes and archives. I want search results that show the context where the search word was found, instead of just the message header.
  • I want tag clouds (or keyword clouds) for each inbox, and keyword lists for each message so I can see categories of other messages related to it. (Using both automatically selected keywords by frequency across my collection of email, and the ability to manually set and block specific keywords.)
  • I want the option of automatically archiving messages older than 30 days.
  • When I flag a message as “Urgent” or “TODO” I don't want it to just change the color of the message subject in my inbox (which gets lost in the sea). I want it to show the TODO messages in a box off to the side of the email window with a quick way to flag them as “done” (perhaps a checkbox).
  • I want virtual views of my email, so I can see, say “all the non-spam messages on all my accounts that arrived in the past hour”.
  • I want graphical views of my email, so I can see where the most unread messages are, where the most storage is being absorbed, or where the oldest messages are, and attack that inbox or folder first.
  • I want all this in a client that runs on my laptop without network access so I can work on a plane or train.

I've seen fragments of these features in existing mail servers or clients, or other applications both online and offline, so they're all within the reach of current technology.

I'll be trying out a few solutions in the next few weeks. Let me know if you have any suggestions.  [O'Reilly Radar]

Cultivating Server Farms

Cultivating Server Farms.

By Tim O'Reilly

Reprising a theme that we wrote about on Radar a few months ago (see Cloudy, with a Chance of Servers), the New York Times has an intriguing article about the resurgence of the data center business.

Building data centers was a booming business before the dotcom bust, but that event left over a million square feet of unused data centers. That glut is long gone now, and there's a rush to build new centers.

The article makes a lot of interesting points — the diversification of data center business beyond technology to mainstream businesses, the rising cost of building data centers (as high as $1000/square foot!), and our favorite, the importance of power as a gating factor.

“The first thing we look at is power,” Ms. Backaus [CEO of Data Center firm Equinix] said. “Getting generators today is the No. 1 thing that will drive your construction schedules.”

Cheap electricity has become so important, it has drawn big companies, like Microsoft, Yahoo and Google, to Washington and Oregon to build their own server farms. Microsoft is building a center of 1.5 million square feet on 74 acres in Quincy, Wash., close to nearby hydroelectric dams. Yahoo is building one nearby and Google is putting up a center across the border in Oregon.

Their moves to the rural Pacific Northwest “just tells you how important power has become,” Mr. Magnuson said. “That’s the driver of the industry now.”

[O'Reilly Radar]

Iraq: Victory Not An Option

Iraq: Victory Not An Option.

The Baker Commission will be providing a report on the options
available to the administration regarding Iraq after the midterm
elections. Of the options, victory is not one of them in the
foreseeable future. Neither is stay the course. The report will outline
at least two major options, draw down and redeploy, or, stability first
in Baghdad.

Victory, as defined by President Bush as an independent democratic
Iraq favorable to peaceful relations with other democracies, is not in
the cards according to the NY Sun's reporter, Eli Lake.
The draw down and redeploy option will recommend removing our forces
from the sectarian violence and focus their resources on al-Queda and
terrorist organizations seeking a foothold in Iraq. The other option,
would focus our troops efforts on quelling the civil war in Baghdad
because, if there is no stability in Baghdad, there can be no stability
in Iraq.

James Baker has headed up a bi-partisan commission established by
Congress to assess the current state of Iraq and options for future
involvment in Iraq by the United States. Baker is also an attorney
representing Exxon-Mobil and the Saudi Arabian government which has
some interesting implications in this story, as yet, undeveloped.
While, the Baker Commission has no authorization nor power over the
White House's pursuit of the Iraq War directly, many key Republican
Congresspersons have hinted or stated that Congress' support of the
Bush administration's 'stay the course' policy may no longer be
supported. Pres. Bush himself in last week's press conference talked of
“adapting” and adjusting direction in Iraq, no doubt, in anticipation
of the Baker/Hamiltion Commission report.

As I wrote here at WatchBlog in Iraq and Befuddled Republicans,
Sen. John Warner (R) is giving the Maliki government in Iraq just 60-90
days to bring about a truce between the Sunnis and Shiites in Baghdad,
or Congress will demand a change in course of the White House's
objectives and prosecution of war in Iraq. Since, Congress holds the
purse strings to the Bush Administration's efforts in Iraq, Congress
has the power to withhold or, redirect its funding for Pres. Bush's
directives and policy direction in Iraq. Because this could result in a
showdown between the Republican White House and Republican leadership
in the Congress, (should Republicans hold control of Congress after
Nov. 7), it is obvious why the Baker Commission report is not scheduled
for release until after the Nov. 7 elections and before Mar. 15.

It is not likely a coincidence that Sen. Warner referred to 90 days
last week as the magic time frame for a change in course and the Baker
Commission's news conference in which its co-chair Lee Hamilton was
quoted by The New Republic:

“This
is a very tough problem, our options are limited, and there is no
silver bullet, no magic formula to the problem of Iraq,” he said. For
reasons that he declined to elaborate upon, Hamilton said the next
three months in Iraq will be “critical,” particularly in the areas of
securing Baghdad, national sectarian reconciliation and the provision
of basic governmental services to Iraqis.

The Sydney Morning Herald, (Australia), reports:

In
his first public comment on the group's report, which will go to the
President, George Bush, and Congress after the elections, Mr Baker said
Mr Bush's “stay the course” policy might need adjusting. Most believe
Mr Baker's group has recommended changes in the Iraqi constitution that
would allow a very loose federation incorporating three virtually
autonomous regions: Sunni, Shiite and Kurdish.

[Third Party & Independents Watchblog]

Consultant's e-mails show Mormon plan for Romney

Consultant's e-mails show Mormon plan for Romney. Despite repeated denials by the Mormon Church and Governor Mitt Romney's advisers, e-mails from a key Romney consultant state that the leader of the worldwide church was consulted on an effort to build Mormon support for the governor's potential presidential bid and that a key church leader has been involved in mapping out the plan. … [Boston Globe — Front Page]

My Personal Security Program

My Personal Security Program.

Several people wrote (of yesterday's entry) to ask that if I disapprove of Symantec's security solutions, what do I recommend? And what do I use myself?

First of all, it's not entirely about software, and certainly not about one product. One package from one vendor can never be the solution to everything. What I do is fairly simple, and I have little or no problem with malware. Here's what I recommend, because here's what I do, in decreasing order of importance:

  • Be behind a router. They're cheap, and extremely strong protection from unsolicited outside connections from script malware, which studies have shown scan your IP address every few seconds. Even if you only have one PC in the house, get a router. They contain a Network Address Translation firewall, and most allow you to block specific ports. The one I use is the Linksys 8-port router/switch, model BEFSR81.
  • Don't use IE. Just don't. Almost anything is safer. I have it installed because the occasional nitwit Web site won't render correctly without it, but IE often goes untouched for months on end. Use either Mozilla Firefox (which is free) or Opera, which has also been free for about a year now, and is superb.
  • Make sure “install on demand” is turned off in your browser. Certainly, if you run IE at all, this is desperately necessary.
  • Don't use either “big” Outlook or Outlook Express for email. These are malware magnets largely because they're so popular, but they also have some fundamental problems that make them spam facilitators. (They also use IE to render HTML-formatted email, which makes them vulnerable to most of the same exploits that plague IE.) I used Poco Mail for a long time, but recently abandoned it for corrupting my mailbase. I use Mozilla Thunderbird now, and am quite happy with it, even though Poco has a few nice features that I miss on occasion.
  • Don't surf to porn, warez, or obscure music sites, especially those that offer deals that seem too good to be true. These are the primary source of browser exploit trojans.
  • Research every piece of “free” software you install, thoroughly”and resist installing stuff on impulse. This especially includes browser toolbars and ridiculous crap like those heavily advertized smileys, which are highly malevolent spyware and have gotten some of my nontechnical friends into a world of trouble. If you must fool with such stuff, buy a copy of VMWare Workstation 5 and learn how to use virtual machines (VMs) as software testbeds. I test everything I download in a VM before I ever think of installing it directly on the hardware. Stuff that I end up not using much I just leave in a VM image on my very big hard drive.
  • If you must surf to dicey sites, do so from a browser operating in a VM. I go to ebook pirate sites regularly watching for pirated Paraglyph material, and when I do so, I pinch off a new image of a standard VM and then revert it to the stored image when I'm done, whether it looks like I was attacked or not.
  • Use a low-profile virus checker like AVG. I have used AVG Free for some time now, having also tried Panda and found it too resource-hungry. Viruses are less of an issue than they used to be, especially if you're not so stupid as to open any email attachment that rides in the door, or install warez downloaded from P2P networks.
  • Use a two-way software firewall. I use Zone Alarm and have some some years now. It allows me to control what actually gets out to the Internet, and it lets me know when any app even tries to connect. Much modern software, even if it's completely legitimate, wants to “phone home” for reasons never entirely explained. ZoneAlarm puts an end to that. Also, if you contract some kind of spyware or trojan, ZoneAlarm will let you know when the bad stuff tries to make an outside connection.

That's what I do. Interestingly, I don't do something that makes a lot of sense, and I really should try it: Run Windows as something other than admin. If you create and work from within a limited-permissions account, malware based on exploits (or anything else, for that matter) cannot install itself. You have to reboot into admin to install software, but how often do you actually install software? If you're configuring Windows for a non-technical person who is primarily interested in Web and email, and perhaps word processing, this is a very good thing to do. Unfortunately, some software doesn't work correctly in limited user accounts, and you may not know which software fails until you try it.

Finally, before all my Mac friends start yelling, you can buy a Mac. Macs are largely immune to malware because the machines are fairly rare and sparsely connected (compared to Windows) and malware authors are looking for raw numbers. However, if Apple would ever get its head out of its own you-know-what and decide to become a major player, the bad guys could turn their attention to OS/X. I have not yet been convinced that Mac software is somehow inherently exploit-free and I think complacency would be a very bad thing.

So there you have it. Note that a lot of my list cooks down to, “Don't be an idiot.” It isn't all about software. It's about common sense and a little caution. The best trainable, configurable anti-malware system is you.  [Jeff Duntemann's ContraPositive Diary]

Connectedness

Connectedness. There is something fundamentally different about a small town where many of the residents work in or around the town. There is a connectedness that you don't find in small towns that are bedroom communities I am sure some bright young sociologist has already mapped this out, but I have discovered a phenomenon in Floyd that I have not experienced elsewhere and I want to jot down my observations before they fade away and lose their freshness.

I am sure there are other places like Floyd, but I have not seen
them. I have lived in several small towns, some of them as small as
Floyd, but they were mainly bedroom communities for commuters.

In these towns, neighborhood relationships were mostly created by
mothers with young school-aged children or by employees of companies in
the same industry. The networks were fragmentary and most families were
essentially isolated, even from their immediate neighbors.

I lived in many small communities where I didn't know the names of the neighbors I could see from my front steps.

Our lives were severely compartmentalized.  Neighbors would get
divorced and move away and the remaining neighbors would not know where
they had gone or what happened.

In Floyd, on the other hand, it seems like everyone is connected in
some way and the effect is multiplied because many people work several
jobs. We discovered this early on when our mailman introduced himself
as our County Supervisor.  By the time we got to town the next day to
go shopping, several people knew who we were and where we lived.  … [Making Ripples: post-corporate adventures]

Merlin's top 5 super-obvious, “no-duh” ways to immediately improve your life

Merlin's top 5 super-obvious, “no-duh” ways to immediately improve your life.

How to get organized and stay that way

Herewith: Merlin’s top 5 super-obvious, “no-duh” ways to immediately improve your life.

  1. Reduce noise – We all have innumerable inboxes, interruptions, and distractions that are part of work and life — you can’t change that. What you can do is get more hard-nosed about the elective diversions that you invite into your world. Cancel a subscription for a magazine you never read or sign off an annoying mailing list. Needles get easier to find when you aren’t constantly adding new hay to the stack.
  2. Write things down – Ever find a piece of paper in your office with seven digits on it? You know it’s a phone number, but whose? Get ruthless about jotting down ephemeral information if you’ll need to recall it later. Remember that your brain is a creative organ with limitless creative possibilities — but it makes a really crummy whiteboard.
  3. Focus on action – My favorite productivity book, “Getting Things Done” highlights how anything you want to do in life eventually comes down to intentional physical activity — even if it’s something as mundane as “take out trash” and “call Mom.” Learn the habit of planning your world around action verbs rather than fuzzy nouns. “Implement Strategy” is not a task; it’s a project. “Call Jim about strategy” is a very do-able “next action” that keeps the ball in motion.
  4. Get out of your inbox – Many of us are habituated to living out of our email inbox, voicemail, and the other “in baskets” of our lives. Instead, try to set aside regular, periodic times when you trawl for the new content in your life — then get back to work! Inboxes are delivery systems, not workspaces. The real work is happening in your brain and practically every other place that’s not an inbox. Stop allowing yourself to be brow-beaten by the latest, loudest, or most dramatic item that’s landed in your world.
  5. Get pickier – You are the sole person in your life who gets to decide where your time and attention can go. Take that responsibility seriously by not wasting time on junk. You know in your heart what’s really important to you — does the current direction of your time and attention reflect that? Is “kid hugging” time where it should be in proportion to “Blackberry checking” time? Be mindful at the highest level about where you focus your energy, and always strive not to squander it on undeserving activities.

  [43 Folders]

Spyware cleanup

Spyware cleanup.

AKA the great cleanup of 2006.

I've always said that I wasn't a good test subject to measure how effective any anti-trojan or spyware application is, so while I've kept a professional interest in the latest and greatest tools, I hadn't been given a really good opportunity to try them out, until last week.

A friend of Angela had a laptop that suddenly had this program called Brave Sentry popping up, telling her she was infected, and promising to fix it once she purchased the software, naturally. It wouldn't shut down, and it had taken over her desktop wallpaper, blocking her from changing it. I suspected that wasn't the only thing causing problems on this machine, but I wouldn't know for sure until I got my hands on it. I agreed to take a look at it, and try to get it back to being usable. . . .

At this point, it was time to do some research about this spyware, which was being identified as Adware.Virtumonde, and it was time to assess where I was in this process. Brave Sentry was gone. Almost all of the spyware and trojan files had been removed. I had a couple of dll's still connecting to the Internet and displaying popups that continually regenerated themselves and their registry entries whenever I tried to delete them and I still couldn't change the desktop wallpaper, though I could get to task manager. Definitely time to set this aside and do some more reading about these specific problems. Tomorrow, I'll talk about what I found out, and how this ends.   [Out of the Frying Pan, and into the Cube]