speaking of new speak, a report from the Archives
As everyone should know, one of the coolest things on the net is Brewster Kahle's Internet Archive, and in particular, his Way Back Machine. The Archive has been collecting copies of the Internet since 1996 (except for those parts excluded either expressly or through a robots.txt file). Using the Way Back Machine, you can see how a web page has changed over time.
As many have noted, the Way Back Machine helps correct one particularly Orwellian feature (bug) of the net -- that it has no memory. You can go back to a web page, and not know it has been changed. And recently, Brewster captured a particularly rich example of this airbrushing of digital code.
On May 1, 2003, the Whitehouse's Office of the Press Secretary released this press release, announcing "President Bush Announces Combat Operations in Iraq Have Ended." But then, with airbrush magic, now the same press release has been changed to this, which reports "President Bush Announces Major Combat Operations in Iraq Have Ended." No update on the page, no indication of when the change occurred, indeed, no indication that any change occurred at all. Instead, there is robots.txt file disallowing all sorts of activities that might verify the government. (Why does any government agency believe it has the power to post a robots.txt file?)
Why would you need to check up on the Whitehouse, you might ask? Who would be so unAmerican as to doubt the veracity of the Press Office? Great question for these queered times. And if you obey the code of the robots.txt file, you'll never need to worry.
| Permalink | technorati

Comments (19)
Excellent point, professor. Government web sites should not post any rules for visitors or their scouts. I wonder how many government web sites have robots.txt files - and what is in those files. Sounds like a project for Ben Edelman and his clever little programs!
government agencies with robots.txt files (not all of them, but at least some of them)
Flap Erupts Over Bush State of Union Ad
Interesting you bring this up Professor Lessig. The power that the Internet Archive and the WayBackMachine have is truly amazing (and sometimes scary)..IE: content you thought you took off the web...is still there! Although, we've known about GoogleCache for some time already, TheWayBackMachine still provides us with another avenue to explore when looking for that recently removed link or morsel of information goodness.
See also this discussion at Slashdot:
Memory Holes and the Internet (updated)
Turned out not to be as sinister as the poster originally thought, but shows clearly how different Internet publishing is from print media. While I accept that the publisher was within their rights in this case, I find it hard to believe that the publisher stands to gain as much from future re-publication of the article as the public loses. It's like the publisher is trying to embrace the business model of the music industry even as they can see it failing for the music industry.
It seems that history has always been easier to rewrite than we'd prefer to imagine, but it's getting even easier as we rely on more perishable media.
Sorry, that link is:
Memory Holes and the Internet (updated)
For what it's worth, the text of the president's remarks states that "[m]ajor combat operations have ended," in both versions. In other words, the press office only "airbrushed" the title of the press release, not the remarks contained therein.
The memory hole.
The White House claims (in this 2600 news story) that the robots.txt file was intended to prevent files from being indexed twice when they existed in both the "Iraq" section of the web site and the main web site. I tend to believe that claim. Robots.txt buys the administration no real secrecy, and using it for evil would risk major embarassment for little or no gain. Meanwhile, writing a robots.txt rule that becomes too broad because of changing site structure can happen to any webmaster, and often does.
Well that's quite a scoop you've got there Larry. I can imagine that the White House wanted to clear up any confusion for those who can't or don't care to read beyond headlines. Perhaps for those who read "Major combat operations in Iraq have ended" and translate that into "He said the war was over!"
very subtle, orbit. though I'm sorry it seems I was too subtle too. My point is that there should be a convention to signal text was edited on a web story -- especially in this case where the president's position about the "end" of the war has evolved significantly. Blogs have such a convention -- update notices. The same should exist here.
As a researcher at AT&T, I developed a tool to notify me when various web pages changed... then an add-on to tell me how they changed. Kind of like the Internet Archive, on a smaller and personalized scale (and predating it).
Some of my favorite examples in the early days of the tool were a conference that silently changed its dates (starting a day earlier, so anyone entering the original date in their calendar might miss it) and students who predicted graduation in May'96...Dec'96...May'97 :)
LARRY IS RIGHT
As a Republican, one who wants to see children illiterate, old people choose between food and medicine, big corporations get bigger breaks, the rich get more than their fair share, and the environment poisoned, I am sorry to say that LARRY IS RIGHT.
The government cannot and should not do that (change releases, the robots.txt thing). They were wrong and intellectually dishonest. There may be valid reasons for a robots.txt file... but not there.
Regardless of the spin from the left, there are as many defenders of this administration as there are critics. This is the kind of thing that makes defending this president that much harder.
Its hard to take the moral highground when you are playing the games you swear the other side plays.
-SeanB
Were you being “too subtle” or are you changing your story? And I note a bit of (comment-order) editing yourself Larry...Why did you move my opening comment down, I wonder. At least you got it before the way back machine could cache it! Take a look in the mirror as you cast your stones.
I don't think you were being subtle at all. I certainly wasn’t. The “subtlety” of your original post shines through with your follow-up: "especially in this case where the president’s position about the “end” of the war has evolved significantly."
And how has it evolved? Do tell…please illustrate with primary sources. I’ll take Bush, Fleischer or McClellan to make it easy on you. But just because *you* say it doesn’t make it so….
...but hey, why not take a "too subtle" partisan stab hidden in a message pondering software configuration management issues in web publishing? Oh wait, got those two reversed...not too subtle, eh? So in your response to my first post to your blog, I’m thinking you’ve cast yourself as either a liar or a poor communicator.
“Blogs have such a convention — update notices. The same should exist here.”
Is there a parity in purpose shared between blogs and whitehouse.gov? Would you be happy with a notice “last updated Aug 18, 2003” or are you looking for redlining? I imagine whitehouse.gov serves to get the president’s message out as accurately as possible, while a blog serves whatever purpose its author(s) desire. (An update notice is a feature, not a convention by the way, whereas your spinning is a distraction and not a feature.) Are you positing the presentations must be equivalent? Refinements and corrections in should be made easier for critics to find and pull apart, to accuse conspiracy and cast aspersion? I don’t think that’s the purpose of whitehouse.gov.
I still don’t see where the _content_ of the president’s speech changed on those two web pages…dare I say I’m not surprised we’re picking apart the form of these two pages and not the content?
I really shouldn’t have pared my first reply down to one paragraph. It dripped of “subtlety.”
p.s. Yeah SeanB, you're a Republican...suuuure...the lying continues...like flies to...
Orbit - I am a Republican...
A real one. I can prove it - i have disagreed with Prof. Lessig on lots of things in the past.
And beyond that - you are right too. THe content of the speech was not changed. What he said, which is most important, did not change. My issue is with the White House Press Office changing the title of the press release after it was released. Amateurish. Silly. It opens another door for the codepink, moveon and the Ted Kennedy/Howie Dean black Helicopter crowd to cry "Conspiracy!"
Its hard enough for the admin to get its message out. Case in pont - the media and the left crying "he said there was an imminent threat in Iraq and there wasnt" - they know full well that GW never said that. In his state of the union he said we couldnt wait until there was an imminent threat.
I cant believe you thought I was a lefty! My favorite commentators on the McLaughlin group when I was a kid were Pat Buchanan and Freddy "the Beetle" Barnes.
Sarcasm and Subtlety... hmmmmm
SeanB,
"...one who wants to see children illiterate, old people choose between food and medicine, big corporations get bigger breaks, the rich get more than their fair share, and the environment poisoned"
I take it this was sarcasm then?
But anyways, let them cry conspiracy where there is none...It makes *them* look like fools. Like I said, "saying it doesn't make it so." The Americans that aren't caught and twisted in their own little lefty world will see the words and the meaning on whitehouse.gov for how they were intended.
"I cant believe you thought I was a lefty!"
My apologies, I know it's a terrible insult considering the how patriotic that bunch are. ;-)
Most of the government robot.txt files I glanced at seem to be relatively innocuous, only preventing access to icons and cgi-bin directories. Both of these have a justifiable technological bias behind them.
The Whitehouse robot.txt file though is rather egregious, and extensive. I doubt there is a technical rational.
Most of the government robot.txt files I glanced at seem to be relatively innocuous, only preventing access to icons and cgi-bin directories. Both of these have a justifiable technological bias behind them.
gay sex
Of course, cheap GW gold you could buy, decide to your deciding!