What's on the WhiteHouse.gov?
by Comrade X, Kleveland/Social Nerve
Have any of you guys been to the White House website lately? Looks kinda nice, its got a pretty design and a beautiful propaganda language that stupefies the absurd mind. But that is all show, we know that behind the false facade of the White House website there is a truth that is not that pretty. We hope that the site doesn't run on a server that houses any important data, and we also hope that the site doesn't include any important data, or does it? Is there info that appears on the site that's there for a brief time just for props and then goes off? Are there things that are hidden specifically for inside info? And does important info ever get through? This I don't know, but it's a reasonable question. But what I do know is that there are certainly many files and folders and images that the site does not want to ever be indexed or viewed by the general public. Well, if we wonder onto the robots.txt page of the White House we can see for ourselves the kind of files that have been Disallowed and which have been taken off.
What Are Web Robots?
What is a robot, you might ask? The site for Web Robots defines a Web Robot as: " a program that automatically traverses the Web's hypertext structure by retrieving a document, and recursively retrieving all documents that are referenced." In other words, a robot can be put on a site so that they are not indexed into a search engine and cannot be retrieved. This is done for several reasons, and the reasons are obvious.
What's the Big Deal?
Well, nothing really if you don't care about where the country's headed and what's going on in the United States. But if you do then you'd want to know what was so special about certain text files, especially about Iraq, that the government felt had to be blocked from the queries of the world? 2600 reported on this matter:
"According to White House spokesman Jimmy Orr, the blocking of search engines is not an attempt to ensure future revisions will remain undetected. Rather, he explained, they "have an Iraq section [of the website] with a different template than the main site." Thus, for example, a press release on a meeting between President Bush and "Special Envoy" Bremer is available in the Iraq template (blocked from being indexed by search engines) or the normal White House template (available for indexing by search engines). The attempt, Mr. Orr said, was that when people search, they should not get multiple copies of the same information. Most of the "suspicious" entries in the robots.txt file do, indeed, appear to have only this effect." (2600)
Either way, we can still get a look at what's going on at the Dissallow section of the White House Web Site. So keep checking it to see what's up. To see what's up type in whitehouse.gov/robots.txt and you'll see the list as it is presently. To see the archived version of that gigantic list where over 1000 files about Iraq were disallowed go here or here: http://www.bway.net/~keith/whrobots/robotsWHcurrent10-24-03.txt
Article written: December 24, 2003