Automated Craigslist Search with Cron

I was looking for a way to have an automated search for some items I’m looking for on Craigslist, but nothing exists in Craigslist to email you daily/weekly for items matching a particular string.  I found a few websites that offered this functionality via an application, but I’m not down for that.  Instead I wrote this simple korn shell script.  Enjoy!

#!/bin/sh
echo "--> Setup...";
directory=/my/dir/for/results/;
filename="craigslist_dell_monitor.html";
path="${directory}${filename}";
website="http://mycity.craigslist.org/search/sss?query=dell+2005fpw&minAsk=min&maxAsk=max"
echo "--> Searching for Dell Monitor...";
wget -O ${path} ${website}
ls -la ${directory}
echo "--> Checking for results...";
cat ${path} | grep "Found:"
if [ $? == 0 ]
then
echo "--> Sending results email...";
echo ${website} | mail -s "craigslist search - Dell Monitor" myemail@gmail.com
else
echo "--> No results.";
fi
exit;

Any guesses on what I was searching for? :)

I’m Back

So, I’ve decided to start blogging again.  This time around I’m going to try and keep things more related to technology.  Less stuff about my personal life and thoughts.  More stuff around application development and emerging technologies that I’m interested in and learning .  I think over the past year or so that I’ve stopped blogging I’ve really gotten behind with architecture frameworks (new and maturing) and new programming concepts.  Plus, I guess I might have had some useful information on my site because I’ve had some former readers trying to get a hold of me to get entries that I posted in my old blog.  Because of this, I’ll also try to post some of my old entries that could be useful for someone.

XSL Cross-Browser Transformations w/ JavaScript

Here is an example of how to display an XML (extensible markup language) document using an XSL (extensible stylesheet language) document in your browser.

First, we will need an XML document containing the information we would like to display. Below is the XML document we will be using.

<?xml version="1.0" encoding="UTF-8"?>
<schedule>
	<day value="2007-3-7">
		<hour value="8:00" suffix="AM">
			<length>1 hour(s)</length>
			<attending>yes</attending>
		</hour>
		<hour value="9:00" suffix="AM">
			<length>1 hour(s)</length>
			<attending>yes</attending>
		</hour>
		<hour value="1:30" suffix="PM">
			<length>2 hour(s)</length>
			<attending>yes</attending>
		</hour>
	</day>
	<day value="2007-3-8">
		<hour value="10:00" suffix="AM">
			<length>1 hour(s)</length>
			<attending>no</attending>
		</hour>
		<hour value="1:00" suffix="PM">
			<length>3 hour(s)</length>
			<attending>yes</attending>
		</hour>
	</day>
</schedule>

Next, we will need an XSL document containing how we would like the data to be displayed.

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method='html' version='1.0' encoding='UTF-8' indent='yes'/>

	<xsl:template match="/">
		<html>
			<body>
				<table border="1">
					<tr bgcolor="#777777">
						<th align="left">Day</th>
						<th align="left">Hour</th>
						<th align="left">AM/PM</th>
						<th align="left">Length</th>
						<th align="left">Attending?</th>
					</tr>
					<xsl:for-each select="schedule/day/hour">
						<tr bgcolor="#EAF2FA">
							<td><xsl:value-of select="../@value"/></td>
							<td><xsl:value-of select="@value"/></td>
							<td><xsl:value-of select="@suffix"/></td>
							<td><xsl:value-of select="length"/></td>
							<td><xsl:value-of select="attending"/></td>
						</tr>
					</xsl:for-each>
				</table>
			</body>
		</html>
	</xsl:template>

</xsl:stylesheet>

Next, we will need a simple HTML document.

<html>
<head>
	<script src="./js/loadXml.js" type="text/javascript"></script>
</head>
<body>
	<button onclick="displayXml('schedule.xml','./xsl/schedule.xsl')">Load</button>
	<button onclick="document.getElementById('output').innerHTML='';">Clear</button>
	<div id="output"></div>
</body>
</html>

Finally, we will need a JavaScript file that will be in charge of loading and transforming the XML and XSL documents.

function displayXml(sXmlLocation,sXslLocation) {
	if (window.ActiveXObject) { // microsoft ie
		// load xml
		var xml = new ActiveXObject("Microsoft.XMLDOM");
		xml.async = false;
		xml.load(sXmlLocation);
		
		// load xsl
		var xsl = new ActiveXObject("Microsoft.XMLDOM");
		xsl.async = false;
		xsl.load(sXslLocation);
		
		// transform doc and write to div
		var out = document.getElementById("output");
		out.innerHTML = xml.transformNode(xsl);
		
	} else if (document.implementation && document.implementation.createDocument) { // firefox and netscape
		// create xslt processor object
		var processor = new XSLTProcessor(); 
	
		// load xml
		xml=document.implementation.createDocument("","",null);
		xml.async = false;
		xml.load(sXmlLocation);
		
		// load xsl
		xsl=document.implementation.createDocument("","",null);
		xsl.async = false;
		xsl.load(sXslLocation);
		processor.importStylesheet(xsl);
		
		// transform doc and serialize to string
		var result = processor.transformToDocument(xml);
		var xmls = new XMLSerializer(); 
		var output = xmls.serializeToString(result); 
		
		// write to div
		var out = document.getElementById("output");
		out.innerHTML = output;
	} else {
		alert('Browser not supported.');
	}
}

Here is what the final product looks like. Click “Load” to execute the JavaScript method.

Click here if you would like to download the complete package.

I hope this short tutorial/example can be of help to someone looking to display XML with XSL in a browser using JavaScript.

Script for Installing Subversion on Ubuntu

I put together a script that will automate the installation and setup process of Subversion and Apache2 on Ubuntu. This script follows all the steps outlined in my Subversion tutorial which can be found here.

A few things to point out:

  1. The subversion repository will be created at the root in a folder named “/svn”.
  2. The script will download a new dav_svn.conf file from my server that has been modified for the script.
  3. SSL is not used.

The script requires 4 arguments.

  1. The Username of the Subversion Authenticated user to create.
  2. The Password of the Subversion Authenticated user to create.
  3. The folder to add to the new Subversion repository.
  4. The name of the new Subversion project.
sh setup_subversion.sh [username] [password] [folder_path] [project_name]

Here is an example:

sh setup_subversion.sh jesse mypassword /home/jesse/myproject myprojectname

This script has only been tested on Ubuntu 6.10 Edgy Eft. If anyone runs the script successfully on another version of Ubuntu or another Linux distribution please let me know. Also, I am 99.999% 100% positive that the script will work on Ubuntu 6.06 Dapper Drake.

Update: I have been informed that the script is compatible with Ubuntu 6.06 Dapper Drake.

Note: A script for Ubuntu 7.04 Feisty Fawn has been added.

Pick your version of Ubuntu:

6.06/6.10 Script or 7.04 Script

Setup Subversion with Apache2 on Ubuntu

Note: I have created a script to automate this process. The post regarding the script can be found here.

This guide is for setting up a Subversion (SVN) repository in Ubuntu. Subversion is a version control system which enables multiple users to modify the same document. Therefore, a tool such as Subversion comes in really handy when developing software on a team. Also, Subversion can be a life saver when you need to revert back to older versions of a document (e.g. you made some changes to your code and now everything is broken).

This guide is for creating a repository that requires authentication for all access. This is unlike open source SVN repositories which allow anonymous read access. If you desire, access permissions can be changed at the end of step 3.

Also, this guide does not use SSL and therefore is only recommended for secure local networks (e.g. behind a router at your house or corporate network) if you are working on something important. Maybe I will add SSL to this tutorial in the future.

When creating this guide I used Ubuntu 6.06, Apache2 2.0.55, and Subversion 1.3.1. Ok, on with the walkthrough.

1) Install Apache2, Subversion, and the Apache modules for Subversion. You can install these packages via the Synaptic Package Manager or you can execute the following commands in the terminal.

sudo apt-get install apache2
sudo apt-get install subversion
sudo apt-get install libapache2-svn

2) Now we need to edit the libapache2-svn configuration file. To open the file, execute the following command.

sudo gedit /etc/apache2/mods-available/dav_svn.conf

3) Edit the file in the following locations:

To enable the repository:

Uncomment…

# DAV svn

To…

DAV svn

To set the path of the repository:

Note: This guide uses /svn for the Subversion repository. You can create the repository in any directory you wish, just make sure to change the path throughout the rest of the guide to represent your chosen directory.

Uncomment & Change…

# SVNPath /var/lib/svn

To…

SVNPath /svn

To enable user authentication:

Uncomment…

# AuthType Basic
# AuthName "Subversion Repository"
# AuthUserFile /etc/apache2/dav_svn.passwd

To…

AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd

To make readers & committers authenticate themselves:

Change…

<LimitExcept GET PROPFIND OPTIONS REPORT>
Require valid-user
</LimitExcept>

To…

<LimitExcept>
Require valid-user
</LimitExcept>

4) Now we need to restart Apache.

sudo /etc/init.d/apache2 restart

5) Next we need to create the Subversion repository. As said before, in this guide we are going to create the repository in the directory /svn of the root directory. This command will create several SVN files and directories in /svn.

sudo mkdir /svn
sudo svnadmin create /svn

6) Then we have to change the ownership of the directory so that Apache can write to it.

sudo chown -R www-data:www-data /svn

7) Ok, let’s create an authenticated user that will have access to the repository. The following command will create a user named “bigbamboo”.

Note: You will be prompted for a password.

sudo htpasswd2 -c /etc/apache2/dav_svn.passwd bigbamboo

8) Now let’s create a project. Probably the easiest way to create a project is to import files that you would like to add to SVN. If you would like to learn more about the SVN import command please visit here.

Note:

  • “/home/bigbamboo/myproject” is the location of the files we want to add to the repository.
  • “projectname” is the name of the project we are adding to Subversion.
  • “-m” is used for adding a message to the commit.
  • “bigbamboo” is the username and “mypassword” is the password of the authenticated user.

Note: The following command is one line.

svn import /home/bigbamboo/myproject http://localhost/svn/projectname -m "creating repository" --username bigbamboo --password mypassword

9) You have now created a SVN repository containing a project. To view the files in your Subversion repository you can go to http://localhost/svn/projectname.

From here you are on your own with adding, committing, updating, and reverting your documents. All of those commands can be done via the command line but there are several GUI applications that really help out. I highly recommend RapidSVN for Mac/Linux, and TortoiseSVN for Windows.

If you notice any errors in the guide please let me know. Also, I am always open to any suggestions as I am still learning too.