The Mighty Text File

Don’t get me wrong, databases are cool. As a PHP programmer and web developer, I use relational databases every day, and for complex data sets, searching or sorting, nothing is better. The problem is that as soon as you are using a database, no matter how small, you suddenly have a whole bunch of extra overhead. You need to have a database server, write SQL queries, optimize results, and (my least favorite) build some sort of admin interface to actually add/remove/edit the data. If you are just doing a simple project, all this seems like a lot of work for a very small result. Enter the text file.

For many purposes, storing the data in a flat text file can be just as effective as a full database, and it is so much quicker and easier to work with. Want to add an item? Open the text file up in Notepad and add it in. Same for removing and editing. No muss, no fuss, no admin interface needed. For the first 3 years or so, this is how LoadingReadyRun.com operated. The one downside of using a text file for this kind of project is that is can be a little tricky to read back in a useful way, my first text file base projects had a lot of file[0].” “.file[2]. Which works fine, but is a little opaque if you don’t know the exact format of the text file.

To solve this problem, I wrote the following handy little function and have used it in a bunch of projects since:

 <?php
 
function parse_data_file($path,$delim='|'){
	$items = file($path);
	$headers = array_shift($items);
	$headers = explode($delim,trim($headers));
 
	$out = array();
 
	foreach($items as $row => $item){
		$item_array = explode($delim,trim($item));
		foreach($item_array as $col => $field){
			$out[$row][$headers[$col]] = $field;
		}
	}
	return $out;
}
 
?>

When given the path to a text file, this function returns all the data in an associative array with the column names based on the first line of the text file.

For example, given the following text file:

first_name|last_name|phone|fax
Bob|Fredson|5551235678|5551235678
Fred|Johnosn|5551875678|5551274678
John|Bobson|5551231178|555123444

The function would return the following array:

array {
	[0] => array {
		[first_name] => Bob
		[last_name] => Fredson
		[phone] => 5551235678
		[fax] => 5551235678
		}
	[1] => array {
		[first_name] => Fred
		[last_name] => Johnosn
		[phone] => 5551875678
		[fax] => 5551274678
		}
	[2] => array {
		[first_name] => John
		[last_name] => Bobson
		[phone] => 5551231178
		[fax] => 5551234445
		}
	}

Note: The function defaults to the pipe character, “|”, as a delimiter. I chose this character because it will probably never come up in the data and it also looks as much like a seperator as possible. If you want to use a different delimiter (“;” or “#”, for example) just pass it as the second parameter.

Comments (7)

  1. David wrote::

    That’s a pretty neat function. I agree that all the overhead associated with a DB is undesirable when dealing with small-scale projects. I will definitely keep this little function in mind on future projects.

    Monday, February 16, 2009 at 12:05 pm #
  2. John wrote::

    I’ve built a couple of websites using enitrely text storage and always found that implementing a database can be incredibly difficult and wasteful. If you do need all the functionality of a database though, a database is essentially just optimised text files so it can be more eficient than it looks.

    Monday, February 16, 2009 at 2:43 pm #
  3. King Kool wrote::

    I’m sorry. Bob Fredson makes me laugh. I can’t help it.

    Monday, February 16, 2009 at 2:59 pm #
  4. Paul wrote::

    @King Kool: I have a real problem coming up with random names. If you look at any of my scripts for LoadingReadyRun, you will see a lot of Freds, Bobs and Johns.

    These days I tend to grab random actor’s names off of IMDB.

    Monday, February 16, 2009 at 3:40 pm #
  5. Emperor Gum wrote::

    Notepad: a great word processor, or the greatest? I stopped using Open Office writer completely about a year ago.

    Tuesday, February 17, 2009 at 5:25 am #
  6. kometbomb wrote::

    There are two valid points in there: “why see the extra work” and “if it works, don’t touch it”. However, here are some counterarguments just for the sake of discussion.

    PHP comes with SQLite which pretty much is a zero-setup SQL database. That means, setting up a database and opening it for using is just one line of code. Creating the table structure above is similarly trivial: mere few lines.

    You don’t need an admin section since you can open the SQL console and enter the table creation stuff there. This is comparable to typing in and uploading the text file so I wouldn’t think that has any overhead. In fact, you can create the database at home and the upload the file on the web server (it really is that simple – one file per database). Editing the data can be done with some third party tool similar to PHPmyadmin.

    Writing queries can’t be avoided. However, I can’t see where’s the overhead in that. The only thing different from writing some table accessing code in PHP is that it’s a different language (which I think is pretty essential to master, for your own good). In the end you probably end up typing in far less SQL than PHP, thanks to the database doing things for you (e.g. checking for dupe names and numbers, not to mention if you need to do any relational stuff at all).

    I think my overall point is that while you can avoid extra work by doing things in a simple way, it easily grows into something that has nothing but overhead. If you see just a tiny bit of extra work in the beginning, it might save you the most work of all compared to the full and minimal mindsets. :)

    Saturday, March 21, 2009 at 6:08 am #
  7. Paul wrote::

    @kometbomb: I have used SQLite a little bit and it seems like a good solution if you want a database, but for some reason can’t access MySQL. It does not, however solve the problem of the need for a good CRUD interface. Most of the sites that I build are for people who are not that tech-savvy and telling them to edit the content of their site with PHPMyAdmin or similar doesn’t really work. Storing all the data in a simple text file allows them to add/edit/remove content using any text editor.

    Sunday, March 22, 2009 at 2:53 pm #