I’ve begun work on a side project to develop a PECL extension for Nephtali to offer significant performance enhancements.  However, I’ve got a few projects that have to be completed before I’ll be able to get back to that project.

In the mean time, I hope you update to the new release and please let me know if you see any issues.

• Updated code formatting to use tabs instead of spaces (now you can set the indentation you desire in your editor), and Nedit now generates code using tabs, too.
• Altered coding style so most of the PECL recommendations are followed.
• Improved error handling within Nephtali, and fixed some related bugs.
• Improved debug output formatting.
• Added n\arr\dump(array $arr) function, which returns formatted (HTML) representation of the array argument.
• Removed some unnecessary count() calls to speed execution.
• Fixed some bugs.

Update and enjoy

Well, the reality that is pales in comparison to the journey that was,  so let’s briefly review the journey.

I was running security tests on my Rackspace Cloud Server after configuring the firewall, and the tests revealed that ports 21, 554, and 7070 were open.  This was not the intention of my firewall, and although I had done a fair amount of reading on iptables over the past few months, I was starting to question my understanding of even basic configuration options.

Eventually, I chatted with tech support to make sure that I wasn’t missing a default configuration of their cloud servers, but to my surprise the tech’s tests of my server did not reveal that the ports were open.  I was tired and wondering if I was imagining things, so I decide it was time to hit the hay.

The next morning, I immediately went back to the computer thinking the abnormal tendencies of my network would turn out to be stupid errors on my part, easily corrected by my sharper, rested mind (OK, I know, even well rested I’m no Einstein, but…)  However, the light of a new day merely revealed the strange truth of my tests.

I had no services running/listening on any of the listed ports, I was using a firewall that appeared to work for many other people just the way I thought it should work for me, and I could see evidence of the firewall working on other ports.  When I tested the ports (telnet, nmap, etc.), they were open.  However, when the tech and later my friend Davin tested the ports on their computers, they were closed.

It appears it was only one bad Apple that was spoiling a bunch of my tests.  I’m using a Time Capsule as my router, and after growing suspicious, a search brought up a rather similar issue.

After isolating the issue, it appears that Time Capsule was trying to “help” the situation by proxying certain protocols.  The issue is that it doesn’t even check with the server on some of the protocols to see if a connection can actually be obtained, it just makes it appear as though one has been granted and passes along follow-up requests.  Needless to say, this approach by Apple didn’t actually “help” me at all.

So, if you’re seeing some unexpected results of ports 21, 554, and 7070 being open on a remote server, I hope you find this blog post so you can quickly check if it’s a piece of Apple networking hardware that’s causing the issue.

Nephtali 3.1.0 has now been officially released.  The newest release of Nephtali:

• Improves the regular expressions used in filtering.
• Adds the ability to send a single row to the n\view\render() function (before you had to wrap a single row in a parent array before supplying it as an argument.)
• Fixes some bugs.
• Allows you to set a default ‘charset’ in nconfig.php (default charset is UTF-8) to be used in the PHP functions that allow you to pass in a charset argument.
• Improves the debug screen output.

Thanks for your support.

The new caching capabilities are especially useful for improving the performance of pages like the homepage for Nephtali, which gets data from multiple web services.  Of note, the homepage also provides an example of how Nephtali tries to improve the experience of its developers.

When a pipe has an error rendering, Nepthali automatically handles the error by displaying the error view for that pipe.  That is to say, when using the Nepthali framework, an error in one dynamic region of a page does not require the visitor be redirected to an error page.  The visitor can still see the other dynamic regions that properly rendered.

When I integrated the first draft of the caching capabilities in the Nephtali website, I came back to the site a few times and realized that on one occasion, the cached version of the page included an error view (one of the web service requests had timed out.)  To handle this situation, I augmented the code in this release so Nepthali only calls the save function for a registered cache if all of the pipes on the page rendered without errors.

[Note: If you're manually handling the errors, you can still avoid having a page cached by calling n\val($name = 'nephtali_is_cacheable', $value = false) to let Nephtali know that the particular request shouldn't be saved.]

Let me know if you have any issues with the new download.

Here’s some example code from the Nephtali homepage which happens to use APC to accomplish the caching:

Code block

<?php
 
// Rest of code for index.php code-behind comes here, then caching below.
 
n\cache\register(
	$stage = n\constant\CACHE_FINAL,
	$fetch_func = function($markup)
	{
		$homepage_markup = apc_fetch("homepage", $success);
 
		if ($homepage_markup !== false)
		{
			return $homepage_markup;
		}
		else
		{
			return null;
		}
	},
	$save_func = function($markup)
	{
		apc_store("homepage", $markup, 3600);
	}
);
 
?>

So, pretty simple.  Caching follows a registration paradigm just like the other features of Nephtali (ports, pipes, vals, etc.), and you merely tell Nephtali which stage of caching to intercept (for now, it’s just the final result, but more will come in the future) and which functions should handle getting and saving the cached results.

1. Resources are few, so how do I limit the cost of parsing the incoming data?
2. How do I ensure those providing the data have properly validated and escaped output coming to my application so-as to preserve the intended message structure?  Anyone familiar with SQL-injection, Email-injection, or XSS attacks will understand the general concerns described here.

There are several data-interchange formats I could use, including XML and JSON.  However, XML parsers can suck up large amounts of resources (and I can’t blame them, parsing XML can get pretty tricky), as can JSON parsers.  Additionally, both JSON and XML offer little in the way of  ensuring that the intended structure of the message is preserved even if one fails to properly validate incoming data that is used to generate the message.

To address these concerns, I’ve developed a new data-interchange format, which I’ve called the Low-memory, Human-readable, Injection-resistant, Data-interchange Format (LHIDF, and pronounced “Lidif”.)  Is the name a ridiculously long, forgettable acronym?  You betcha!

Anyways, let’s first look at a sample of LHIDF:

{leaders:256}
    {leader:256}
        {username:256}bill178{username:256}
        {rank:256}78{rank:256}
        {time:256}89.56{time:256}
    {leader:256}
    {leader:256}
        {username:256}thomas890{username:256}
        {rank:256}45{rank:256}
        {time:256}109.52{time:256}
    {leader:256}
{leaders:256}

Hopefully you agree that it is human readable.  I wouldn’t call it beautiful, but I can easily read the data and make edits as needed.

Every node follows the simple convention below:
{nodeName:messageID}content{nodeName:messageID}

Additionally, there are no attributes like those found in XML, there must be one top-level node, and each node can contain other nodes -OR- a simple value (unlike XML, nodes can’t contain a mixture of other nodes and simple values.)

Of note, the message ID is the same for all of the nodes, and we’ll get to its use in a bit.  The important point is that the format is really simple, and it’s this simplicity that facilitates the “low-memory” component of its name.  Because of the simple structure, a PHP class such as the one below is all that’s needed to parse a LHIDF response:

Code block

<?php
/**
 * Simple class to parse LHIDF (pronounced "Lidif") data-interchange format.
 * The Low-memory, Human-readable, Injection-resistant, Data-interchange Format.
 *
 * @author Adam Richardson of Envision Internet Consulting, LLC
 * @license http://www.opensource.org/licenses/mit-license.php MIT License
 */
class LHIDFNode {
    public $name;
    public $value;
    public static $messageID;
    /**
     * Create a new node object.
     * @param string $name
     * @param string $value
     */
    function __construct($name, $value) {
        $this->name = $name;
        $this->value = trim($value);
    }
    private static function setMessageID($data, $rootNodeName) {
        $temp = next(explode('{'.$rootNodeName.':', $data));
        $positionClosingCurly = strpos($temp, '}');
        self::$messageID = substr($temp, 0, $positionClosingCurly);
    }
    /**
     * Create the root level node and store the messageID.
     * @param string $data
     * @param string $name
     * @return Node
     */
    public static function getRootNode($data, $name) {
        self::setMessageID($data, $name);
        return new LHIDFNode($key = $name, $value = next(explode('{'.$name.':'.self::$messageID.'}', $data)));
    }
    /**
     * Retrieve the matching nodes found within the current LHIDFNode object.
     * @param string $name
     * @return array An array of LHIDFNode objects.
     */
    public function getChildren($name) {
        $tempArray = explode('{'.$name.':'.self::$messageID.'}', $this->value);
 
        if (count($tempArray) < 3) throw new Exception("The node $name could not be found.");
 
        $children = array();
 
        for ($i = 1; $i < (count($tempArray) -1); $i++) {
            if (strlen(trim($tempArray[$i])) < 1) {
                continue;
            }
            else {
                $children[] = new LHIDFNode($key = $name, $value = $tempArray[$i]);
            }
        }
 
        return $children;
    }
    /**
     * Retrieve the matching child node of current node (multiple matching nodes will throw an exeption.)
     * @param string $name
     * @return Node
     */
    public function getChild($name) {
        $tempArray = explode('{'.$name.':'.self::$messageID.'}', $this->value);
 
        if (count($tempArray) != 3) throw new Exception("The node $name could not be found.");
 
        return new LHIDFNode($key = $name, $value = $tempArray[1]);
    }
    /**
     * Retrieve the value of the current node.
     * @return string
     */
    public function value() {
        return $this->value;
    }
}
 
?>

You’ll notice the class is pretty small, and this class makes it easy to parse the contents of an LHIDF message (that said, you could take a more SAXy approach, as I’m doing in Objective-C, and do much better in terms of memory, this is just a simple example.)  To parse the example LHIDF, the code below would work:

Code block

<?php
 
class Leader {
	public $username;
	public $rank;
	public $time;
}
 
$rootNode = LHIDFNode::getRootNode($data, $name = 'leaders');
$leaderNodes = $rootNode->getChildren($name = 'leader');
$leaderObjects = array();
 
foreach ($leaderNodes as $leader){
	$newLeaderObj = new Leader();
	$newLeaderObj->username = $leader->getChild($name = 'username')->value();
	$newLeaderObj->rank = $leader->getChild($name = 'rank')->value();
	$newLeaderObj->time = $leader->getChild($name = 'time')->value();
	$leaderObjects[] = $newLeaderObj;
}
 
?>

So, what about the funny message ID’s, what do they do?  Well, the reason injection attacks work is that the attacker knows some of the general qualities of the intended structure of the message.

But, what if we wrote a data-interchange format and used it only once?  Attackers wouldn’t be able purposefully craft messages that alter the intended structure of a message if they didn’t know the message structure in the first place.  That said, changing the entire format every time you need to send a new message would prove very wasteful.  But, do you really need to change the entire format to neutralize injection attacks?

No, as it turns out you can just append a random message ID to the nodes your expecting, and now injection attacks are thwarted whilst maintaining a simplistic parsing scheme.  For instance, let’s say an attacker wanted to try and inject another leader node through the first username field, they might try injecting the following value:

                       user_name1{username:111}
        {rank:111}1{rank:111}
        {time:111}1{time:111}
    {leader:111}
    {leader:111}
        {username:111}extra_username

A new message ID is randomly generated every message, so the attacker would now have to successfully guess the message ID of a particular message to perform the attack. Because the message ID of the injection attempt above does not match the nodes of the current message, this username would all be parsed as one long, ugly username. No injection!

Now, you might be wondering why the message ID is so small. Eventually, an attacker could guess a 3 digit number. However, LHIDF doesn’t care how big the message ID is, it just uses the root node to determine it’s value and then parses accordingly, so if you want more security, you can have it by using a larger number. You just have to balance the security against the bandwith requirements of your particular application.

Finally, just to be clear, this isn’t a form of encryption or message signing.  There are already tried-and-true options out there for those needs.  This data-interchange format merely protects the intended structure of the message in this format, so if the message was intended to encode data for 2 leaders, the person decoding the message would also receive data for 2 and only 2 leaders.  If you go on to use the data from the message in a SQL query, as output on a XHTML page, or in an email, you’ll still have to properly validate and escape the data.  That said, I believe one could extend the above technologies to be injection resistant, too, such as augmenting XHTML so all tags and attributes include a randomly generated runtime message ID, thereby rendering injection attacks futile.

[Source image found at Sebastian Bergmann's Flickr feed.]

Check it out, it’s simple, relatively efficient, and you can even nest templates if you’d like.  There’s also a screencast that walks you through the new templating capabilities.

Now, back to getting the caching capabilities integrated.

Once I have the caching in place, I would like to develop a simple application using some of the other available PHP frameworks (CakePHP, CodeIgniter, perhaps Yii) to contrast the approaches for the different frameworks.  I hate seeing the comparison of frameworks using stupid “Hello World” examples.

Additionally, I’ll have a new screencast that walks you through some basic DB work with Nephtali in the next few days.

All this to say, things are still improving, and I hope you come back to check on the progress