Python does not have a built in package to connect to MySQL as for example PHP does and therefore PyDumpy relies on MySQL for Python package to work. PyDumpy also relies on mysqladmin to do the actual dumping.

PyDumpy is very simple to use. For example to dump a maximum of 50 000 rows from each table type:

./pydumpy.py -H host -u user -p pass -n dbname -limit=50000

PyDumpy also allows to specify row limits and sorting preferences for each table specifically:

./pydumpy.py -H host -u user -p pass -n dbname -limit=50000 –ask-to-limit –ask-to-sort

If you find this tool useful please feel free to provide feedback by leaving a comment.

This inspired me to write a new version of XML formatter. It’s based on a SAX parser which is kind of “ugly” to implement and build around but because of it’s event based nature it’s super fast and has a very low memory footprint. The new version of the formatter shouldn’t peak higher in memory than 200 – 300kb even when the XML files start to weight over a megabyte. It also should not have any problems with indentation because it no longer tries to tokenize the XML itself and uses libxml to do the job. I also tried to make the tool documented and extendable.

It’s usage is really simple. All you have to do is initialize an object of XML_Formatter by passing an xml input stream, xml output stream and an array of options and call the format method. You might wonder why it requests input and output streams instead of a file name or a string. It does that to avoid high memory consumption. Here’s an example of how one might use XML_Formatter:

require('XMLFormatter.php');

$input = fopen("input.xml", "r");
$output = fopen("output.xml", "w+");

try {
    $formatter = new XML_Formatter($input, $output);
    $formatter->format();
    echo "Success!";
} catch (Exception $e) {
    echo $e->getMessage(), "\n";
}

Nevertheless this tool is quite powerful in what it can do  (I was able to format other website’s XHTML or tidied HTML sources) it also has some problems which are not actually related to the formatter but may seem odd to the user. The PHP xml parser does not understand such entities as   or unsescaped ampersands like in ?x=1&y=1. So it’s the user’s responsibility to provide “correct” XMLs to the formatter.

Other than that I hope it will prove useful to someone. Download the latest version of the XML_Formatter.

Nevertheless i decided to atleast try and see if i would be able to build it and maybe change something, run some tests. As I am a Windows user i found out the only way for me to build Drizzle is through Cygwin. I started with installing the latest stable version of Cygwin 1.5.25-15. I must say that their installer is really nice but i would offer to add a package search feature. Might help when you want to install numerous packages.

So what’s next? I found this wiki page about building drizzle and figured first thing i should do is get Bazaar. I installed the following packages using Cygwin installer:

• bison
• bzr
• gettext
• readline
• libpcre0
• pcre
• pcre-devel
• libtoolize
• gperf
• e2fsprogs

And then went on to get the Drizzle sources:

mkdir ~/bzrwork
bzr init-repo ~/bzrwork
cd ~/bzrwork
bzr branch lp:drizzle

Now onto building. Here’s where all the fun begins.

Drizzle requires a tool named libevent which is not available through Cygwin installer and you must build it yourself. And still you can’t build libevent with the latest version of Cygwin because it lacks certain functionality. After some googling i found a patched IPV6 version of Cygwin that fixes these issues. Added the #define EAI_SYSTEM 11 to http.c and finally were able to ./configure && make && make install libevent.

You also need protobuf installed. And there’s no package for that either. Actually this protobuf is quite nice stuff. Protocol Buffers are a way of encoding structured data in an efficient yet extensible format. Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats.

Now that we seem to have all the packages installed we can start building drizzle. It should be as easy as this:

cd drizzle
./config/autorun.sh
./configure
make
make install

It is not. First to be able to compile Drizzle you need to have gcc4. And even if you do, ./configure must need to know where it is. So we need to use additional flags CC and CXX. Then you need to show ./configure where libevent is installed by adding a flag –with-libevent-prefix=/usr/local or any other place you have it in. I also found a really ugly problem with warnings. I wasn’t able to compile drizzle because it stopped somwhere in gnulib complaining about some warnings that were treated as errors. Funny enough there is a sarcastic option to disable these warnings: –disable-pedantic-warnings. You also probably want to install Drizzle somwhere else than usual by using:  –prefix=/some/deploy/dir.

In the end you come up with something like this: 

./configure CC=gcc-4 CXX=g++-4 –with-libevent-prefix=/usr/local –disable-pedantic-warnings –prefix=/some/deploy/dir

That’s how far i’ve got with it. Though i’m still not able to compile it.  I get an error somwhere in mystrings library that is related to some datatype casting issues. Hopefully i’ll be able to hack through this

We came up with an idea to require developers to write down SVN revision numbers in commit messages. This helps to relate code changes to actual tasks if such a need arises. To do that we’ve created a pre commit hook that requires developers to insert internal project management tool task number to the SVN commit comment field.

A pre commit hook can be a simple bash script. In our case it was this script:

#!/bin/sh
REPOS="$1"
TXN="$2
# Make sure that the log message contains some text.
SVNLOOK=/usr/bin/svnlook
$SVNLOOK info -t "$TXN" "$REPOS" | grep -P 'FB:\s*\d{3,6}' > /dev/null
if [ $? -ne 0 ]; then
echo -e "FogBugz case number is missing in the comment.\n" 1>&2
echo -e "Please add text 'FB: CASE_NUMBER' to your comment." 1>&2
exit 1
fi
exit 0

And then recently i found FirePHP while reading php|architect and it seems to me a such a nice tool it’s well worth to blog about it. If you want to debug ajax applications with FireBug you must do your own variable dumps which break the output. While FirePHP is capable of sending all this data through http headers without breaking the response of your json, xml, image responses. It also provides a very nice library for logging various stuff.

One of the things I really liked is the ability to easily view backtraces through FirePHP. Though it seems to me it can only view backtraces of your own defined local php function calls. It would be nice to see FirePHP integrated with Xdebug. Then it could do a full stack trace with all the parameters involved and even memory usage.

I also love the fact that there is a Zend_Log_Writer for Firebug. It also can send information to firebug console. But it’s not as “sweet & cute” like FirePHP.

I believe that FireBug, FirePHP, Xdebug are the must have tools for every php web developer. It saves more than a reasonable amount of time spent debugging.

Well it’s quite funny but the company behind FogBugz doesn’t really want to provide this kind of functionality. Because they believe this would make people provide bad estimate data to FogBugz. You can read more about it on Joel on Software blog post about amnesia. They might be true about the bad data thing. But in my opinion they aren’t the ones who should decide how people should to use their tools. If people want to shoot themselves in foot – explain to them that it’s wrong and what will happen if they won’t listen and then let them shoot themselves in the foot.

We for example need to know on what tasks our developers were working during the day and what are they doing at any moment of time. And ofcourse seeing that a certain task took too long or that a developer was doing something without a task for 4 hours is very valuable. In other words if people know for what reason this tracking data is gathered they might as well not lie and provide good data. Don’t force your developers to have 8 hours long reports. You should know they spend atleast 2 hours doing whatever they like.

I tried to look for 3rd party solutions that would in some way allow us to have time tracking reports, but I didn’t like any of them and of course they are all commercial solutions. I got really excited to find out that FogBugz has an API that allows to do various actions. One of those actions allows to get a user time sheet report for a certain time interval. So I quickly developed a small php application that would login to the system with all the users we have and aggregate their time sheets. The application itself is no piece of art but I think it’s simple and it gets the basic job done. You are welcome to try y fogbugz time tracking application out yourself. It has a dependency to ZendFramework’s HTTP client. You can easily replace it with anything you like. To install the application you need to configure the index.php by providing the api url, fogbugz users list, current timezone, and path to the zend framework. If you have any problems or requests please feel free to contact me.

I’m also adding a screenshot if you care how the reports look:

What does this have to do with php|architect? Well in the 2008 july edition of php|architect there is a really lovely article about web application tools for mobile phones. I found there are two more PHP tools that a mobile web applications developer should know of. It’s Tera-WURFL and HAWHAW. And again to make it simple Tera-WURFL is a mysql database for WURFL to make WURFL super fast and HAWHAW is a object oriented toolkit to create mobile web applications. Basicly with HAWHAW you can construct pages using objects and then HAWHAW renders them to apropriate formats such as WML, XHTML MP, XHTML using the data it gets from Tera-WURFL. According to the php architect article Wikipedia is using HAWHAW.

I don’t have to do much with mobile web applications I think it’s great that these kind of tools are available because I know it is easy to develop these kinds of applications if I or others have to.

To my knowledge this is the first available php google page rank retrieval implementation that works on any platform. We have spent hours searching for such a thing online but we couldn’t find it. There are some php google pagerank tools online, they all work, but they are all limited to windows machines.

Why is so you might ask? Well originally the google pagerank retrieval algorigthm is not public. But google made a browser plugin that was able to calculate the google pagerank for any website you visit. So some freaky geeks dissasembled that plugin and got their hands on the google page rank calculation implementation.  Then this implementation was ported to various languages such as Javascript, PHP. The google page rank implementation is sort of protected by calculating a “unique” hash of the given URL. And here the MAGIC begins.

To calculate this hash the algorithm overflows 32bit integers on XOR operations. Aaaand.. 32bit XOR overflows work quite differently on windows and linux in PHP! If you overflow a 32bit integer on windows it just truncates the result to 32 left most bits and returns a new integer. SMART! And on linux XOR overflow just returns the MAX INTEGER value. What did we do? Oh.. We created a simple class to simulate windows 32bit XOR operations overflow using the PHP gmp extension. Tadam! We have also cleaned up the code, documented and made it look shiny

You can download PHP Google Pagerank Class and use it at your own will. I hope this will help you. If it did just leave a comment and say thanks because we are such nice guys to help you out

To use the class try:

require_once("GooglePageRank.php");
echo GooglePageRank::get("http://www.yahoo.com");


Happy Programmers Day!

p.s  Google™ search engine and PageRank™ algorithm are the trademarks of Google Inc.

Update: PageRank class relies on the GMP extension which is not always enabled by default. On Linux Ubuntu it comes as a separate package php5-gmp.

Notice that this is a simple CMD like window but it now has some nice colors and new shiny tools.Now in order to make it even more cool you need a tool named PuTTYcyg. PuTTYcyg is a patched version of PuTTY that, in addition to telnet, rlogin, ssh, and serial connections, can also be used as a local Cygwin terminal instead of the Windows console. Download the package and unzip it somwhere. I put it in C:\Program Files\puttycyg. Now double click on a file named putty.exe. Now fill everything as in the image bellow and press save.

Now make a shortcut of this putty.exe on your Desktop. And change it’s properties like this:

Click apply and you are ready to go myfriend! Double click on the shortcut and you should be able to do thins like this including full screen resize yay!

Now .. Begone CMD!