Software development produces products as any other industry. It is an interesting issue that products produced by the software development industry are not always as well tested as in other industries. One of the main reasons is due to the fact that in most companies software is manually tested by the same developers who built it.

Developers are not necessarily good testers. Good software developers while sharing some traits are different from good testers.  A good developer may be excellent at software design but won’t notice spelling mistakes. Developers tend to take the testing path that usually works while cutting corners on special cases. It is also known as happy testing.

Most reasons why software is tested manually boil down to:

• Not knowing how
• Not having the time
• Maintaining legacy code

Hardly any of these reasons are valid excuses. Truth be told automated testing is not easy. It is a skill that has to be learned and doesn’t come naturally. It’s a skill that takes years to master. It may seem that automated tests take too much time and at first it will. Due to the same fact that it’s a skill and takes time to master. In some way it is like skiing. Before going down a steep mountain racing the wind one has to learn how to otherwise it’s an unpleasant struggle climbing down with your skis across the shoulder. Automated testing won’t do miracles but when done properly it may increase productivity by 10 - 20%.

There are many types of software testing with their own benefits and downsides. Bellow are the three most important and common types of testing.

Unit Testing

Unit testing is about testing the smallest individual parts of an application. For example instead of testing the whole engine of a car all individual pieces are tested separately as units. All dependencies are replaced with stubs or mocks or fakes. Unit tests are meant to be lightning fast and execute thousands of tests in a few seconds therefore they shouldn’t connect to the database, web services or send emails. Unit tests are most effective when executed very frequently during development to inform the developer of any broken parts allowing to fix the problem immediately without losing line of thought.

Integration Testing

Integration tests ensure that all of the application’s parts work correctly when they are assembled in a simulated production environment. An example of an integration test can be a batch job executing against a database with test data. Integration tests are a lot slower than unit tests and usually work best integrated with a continuous build system. While integration tests are not good at identifying broken parts they are excellent at testing if an overall group of components is correctly wired together and produces the desired outcome.

Acceptance Testing

Acceptance testing also known as functional testing or QA testing involves testing a complete system that is usually identical to the user’s anticipated system. Acceptance tests can be automated or may be carried out by a QA team. In agile software development terms an acceptance test tests a business story. For example a simple story may be “As a manager I am able to list invoices, filter them by name and date and approve or reject them” An acceptance test would test that scenarios mentioned in the story can be done on a completed system. Automated web application development acceptance tests can be carried out by selenium that simulates recorded browser actions or by asserting HTML structures. Acceptance tests shine in proving if the user story scenarios can be completed but are usually not very good at pinpointing the nature of a problem. They tend to be slow and complicated to set up. Even testing a relatively simple back office application would require to set up an up to date test database, provide datasets for every possible scenario, provide a way to test sending out emails and their contents.

Software development testing is a vast topic that deserves more attention from the the ever growing industry. Automated software testing is often times disregarded as boring, time consuming or ineffective. Mainly because of lack of knowledge and skill to make it work for you and not against you.

acceptance, integration, software, Testing, unit Hide

Recently I’ve set myself of a new journey since I’ve decided to help my friend’s business to  battle the crisis back home by creating them a new website. It’s a bit ironic but I didn’t know where to start, because at work I usually work with custom made websites which very rarely use a content management system.

The content management system I’ve chosen to use is Drupal - a widely adopted opensource content management system written in PHP. It has a vast community and enormous amounts of modules developed by other people. It took me about a week’s worth of evenings to get to know the system and launch the website. Here are the steps involved to create a Drupal website:

• Install Drupal. The installation was really easy and simple. Put it on your webserver, access the website, follow an easy guide and you’re done.
• Configure Drupal. To a new user Drupal configuration may seem hectic or chaotic at first. It may take a while to get the hang of things. Figuring out how to change website information, setting up menus, changing themes, hiding things that you don’t want to display.
• Pick a theme. It’s generally better to pick an already made theme and modify it to fit your needs. Themes are designed to integrate with Drupal nicely. They will likely look the same on all popular browsers, will be HTML standard compliant, optimized for SEO and may even be optimized usability wise. I found it very easy to pick a theme using theme garden.
• Install modules. Drupal is a modular content management system and comes with a few useful bundled modules it self. One of the strongest Drupal’s key points is that it has a vast community actively developing modules for it. If you ever need to do something on your website most likely there is a module to do it.

More about modules

Drupal has many useful modules such as Blog, Comments, RSS, Forum, Search, Localization, Content categorization. But the true power lies in modules developed by the Drupal community. A few examples:

• CCK. Content construction kit allows you to add custom fields to content nodes.
• Views. One of the most essential modules for Drupal. Alows to change website’s representation in many ways.
• Pathauto. Allows to configure how website’s URL’s are constructed. A very powerful module for anyone interested in SEO.
• Nodewords. Allows to change meta tags. Very useful to provide custom meta descriptions for content pages. Descriptions are important for SEO.
• Page Title. Another useful Drupal SEO module that allows to provide custom page titles.
• Lightbox2. Very nice plugin to display images on the website. Also supports slideshow.
• Wysiwyg. Allows to replace a simple content text editor with a rich text editor of your choice.
• Node Gallery. A nice lightweight image gallery for Drupal. Still in alpha stages but very easy to use and provides lot’s of configuration. Integrates with Lightbox2.
• Backup and Migrate. Creates scheduled website backups in case there’s an emergency.
• And many other modules…

Pros and Cons of Drupal

Drupal has many pros:

• Extremely easy to install on any webserver.
• Has a vast community developing modules and providing technical help.
• Has a huge amount of freely available themes to pick from.
• Is very well adopted and maintained which means that bugs are fixed, security patches are released and new cutting edge features are always on the horizon.
• Drupal is fast. Maybe it’s not the fastest content management system in the world but it certainly is fast. It’s very easy to set Drupal cache settings which give an immediate boost to the website.
• It’s relatively easy to set up a website that is Search Engine Optimized aka SEO.

Like everything in life Drupal has a few cons:

• For new users Drupal may be overwhelming somewhat chaotic and hectic. It’s still very easy to set up a theme and enter content. But you may have to scratch your head for a while how to add localization support to Drupal.
• Drupal is quite old and even it’s actively developed lot’s of it is written in procedural PHP. Which isn’t necessarily a bad thing, but in some way means that it’s not a top cutting edge software modelling masterpiece.
• Even though Drupal has a huge community which develops modules for it some of the modules don’t have very good documentation. More often than not these are the less used ones. It’s not Drupal’s fault but it’s still confusing and somewhat frustrating to try and figure out where and how you can configure some module you’ve just installed.

All in all I’m happy with Drupal and I think it’s an amazing project and I’m giving my thanks to the Drupal community for all the greats things they are doing.

cms, drupal, modules, PHP Hide

Every web developer probably at some point heard something about MVC unless he or she was living in a cave. I definately have heard and read a lot about it. I won’t probably lie too much to say that most people know that MVC is the nowdays defacto design pattern for web applications. Atleast for PHP it is.

If you have ever had interest in design patterns and did some research on them you may know that design patterns may be interpreted and implemented different every time one tries to. And MVC is no exception to this rule. In my own career path I have seen many projects that claim to implement the MVC design pattern. And if it actually doesn’t - it may be called a hybrid of MVC. As ridiculous as it may be I think because of the MVC hype and everyone trying to be able to claim “yes we use MVC” it is one of the most misunderstood patterns of them all. And because of this … There are a LOT and i mean a LOT of articles and blogs and forums trying to explain MVC the way it should be.

And I myself have read a lot of versions of these blogs and articles. And to be honest I couldn’t answer to you for example what a controller should do and should not do. Well ofcourse I know it shouldn’t contain any business logic. If you would try to research that you would probaly find people saying that the controller should initiate the model, do something with the model and pass the result to the view and render it. You can even find some examples..

But to some extent I find it all synthetic and not very realistic. Most examples are of the level of Hello World program. I think the devil is in the details. If you would try to find any sample php mvc applications you probably wouldn’t find much. There are a few very simplistic sample MVC projects but I don’t find that to be an eye opener that goes deep into details.

I think the PHP community needs such an example. I believe Zend Framework is a great start for MVC. But it isn’t enough. It still doesn’t show you how a real life model or controller would look like. What each part of MVC would do and would not. I believe that one good example is better than a thousand words. I feel trully interested to try and find the “Equilibrium” of the famous MVC design pattern. Don’t you?

controllers, design, Model, mvc, Patterns, PHP, views Hide

I’ve been developing web applications for quite a while now. And overs years it became as a form of art for me. Creating maintainable, performant, scalable, secure and highly available web applications trully requires a lot of knowledge and experience.

It’s easy to say some applications are better than others. Like paintings or music. Some are masterpieces and some are just a pile of crap. Maybe it’s just me but I think that it’s easy to learn what *good* music is, who the *greatest* painters are and see their work and learn something. So I dare to ask the question. Is it the same with web development?

And if you think what i’m thinking you are thinking then … Yes there are open source web projects! And yes some of them are great! But what kind of projects are these projects really?

There’s phpBB. The famous php forum. It’s crap inside. And there’s phpMyAdmin .. it’s also a view of art that you want to wash your eyes with a bleech after. Even though I use it everyday. OK I’m choosing bad examples here. Lets take … WordPress? Well it’s better .. But really? It’s still not a master piece. You can learn more than from others but still not much. And then there are content management systems like Drupal and others which also creeps me out. And finally iI see frameworks. I personally love the ZendFramework. I think it’s one of the greatest things that happened to PHP in recent years. But it’s a framework! You can’t find there any working models, controllers, web services and such. Ofcourse you can find a lot of *other stuff * there  to learn from (like how to create libraries, components or frameworks).

Yes there are great books, blogs, conferences, your company projects. It adds up. But I want to find out who is Pablo Picasso in PHP. I want to see his work and say to myself .. “God! He’s a genius! I love it! It’s so simple and elegant. Why didn’t I do it the same way before?” I think it is healthy for one to evaluate himself among the best and see how far the road he is.

What do you think?

drupal, open-source, oss, PHP, wordpress Hide

In my previous blog post I wrote that me and my friend probably developed a first working google page rank check php implementation on linux. Seems I was wrong. Jan Bogutzki has an implementation on his functions-online.com website that also works on linux. He sent me his version of implementation and I must admit it looks cleaner and more simple than ours. You can download the copy he sent me if you are after a better approach.

free, google, pagerank, PHP, Tools Hide

If you found this blog post on google while searching for a php google page rank class implementation that works on non windows machines then it is your lucky day! You found it! Congratulations!

To my knowledge this is the first available php google page rank retrieval implementation that works on any platform. We have spent hours searching for such a thing online but we couldn’t find it. There are some php google pagerank tools online, they all work, but they are all limited to windows machines.

Why is so you might ask? Well originally the google pagerank retrieval algorigthm is not public. But google made a browser plugin that was able to calculate the google pagerank for any website you visit. So some freaky geeks dissasembled that plugin and got their hands on the google page rank calculation implementation.  Then this implementation was ported to various languages such as Javascript, PHP. The google page rank implementation is sort of protected by calculating a “unique” hash of the given URL. And here the MAGIC begins.

To calculate this hash the algorithm overflows 32bit integers on XOR operations. Aaaand.. 32bit XOR overflows work quite differently on windows and linux in PHP! If you overflow a 32bit integer on windows it just truncates the result to 32 left most bits and returns a new integer. SMART! And on linux XOR overflow just returns the MAX INTEGER value. What did we do? Oh.. We created a simple class to simulate windows 32bit XOR operations overflow using the PHP gmp extension. Tadam! We have also cleaned up the code, documented and made it look shiny

You can download PHP Google Pagerank Class and use it at your own will. I hope this will help you. If it did just leave a comment and say thanks because we are such nice guys to help you out

To use the class try:

require_once("GooglePageRank.php");
echo GooglePageRank::get("http://www.yahoo.com");


Happy Programmers Day!

p.s  Google™ search engine and PageRank™ algorithm are the trademarks of Google Inc.

Update: PageRank class relies on the GMP extension which is not always enabled by default. On Linux Ubuntu it comes as a separate package php5-gmp.

free, google, pagerank, PHP, Tools Hide

I was really happy to see that most of the Drupalcon 2008 talks videos were published. There are a few general talks that I would like to highlight.

One talk that really impressed me was “Rasmus Lerdorf keynote - Simple is Hard“. I found it to bee very interesting, exciting, funny and shocking. It opened my eyes to some extent not to overdo with abstractions and mega layers and take a look at simple things like transactions per second. The inclued pecl extension just blew my mind. Sadly there isn’t a DLL compiled for windows. But I’m definitely trying out this extension on our applications to see what kind of a mess we have.

There were also a few by Rasmus about PHP security at the end of the presentation that were really interesting. Few other talks worth mentioning: “Indexes and denormalization: keys to scaling sites with massive content” a nice introduction to indexes and how they work. And most importantly how you can’t have indexes on two tables using open source relational databases and how denormalization helps.

“High availability solutions for MySQL: An Overview and practical demo” this one speaks for itself. If you are interested in replication, clusters and that alike this talk is for you.

drupalcon, lerdorf, PHP, rasmus, talks, videos Hide

“Essential PHP Security” is an introduction to the PHP applications security written by Chris Shiflett. Though you could say that it suits any other language too. Well atleast to some point it does. It is a really short easy read. Developing PHP applications for quite a few years I have never tried to take a look at all the security issues in one place to see if I’m up to date. To say the truth I expected something more but that is why the book name includes a word “essential”. Let’s take a closer look at the book.

Chapter I - Intro

Basically it tells you to avoid the deprecated register globals (that are going down in PHP6), turning on the error_reporting(E_ALL | STRICT), “Defense in Depth”or otherwise adding additional “safeguards” where possible to make it harder for the attacker, “Least Privillege” or otherwise giving only required permissions,  writing simple, elegant code to easier spot security issues.

Chapter II - Forms and URLs

Basic, basic, basic .. Don’t do things like [email protected],  use is_file_uploaded() and move_uploaded_file() for uploads, prevent XSS using htmlentities with ENT_QUOTES and the encoding that you specified in the HTTP response of your application, the CRSF topic is quite old now but quite not taken seriously and a few thoughts not to take HTTP_REFERER too seriously.

Chapter III - Databases and SQL

Keep your db credentials outside of document root or atleast not visible from outside, better yet in server environment variables,  avoid SQL injections using database specific escape functions, prepared statements or addslashes() and encrypt stuff like SSN or credit card numbers.

Chapther IV - Sessions and Cookies

Things get more interesting here. Cookie theft or it’s just a subtopic of XSS, session fixation is a topic like CRSF - known for a long time and not taken seriously. For those of you that don’t know session fixation is about regenerating your session id every time a user in your application gets higher permissions like logging in. Also an old trick to check whether user’s USER_AGENT suddenly changed on every page load to make it harder for the attacker.

Chapter V - Includes

This chapter explains to keep as much as possible of your application outside of document root, filtering every user given variable that you use for includes with basename, realpath, pathinfo or any other tools. Also  disabling allow_url_fopen though I prefer disabling allow_url_include more.

Chapter VI - Files and Commands

Almost the same as Chapter V. Except that you learn new functions like escapeshellcmd() and escapeshellarg().

Chapter VII - Authentication and Authorization

A nice idea to use 15 second login timeouts to make it harder for the attackers while keeping the legitimate users safe. Use SSL.  Try to avoid persistent logins. And if you can’t don’t store sensitive information in the cookies like usernames or passwords. A really nice idea to generate one-time-use tokens that you save in cookies with a timeout for persistent logins and regenerate them every login.

Chapter VIII - Shared Hosting

It’s best to avoid PHP shared hosting. The book was written in 2005 when PHP had a lot more security issues. Most good hostings have virtual machines or have open_basedir, safe_mode enabled with lots of stuff disabled. Safe mode will be removed in PHP 6 but it has a really nice feature like allowing to run executables from a certain directory.

Apendix A

Not too interesting. Configuration directives. Disable allow_url_fopen, use disable_functions, turn display_errors off, disable enable_dl, set error_reporting E_ALL | E_STRICT, log errors, turn off magic_quotes_gpc, set memory limit 8mb if you can, use open_basedir, turn of register_globals, use safe mode.

Apendix B

Avoid eval, avoid functions that allows to execute commands on the server, use includes carefuly, be careful when using preg_replace modifier /e.

Apendix C

Nice idea to use salt when generating password hashes to make rainbow tables ineffective.

All in all it’s a very good book to get started. Maybe not as detailed as the latests Steffan Essers recent findings but if only every site on the web would do the things described in the various book chapters… Projects like Google Hacking Database show how secure the web really is. And for the last words .. It’s not the PHP that is not secure .. Enormous amount of effort was put into PHP to make PHP more secure.  Just to give you an idea listen to “The state of PHP security” or read about what Steffan Esser is doing. Yes there still are security bugs in PHP .. But honestly like Steve Balmer would say it’s all about: Developers, developers, developers. PHP deserves a better name.

book, PHP, review, Security Hide

Did you know that most widespread encryption algorithm in electronic commerce is RSA? For two computers that have never communicated before to start a secure conversation RSA uses a public key and a private key.

For example if Tom wanted to send a secure message to Suzi he would encode his message with Suzi’s public key and send it to her. The message can only be decoded using Suzi’s secret private key that only she knows.

Interesting thing is that a private key is made of two large distinct random prime numbers. And a public key is the product of those two numbers. So RSA is only as safe as it’s hard to find those two prime numbers using the public key to get the private key. But more interestingly it turns out to be almost impossible. There is no such efficient algorithm on earth to find those two prime numbers using a public key. It would take years to decrypt a single number. You can actually earn some money trying to decrypt RSA messages.

RSA key’s are typically 1024, 2048 bits long. A shorter 256 bit key can easily be decrypted using a home computer. A 512 key can be decrypted with a Computer Cluster. Even 1024 is not considered really safe anymore. Even more Shor’s algorithm proves that a RSA key could be decrypted as fast as a computer can do a quick sort. Though that would require us to have a quantum computer. It’s a theoretical device by now but who knows what the future holds?

There are other security attacks known, but they are way too much complex for a simple presentation like this. So..

Shop safely!

algorithm, RSA, Security Hide

Generally I dislike most PHP frameworks. Especially after the Ruby on Rails fashion wave after which a lot of PHP frameworks have been born with ActiveRecord implementations, scaffolding and so on. It makes you creep after watching yet another Blog development screencast in 15mins “using the best new framework”!

Not so long ago Zend Framework released a stable release and today I’ve spent my time at work watching screencasts and webinars to take a closer look at it.

And I LOVED IT Here’s why:

• Whole framework is just one directory of classes that you don’t ever care about with no predefined application structure and design by default.
• It stands behind a respectable and well known company ZEND that has a lot of to do with PHP.
• It has an outstanding team of professional contributors with a lot of brain power and experience.
• Lots of well developed components following actual design patterns.
• 80% > unit test coverage
• A useful documentation that most others are missing.
• MVC components, controller plugins, helpers.
• A possibility to develop a modular application where each module has it’s own views, controllers, models that can easily be removed by anyone in seconds while other frameworks force me to split MVC components to other directories and making it hard to separate from hundreds of others.
• It does not have ActiveRecord and Scaffolding

Though there are some issues people might spot:

• Zend_Controller_Helper_Abstract. Class names lack namespaces. Though there were no namespaces available while Zend Framework has been actively developed and does it really matter?
• You need to understand a few design patterns and OOP to use full Zend framework’s potential. And that does not only requires time spent learning but also experience working with it.
• It is big, it is heavy, it eats memory and it has a lot of includes. Though I usually tend to agree with the “hardware is cheaper than development” theory ..

All in all I find Zend Framework the best PHP framework there is. In the past I looked at CakePHP, symfony, code ignite. I’m going to give Zend Framework a try at work and implement a not so very important project with it. If it works out well I might start migrating all our frameworkless applications to it.

framework, mvc, PHP, review, zend Hide