Wearing a White Hat

Earlier this summer I was browsing for some recipes on findusfoodservices.fi with my iPad when I encountered this error:

Front page of findusfoodservices.fi with iPad

1st Sin: Site is unusable with iPad
2nd Sin: Site shows user the error-message

A PHP-fatal error on public website - interesting... And that path to file originating the error looks a lot like public web-root... Had to check it out. 

Typing the php-files path to url shows white page - not a 404 or 403 but plain 200-page. Something just got executed but it did not output anything. Next logical step was to check what the folder looked like. And sure enough Apache told me that - including it's version number and operating system it was running on - a black hatted person could use that information.

3rd Sin: file structure of site is browsable
4th Sin: Apache reveals quite vital information about the environment

When I had browsed trough the public folders that were under findusfoodservices.fi/plugins/mobile I decided to "misuse the robots.txt" as my co-worker put it.

Now we where talking - by telling the robots where not to look they were telling me exactly where to look.

Only a few folders were interesting, /administration told me that this was a Joomla-site (and apparently they do their security quite well). By "randomly" trying I found out that they had PHPMyAdmin running at /phpmyadmin - and that is not good.

5th sin: PHPMyAdmin on production server

But the best (worst) is still coming. There were folder /tmp...

Content of /tmp

What are these I wonder? Well, let's find out. And it turns out they had user information on them - some sort of database dumps maybe. Those files - on public folder on public site - contained names, titles, phonenumbers, addresses and emails. Not that many, but still 399 unique entries. That's a no-no.

6th Sin: Confidential data publicly available

What an opportunity those information would be for social engineering! Not just for findusfoodservices.fi but for who knows how many other systems. As we all know people tend to use same passwords at multiple services.

For long I considered if I should contact Findus or the media about my findings. I decided to throw the ball to them. I emailed all my findings to Findus and gave them a week to fix the issues or else I would tell the media about them.

And they did fix them. They even sent me an email to thank me.

They've fixed all my findings - well almost all. I still get the PHP-fatal that led me to these findings when I go to the site with iPad. I guess I cannot have everything. But still - it feels good to know I have had small part in making web tiny bit safer.

Days Without blogging: 0

My latest project "Days Without" went online more than a month ago.

Update 2013-12-12: Could not resolve the problem, somewhat related to Symfony2 version, rolled back to MVP version.
Update 2013-12-04: New version of "Days without" is currently broken - Ill try to fix it during the weekend.

Days Without is a day counter for ... anything, really. The idea came from need to have a counter for how many days I've been without sweets. But why stop there? I could make counters "on demand" - in fact anyone could make counters.

Original idea was to have two buttons at counter, "+1" and "Reset". The Plus-button would increase the counter by one day and Reset would reset it to zero. After presenting my idea to coworkers I realized that having to push the +1 button every day would be a pain in the ass. Just as easily (if not even easier) I could make the counter "automatic" - so I did.

Minimum Viable Product

I wanted to get Days Without out as fast as possible. So I made list of what would be the MVP.

• Counters could be made
• Counters would count the days since last reset (or creation)
• Counters could be reseted
• URL's to counters would be human readable and easy to remember

The next steps would be

• Password-protected counters
• Counter listing (newest, last reseted etc)
• List of users counters
• Sharing to Facebook & Twitter
• Commenting on reset
• Reset history of counter (date, day-count, comment ...)

The MVP is done, password protected counters are almost. The app is still quite fresh and might (probably will) lose it's data at some point. Do not be suprised if that happens.

Codes can be downloaded and contibuted at GitHub and the site is live at Feikki.fi/dayswithout

All feedback is more than welcome. Comment below or tweet me @JSomerstone #DaysWithout

Coming up next: "Wearing a White hat"

Behaviour testing with PHP

What it I told you that you can write tests for your application in plain english? Not a single semicolon or brace required? Well, it is possible - and not entirely true. I'll get back to that.

Behat is a (or rather the) BDD (Behavior-driven development)  framework for PHP. You describe the behavior of the application with simple "Given - When - Then" steps and there is your test. And when you got multiple tests, you have specifications of how the application is supposed to work - all in human readable format.

An example test case

Feature: Login functionality allows login only from user with correct username/password 

    In order to use all sites features 

    As a user 

    I need to be able to log in

Scenario: User is able to log in 

    Given system has a user "BehatTester" that has password "correctpassword"

    When user logs in with user name "BehatTester" and password "correctpassword"

    Then user is redirected to "/welcome" And message "Welcome BehatTester" is shown

Is that readable (and writable) or what? People who write tests no longer have to have any programming experience.

• Feature-statement is used to describe one feature of a program and Scenario-statement is describes one use case of that feature.
• Suites have multiple Features that have multiple Scenarios that have multiple Given-When-Then -steps.
• Given-statements normally change the status of the program, in the example scenario the given step would save a user "BehatTester" to system and give it password "correctpassword".
• When-statemets simulates actions made by user (or another actor), for example form submission, opening a page etc.
• Then-statements verify that expected things happen, in similar way that in PHPUnit uses "assert"-functions.
• Multiple Given, When and Then -statements can be given simply with "And"

Peek under the hood

Unfortunately the scenarios are not enough to get the tests to work. And this is where comes the actual programming of tests. When a Behat-test is executed, Behat parses the scenarios and their steps and tries to find matching function from "Feature"-classes that are written in PHP. Then Behat executes those functions in the order specified in the scenario.

In order to setup and teardown the state of the system under test Behat implements hooks. They're Feature-classes methods that are marked with "@EventName" comments. A very useful hook is "@BeforeFeature" - the function marked with this tag is executed before the first step of every feature. It has a counter-part "@AfterFeature"-tag.

Behat has a very useful feature: if no matching function is found for step, then Behat prints a "code-snippet" that you can just copy+paste into your Feature-class and then implement. This speeds up the test-writing process nicely.

Behat-tests do not remove the need for Unit-tests. But they are very useful as acceptance-level tests.

I'll write a more in-depth blog about using Behat. In the meantime, if interested, do visit Behat Docs.

Sideproject(s)

Busy, busy, busy

The Cimbic has been on hold for a while. I've been "too busy" doing other stuff - like working and watching Netflix. But I haven't been completely idle with my hobby. I've made a little library with handy PHP classes.

jaxToolbox - library

It all started with need to make cURL-requests from script. Sure, there were the PHP-functions, but that produced ugly code. I wanted a wrapper around them and so the first version of jaxToolbox was born.

Curlifier

Object oriented wrapper for making cURL-requests and receiving the response. Curlifier supports method chaining, post- and get-requests, cookies and custom user-agents. It also provides interface to validate responses header and body against given regexpes.

$curl = new \jaxToolbox\lib\Curlifier();
$curl->request(array('url' => 'http://localhost/');
$restponseBody = $curl->getResponseBody();
$hasBob = $curl->bodyMatchesExpression('/Bob/i');

User doesn't have to know any CURL_OPT_* constants to modify requests, all (that I needed) have been wrapped to neat and clear methods. See the Wiki for more complete API.

$curl
    ->setUrl('http://localhost')
    ->setGet(array('foo' => 'bar'))
    ->setPost(array('name' => 'Bart'))
    ->setCookie('nameOfCookie' => 'VALUE_OF_COOKIE')
    ->request();

Curlifier gives the user the liberty of choosing to use various setters or passing all settings as parameter for the request()-method. Settings set via setters are permanent and are present with all request()'s where the parameters given to request() will affect only that single request.

$curl->setGet(array('one' => '1'))
    ->request();
//Is the same (for this request) as:
$curl->request(array(
    'get' => array('one' => '1')
)); 

The power of the two different approaches comes when you need to perform multiple requests:

$curl->setUrl('http://localhost/pageApp.php')
    ->setGet(array('query' => 'Bob%')); //will be used with all requests
for ($i = 0; $i < 10 ; $i++)
{
    $curl->request(array(
        'get' => array( 'page' => $i ) //localhost/pageApp.php?query=Bob%&page=$i
    ));
 
    if ($curl->getHttpCode() !== 200)
        break;
}

Curlifier is not complete. Features on To-do-list include:

• Support for multiple Cookies
• Support for "cookie jar"
• Auto-parsing responses "Set-cookie" -headers and setting cookie accordingly
• Returning PHP-array from JSON response
• Returning PHP-object/array from XML response

StringEnumerator

Class that can be used to iterate trough all possible enumerations of given character-pool and string length. (Brute forcing with PHP, anyone?)

Character pool is given simply as string like "abcd1234", all generated strings will contain only those characters. Length of strings is given as integer and all generated strings will be that long.

$permutator = \jaxToolbox\lib\StringPermutator('abcdefghijklmnopqrstuvwxyz', 4);
while ($permutator->hasNext())
{
    echo $permutator->getNext(), "\n";
}

Code above will output:

aaaa
baaa
caaa
daaa
…
zzzx
zzzy
zzzz

On to-do-list is:

• Support for multibyte-characters
• Version that iterates anagrams of given string

ConsoleRequest

PHP supports also console-scripts but in my opinion the way it takes the arguments is - hard. To ease creating console-applications I made ConsoleRequest. It takes the arguments from the $argv-global and serves them as associative array or through getter.

Parameters are passed to script like arguments (or flags) in linux console applications.

Examples:

"-u root -p password" -> array( 'u' => 'root', 'p' =>'password' )
"-qwe" -> array ( 'q' => true, 'w' => true, 'e' => true )
"-ap 2" -> array( 'a' => true, 'p' => '2' )
"--help" -> array( 'help' => true )
"--output result.txt" -> array( 'output' => 'result.txt' )

Or any combination of those.

<?php
$request = \jaxToolbox\lib\ConsoleRequest($argv);
echo $request->get('hello'); //if parameter "--hello" is given, will echo it's value
var_dump($request->getArguments()); //will output all arguments passed to script

Update 2013-02-15

Curlifier now has support for multi-cookie and JSON/XML response parsing.

Building secure web application - my two pennies

I recently ran into discussion about PHP application security. The main question was:

Can anyone tell what security issues should I consider to make it (a PHP+MySQL+Ext JS application) a secure application.

My first tough was to send a link to Owasp TOP-10. But that would not be enough - If one needs to ask such broad question on Linked In then he/she is not ready to make a secure web application. Making a web application secure is not a topic that can be inclusively answered with a single forum- (nor blog)-post. There are quite meny books written on the subject.

But never the less i'll try to give my contribution on the matter and write down few (as simple as possible) rules that anyone even considering making a web application should obey.

Treat every request as it came from your worst enemy

... and it is trying to break your application. Validate every single input of the request before using it anywhere. If possible, build your software in a way that you cannot bypass the validation in your code even if you wanted to. If some input needs to accept pretty much everything then make a rule that says "pretty much everything allowed" - this shows that you have even thought about input sanitation. Finding the right balance between too tight and loose validation rules is the hard part. Owasp TOP-1 - Injection.

Escape every single output

... as close to the actual outputting as possible. It doesn't really matter where the input came from - GET-parameter, a form the user just posted, database or some exotic setting file. It might have been compromised and if it has, you do not want output it in a way that it would be executed in users browser. Because that is the way to Owasp TOP-2 - Cross-Site Scripting.

Do not let anyone access any restricted data

... unless they're specifically allowed to do so. Whenever anyone asks for any non-public data, check that the user has rights to access it. And if not, the safest way to handle the situation is to give a 404. This way you don't tell that the object tried to access actually exists - but the user doesn't have rights to access it. Owasp TOP-4 - Insecure Direct Object References.

Use random tokens to every state-changing request

... to make sure that the request came from your UI - not some malicious site the logged in user happens to be browsing. Simply put: store a random string to session-data and when the application receives request to state-changing action, make sure that it is the same that you have stored in session. The token doesn't need to be unique for every request, but it needs to be unique for that session. When encountering missing or inaccurate token the failed request should be logged as it might be Owasp TOP-5 - Cross-Site Request Forgery. Also reset the session token - cannot brute-force something that is constantly changing, can you?

Do not take any redirect parameters directly from the URL

... because those do not belong there! If you do then anyone can make a link that starts with your domain but lands somewhere totally different - like their phishing site that looks exactly the same as yours. And *puf* the hackers now have valid credentials to your application. Owasp TOP-10 - Unvalidated Redirects and Forwards.

And by mother of a supernatural intelligence:

Never ever store the passwords in plain text!

I hope that I don't need to tell you why.

Wind of change

Totally dumping current CSS-library - Goodbye Blueprint, welcome Bootsrap! Main reason is possibly Bootsrapts support for responsive desing.

I've changed the CSS-library, that is all about everything I have managed to do. Last month has been quite eventfull. Went to London 2012 paralympics to support Finnish goalball team. The woman came fourth and the men wom gold! Considering how small country Finland is our six medals (4 golds, a silver and a bronze) is good result. But that's enought about sports.

New responsive layout is working great. Bootstrap has great documentation and examples to learn from. The minus side is that I just copy what others have done, do some minor adjustments but have no idea how it actually work. What are the CSS definitions that do the magic? Whitout that knowledge I will always be depending on someone elses code. But for now it will have to do. My spear time is so rare that I have to priorize - and the family comes first.

This post is quite stub, but what can I do when there hasn't been any progress in Cimbic or JSFramework besides the Bootstrap.

Over and out!

Roadmap of Cimbic

Things have moved forward sinse my last blog. Not mutch but still. From a blog by J.Boye I formed the roadmap of Cimbic. Some of the features had already been implemented, others are on the way.
Cimbic roadmap is:

1) Separation of content and presentation - DONE
2) Rich text editing
3) Access control lists 
 - E.g restricting who can view/edit certain areas 
4) Automatic handling of change in navigation / site structure 
5) Short, meaningful, friendly URLs 
 - without file extensions (e.g. .aspx) - DONE 
 - and extremely short URLs 
6) Automatic content import 
 - Eg. RSS / XML feeds 
7) Multi-delivery to different platforms 
 - Including mobile and printer-friendly 
8) Multi-lingual
 - At least western European charset / UTF8
 - Some way to handle sites in multiple languages
9) Broken links checker

Latest improvements to Cimbic are page and site specific settings stored in .json-files. I chose that format because it can present quite complex hierarchy and deconding and encoding is trivial with native php.

Currently I'm implementing point 2 on roadmap - content editing with Create.js. It has quite many dependencies: jQuery, jQuery UI, Underscore, Backbone, VIE, the editor of my choise - hallo.js
One problem I encountered was providing Cimbics static files (CSS/JS/image) when the Public-folder of site not located under Cimbic. Site starts an instance of Cimbic given the physical location of the sites folder on server and then Cimbic would handel the rest. First solution was to refere those files via Controller/method, like StaticFile/css/name/of/file. This worked and I was able to provide static files stored under Cimbic to any site using it and it was quite easy.

But there was one huge but. Providing images refered in 3rd party CSS (and/or JavaScript) is no longer trivial. I do not want to change every reference to images on the CSS/JS files so that I could provide them by Cimbic. And because Cimbics CSS/JS/image files are not responsibility of the Site using Cimbic and therefore those files shouldn't be stored under Site. But the Public folder IS under Site and there's the problem. I don't want to change the hiearchy so that Public folder is under Cimbic and it determinates what Site to present.

My solution was to make install-script that copies or creates symbolic link(s) from Sites public folder to Cimbics relevant folder(s). This way Cimbic would hold it own CSS/JS/image files but they can be refered from Sites public folder. Future step would be to create new blank site with same or similar script. I haven't actually done the script yet - haven't even desided between shell- and php-script.

I also have given step 7 of roadmap some thought. Being quite active iPad user I think that the Sites using Cimbic should use responsive layout to support mobile devices. So I'm studying the matter from Net Magazine and Design Shack. The goal is to provide responsive layout for Sites by with just few class definitions to template divs. We'll see how that goes…

'Till the next time: Keep on coding!