Integrating Stoyans 3PO ruleset to YSlow Commandline

Stoyan Article :

Stoyan Yslow Fork :


Setting up HttpArchive private instance

Create mysql instance for httparchive using the following schema

Download the source code from

  • This source code contains the two components
    1. UI components : Files under checked out home directory. Responsible for showing the HttpArchive WebApplication.
    2. Batch processing components : Files under bulktest folder under home directory. Responsible for submitting the URLs to WPT and get the details like HAR, page load time, getting other stats and trends.

Move the checked out directory into webserver folder

Customize the code for your instance

make the changes into following files in the checked out code

File :

  • Update WebPageTest(WPT) Instance URL. If you have a private instance set this to private URL else set to
  • Update Mysql DB Name, username, password
  • Update HttpArchive WebApp URLs of local instance
  • Update HttpArchive filesystem path
  • Update PrivateInstance  to True

Optional : If you have a private instance of, you can change the default value of IE8 to your own location preference for WPT private instance in all the files in the source code.

File : bulktest/

  • Update Locations(in lieu of IE8) and WPT Key

Once the setup is complete. You can see the application running in the following URL

HomePage : http://localhost/httparchive/index.php

Adding URL Page : http://localhost/httparchive/addsite.php

Approving the URL for crawling : http://localhost/httparchive/admin.php

The crawling of the URL happens in the background as the batch process through the following scripts

  • run “php batch_start.php” to kick off a new batch testing.
  • run “php batch_process.php” repeatedly to perform a single batch testing.
  • run “php statscompute.php” to generate the stats after the batch_process is complet

Basic documentation for batch process available in the code Readme :

I have setup the following crons jobs on my instance

  1. Run batch_start.php every 00:00 hrs
    • crontab 0 0 * * * user httparchive/bulktest/batch_start.php > /home/user/logs/yhttparchive/batch_start_out_err.txt 2>&1
  2. Run batch_process.php every hour except 00:00hrs
    • crontab 0 1,2,3,4,5,6,7,8,9,10,11,12 * * * user httparchive/bulktest/batch_process.php > /home/user/logs/yhttparchive/batch_process_out_err.txt 2>&1
  3. run php statscompute.php to populate the stats table
    • crontab 0 14 * * * user httparchive/bulktest/statscompute.php > /home/user/logs/yhttparchive/stats_compute_out_err.txt 2>&1

Database Tables Explaination:

  • urlschange : http://localhost/httparchive/addsite.php –> Newly added URLs go into ‘urlschange’ table
  • urls : http://localhost/httparchive/admin.php –> Admin will approve these newly-added-urls and then these get into ‘urls’ table
  • status : batch_start.php will create the entries into ‘status’ table by reading from ‘urls’ table
  • pages : batch_process.php will create the entries into ‘pages’ table with data pertaining to pageload time, pagespeed rank, total requests etc.. This table data is used to display in ‘viewsite.php’
  • requests : batch_process.php will create the entries into this table with data pertaining to all the requests generated for loading the page.(like HAR content).
  • stats : statscompute.php will create the entries into this table. This table data is used in ‘trends.php’ page
  • There are other tables which precede with mobile. These are for mobile URLs

Importance of webpage performance

Refs :

Setting up webpagetest private instance

PhantomJS – headless webpage testing

PhantomJS ( is a headless WebKit scriptable with JavaScript or CoffeeScript.

PhantomJS is the closest to real browser.

PhantomJS can be used to generate HAR from URL by commandline without requirement of a browser.

Reference Links :

  1. Google Code Page describing the usecases
  2. Quick Start tutorial page :
  3. Build Instructions from Source :
  4. ReadyMade binary files for different OS :
  5. Binary I have tested on RHEL-6 :
  6. Nagios using PhantomJS page loadtime : 

Example Commandline for generating HAR file from URL

  • bin/phantomjs examples/netsniff.js > movies.har

Example Commandline for generating PageLoad time from URL

  • bin/phantomjs examples/loadspeed.js
    • Page title is Yahoo! Movies - Upcoming Movies, Trailers and Showtimes
      Loading time 3143 msec

Cons : Flash might not work(!)

W3C mobileOK Checker — URL Mobile Friendliness check

This checker performs various tests on a Web Page to determine its level of mobile-friendliness. It gives the result in a rating from 0 to 100.

Checker Tool URL

Sample example of test result

More Documentation

Main Documentation Page :

Usage Documentation :

Jar File :