Saturday, July 5, 2014

CLIFramework - A command-line application framework for PHP and the difference between symfony/console

If you're using Symfony/Console component as your command-line application base library, you should already know the API design of Symfony/Console is pretty cumbersome, you need to write more lines of code to define an argument or an option.

Thus, you might want to know a simpler / powerful comand-line application framework, CLIFramework is designed for simplicity / extensibility.

The main difference between CLIFramework and Symfony/Console is that:

1. Fluent-style for command argument definition and option definition.

2. Option spec with concise expression.

3. Powerful ZSH completion generator. (With Symfony/Console, you have to write zsh completion script for each application.. "by hands!")

The zsh completion generator can convert your hierarchical commands / options into a zsh completion script.

Let's see the power of zsh completion generator, the following completion demo is used in phpbrew. The completion script is auto-generated by CLIFramework:





Tuesday, June 10, 2014

Perl URL Router Benchmarks - The URL Router for C100K

To improve the overhead of URL routing, I developed a tiny URL routing library in C - R3, which is designed for high performance and low memory usage.

The implementation uses the trie data structure. Trie is an efficient information retrieval data structure, which uses a prefix tree to search inserted strings by their common prefix, thus it's pretty fast for matching routes.

The tricky part of R3 is that we need to support regular expression patterns in our routing definition to dispatch path dynamically, thus R3 implemented a variant of Trie by mixing some concepts of DFA, not just simply inserting plain string paths into the prefix tree.

There are several comparison types in R3: plain string, opcode or regular expression pattern. Each node have its own comparison type to dispatch the path to their own children.

For those simple regular expression patterns, R3 compiles the regular expressions into opcodes to enhance the comparison speed and it's twice faster than using PCRE library to match the path with the patterns.

And since there are a lot of things to do before the matching operation, the whole tree structure needs to be compiled before dispatching paths.

Here is the continuous benchmark result of R3 itself, The C version R3 can dispatch over 11 million plain string paths per second.

A talented Perl hacker - Cindy Wang developed a CPAN module based on R3 library - Router::R3, which has become the fastest routing module on CPAN. It's so fast that you just can't deny.

The Perl version R3 (Router::R3) can dispatch nearly 1 million static paths per second while Journey (The Rails router) dispatches 9.9 thousand static paths per second. and it's 466% faster than the second fastest module (Router::Room) in static path dispatching, 751% faster in first character matching.

We also used the routing path generator from the rails' pull request stevegraham/rails/pull/1 to benchmark the Perl URL routers on CPAN, including HTTP::Router, Router::Simple, Router::Boom and Router::R3

There are over 3 hundred generated routing paths in the benchmark script. we tested the "static path matching" with the route in the middle of the list, and also the first route matching, regular expression matching.

Here comes the dispatching speed result:

Benchmarking 'plain string matching' by path '/corge/quux/bar'
===============================================================
                Rate  HTTP::Router Router::Simple  Router::Boom    Router::R3
HTTP::Router      203/s            --           -89%         -100%         -100%
Router::Simple   1782/s          779%             --          -99%         -100%
Router::Boom   168658/s        83094%          9365%            --          -82%
Router::R3     954407/s       470684%         53461%          466%            --
Benchmarking 'regexp string matching' by path '/post/2012/03'
===============================================================
                Rate  HTTP::Router Router::Simple  Router::Boom    Router::R3
HTTP::Router     1076/s            --           -88%          -99%         -100%
Router::Simple   9309/s          765%             --          -91%          -97%
Router::Boom   104387/s         9602%          1021%            --          -66%
Router::R3     306925/s        28426%          3197%          194%            --
Benchmarking 'first character matching' by path '/'
===============================================================
                    Rate  HTTP::Router Router::Simple Router::Boom    Router::R3
HTTP::Router      3839/s            --           -87%         -98%         -100%
Router::Simple   30545/s          696%             --         -83%          -98%
Router::Boom    180555/s         4603%           491%           --          -88%
Router::R3     1535999/s        39910%          4929%         751%            --

Wednesday, March 20, 2013

An Efficient Way Of Developing PHP Extension

Many people think developing PHP extension is really hard and too complex for them, so they rarely consider developing PHP extensions by themselves.

Here we want to show you an efficient, easier way of developing PHP extension that might make you reconsider about it. :-)

Creating Extension Skeleton

To make a basic extension works, you need 3 files to build your extension.
  • config.m4
  • your_ext.c
  • your_ext.h
Yes, only three files you need to create. But to write the C header files and the .c file, there are still many things to understand and to be configured well.

With GenPHP http://github.com/c9s/GenPHP you don't need to write these files by yourself. By using GenPHP, you can simply run a single command to generate these files through a simple template engine.

To get GenPHP works, first you need to install it:
$ curl -O https://raw.github.com/c9s/GenPHP/master/genphp
$ chmod +x genphp
$ mv genphp /usr/bin/
Then, to generate something you need to install the GenPHP flavours:
    $ git clone https://github.com/c9s/GenPHP-Flavors ~/.genphp/flavors
Now run genphp list, you should see the flavour list:
    $ genphp list
    Available flavors:
        extension   /Users/c9s/.genphp/flavors
OK, now we've got the extension flavor, to use it, simply type:
    $ mkdir app
    $ cd app
    $ genphp new extension app
    Loading flavor extension...
    Running main generator...
        render        config.m4
        render        php_app.h
        render        php_app.c
        copy          .gitignore
    Done
See? it's really easy.
The "app" above is your extension name, you may just replace the "app" with anything you want.
To build the basic extension, you can type the commands to build the extension as usual:
    $ phpize
    $ ./configure
    $ make
    $ make install
The extension flavor creates a basic extension for you, which contains a simple PHP function, see below:
    PHP_FUNCTION(app_test)
    {
        RETURN_STRING("Hello World", 1);
    }
Now you can simply start here with ease!

Testing

To test your extension code, there is already a tool called "run-tests.sh", which is shipped from PHP distribution.
If you ever used it, you should know it's really hard to do the tests and use gdb to debug.

The Old Way

To debug your php extension, you need setup these configuration by yourself:

You need to create your own .gdbinit with the content below:

file /Users/c9s/.phpbrew/php/php-5.4.12/bin/php set args -d 'output_handler=' -d 'open_basedir=' -d 'safe_mode=0' -d 'disable_functions=' -d 'output_buffering=Off' -d 'error_reporting=32767' -d 'display_errors=1' -d 'display_startup_errors=1' -d 'log_errors=0' -d 'html_errors=0' -d 'track_errors=1' -d 'report_memleaks=1' -d 'report_zend_debug=0' -d 'docref_root=' -d 'docref_ext=.html' -d 'error_prepend_string=' -d 'error_append_string=' -d 'auto_prepend_file=' -d 'auto_append_file=' -d 'magic_quotes_runtime=0' -d 'ignore_repeated_errors=0' -d 'precision=14' -d 'memory_limit=128M' -d 'extension_dir=/Users/c9s/src/work/php/fileutil/modules/' -d 'session.auto_start=0' -d 'zlib.output_compression=Off' -d 'mbstring.func_overload=0' -d 'extension=fileutil.so' -n {your PHP script}

Now you should understand the hard place.

2. The phpt does not show the expected result and actual result very clearly, please see the actual output below:
=====================================================================
PHP         : /Users/c9s/.phpbrew/php/php-5.4.12/bin/php
PHP_SAPI    : cli
PHP_VERSION : 5.4.12
ZEND_VERSION: 2.4.0
PHP_OS      : Darwin - Darwin c9smba.local 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan  6 22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64
INI actual  : /Users/c9s/.phpbrew/build/php-5.4.12/ext/tidy/tmp-php.ini
More .INIs  :
CWD         : /Users/c9s/.phpbrew/build/php-5.4.12/ext/tidy
Extra dirs  :
VALGRIND    : Not used
=====================================================================
TIME START 2013-03-20 15:54:16
=====================================================================
PASS Check for tidy presence [tests/001.phpt]
PASS tidy_parse_string() [tests/002.phpt]
PASS tidy_clean_repair() [tests/003.phpt]
FAIL tidy_diagnose() [tests/004.phpt]
PASS tidy_parse_file() [tests/005.phpt]
FAIL Verbose tidy_get_error_buffer() [tests/006.phpt]
PASS Verbose  tidy_getopt() [tests/007.phpt]
FAIL Accessing the error buffer via $obj->error_buf... [tests/008.phpt]
PASS tidy_doc object overloading [tests/009.phpt]
FAIL Accessing root, body, html, and head nodes.. [tests/010.phpt]
PASS Accessing attributes of a node [tests/011.phpt]
FAIL Accessing children nodes [tests/012.phpt]
PASS Parsing a file using constructor [tests/013.phpt]
PASS Passing configuration options through tidy_parse_string(). [tests/014.phpt]
PASS Passing configuration options through tidy_parse_file(). [tests/015.phpt]
PASS Passing configuration file through tidy_parse_file() (may fail with buggy libtidy) [tests/016.phpt]
PASS The Tidy Output Buffer Filter [tests/017.phpt]
PASS binary safety [tests/018.phpt]
PASS tidy_repair_*() and invalid parameters [tests/019.phpt]
PASS OO API [tests/020.phpt]
PASS tidy_get_opt_doc() [tests/021.phpt]
PASS tidy_repair_*() and invalid parameters [tests/022.phpt]
PASS tidy and tidyNode OO [tests/023.phpt]
SKIP libtidy handling of 'new-blocklevel-tags' [tests/024.phpt] reason: old libtidy
PASS tidyNode tests [tests/025.phpt]
PASS tidy.clean_output test [tests/026.phpt]
PASS Bug: tidy segfaults with markup=false [tests/027.phpt]
PASS tidyNode::getParent() [tests/028.phpt]
PASS tidy_get_body() crash [tests/029.phpt]
PASS getConfig() method - basic test for getConfig() [tests/030.phpt]
PASS tidy_config_count() function - basic test for tidy_config_count() [tests/031.phpt]
PASS tidy_error_count() function - basic test for tidy_error_count() [tests/032.phpt]
PASS tidy_warning_count() function - basic test for tidy_warning_count() [tests/033.phpt]
PASS tidy_access_count() function - basic test for tidy_access_count() [tests/034.phpt]
PASS tidyNode::__construct() [tests/035.phpt]
PASS Tidy::diagnose() NULL pointer dereference [tests/bug54682.phpt]
PASS Bug #50558 - Broken object model when extending tidy [tests/bug_50558.phpt]
PASS Ensure tidy_get_status() returns correct status [tests/tidy_error.phpt]
PASS Notice triggered by invalid configuration options [tests/tidy_error1.phpt]
=====================================================================
TIME END 2013-03-20 15:54:19

=====================================================================
TEST RESULT SUMMARY
---------------------------------------------------------------------
Exts skipped    :    0
Exts tested     :   49
---------------------------------------------------------------------

Number of tests :   39                38
Tests skipped   :    1 (  2.6%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    5 ( 12.8%) ( 13.2%)
Expected fail   :    0 (  0.0%) (  0.0%)
Tests passed    :   33 ( 84.6%) ( 86.8%)
---------------------------------------------------------------------
Time taken      :    3 seconds
=====================================================================

=====================================================================
FAILED TEST SUMMARY
---------------------------------------------------------------------
tidy_diagnose() [tests/004.phpt]
Verbose tidy_get_error_buffer() [tests/006.phpt]
Accessing the error buffer via $obj->error_buf... [tests/008.phpt]
Accessing root, body, html, and head nodes.. [tests/010.phpt]
Accessing children nodes [tests/012.phpt]
=====================================================================

You may have found a problem in PHP.
This report can be automatically sent to the PHP QA team at
http://qa.php.net/reports and http://news.php.net/php.qa.reports
This gives us a better understanding of PHP's behavior.
If you don't want to send the report immediately you can choose
option "s" to save it.  You can then email it to qa-reports@lists.php.net later.
Do you want to send this report now? [Yns]:

As you seen, it only shows "FAIL" and "PASS" and it's asking you whether to send the test report.. it's really excruciating..

3. If you have both pure PHP implementation and the C extension implementation, it's hard to switch and test them.

ExtUnit

We developed ExtUnit to improve the extension unit test, you can even write your extension in PHPUnit, and with ExtUnit, you can easily run the tests with your pure PHP implementation or with your extension implementation by two simple commands.
Install ExtUnit:
    $ pear channel-discover pear.corneltek.com
    $ pear install -f -a corneltek/ExtUnit
To let ExtUnit work, you need to define the extunit.xml config file:
<?xml version="1.0" encoding="UTF-8"?>
<extunit>
    <!-- local extensions for php to load -->
    <extension>fileutil</extension>
</extunit>
Then you can simply type the command to phpunit tests:
    $ extunit --phpunit
Or use gdb to debug the test cases:
    $ extunit --gdb --phpunit
Or simply run a php script with gdb:
    $ extunit --gdb example.php
ExtUnit runs "make" before you run the tests, so you can simply run single command to test, which improves the extension development iteration.
Now you might want to test your pure PHP implementation, just type the phpunit as usual:
    $ phpunit
Done!

Monday, February 18, 2013

Golang: the performance of Goroutines

Found this article about the performance of Goroutines: http://en.munknex.net/2011/12/golang-goroutines-performance.html

The performance is just tremendous awesome, you should really check out this article.

Sunday, February 17, 2013

GoTray 1.4.2 is released

GoTray 1.4.2 is released:
  • "Watch and Test" feature is added.
  • Multiple window to monitor projects, tasks.
  • Support CGO environment variable
  • Project settings editor.

No more talking, here is the screenshot:


You can use GoTray with your favorite editor to do the automatic continuous testing:


Friday, February 15, 2013

GoTray 1.3.3


GoTray 1.3.3 is released, GoTray aims to be a small, faster continuous integration system for your local environment.

Just download it and enjoy the fast iteration with GoTray.

Sunday, February 10, 2013

GoTray for Mac OS



I made this Mac OS App after work today - GoTray, which is an application manages your godoc server process, you can simply click to start a godoc server or stop it.

It will automatically opens your browser to open the local godoc web documentation.

While I decided to upload this Mac OS App to the Mac OS App Store, and I found one thing, which is that developers *MUST* enable the "Entitlement" feature in their XCode, and the App Sandboxing should also be enabled to pass the binary validation.

Unfortunately this was announced in the last year, and which means any application related with filesystem, system commands won't be uploaded to Mac App Store...

Click here to download "GoTray"